How does the Real-World Protection Test differ from “traditional” static on-demand detection tests?
The “Real-World Protection Test” is a joint project of AV-Comparatives and the University of Innsbruck’s Faculty of Computer Science and Quality Engineering. It mimics a user surfing the Internet and opening internet links in email. This web attack angle allows us to test all of the protection features in a product. As well as signatures or heuristic file scanning (locally or in the cloud), any defence mechanism developed by the vendor, such as web filters and behaviour blockers, is tested.
The Real-World Protection Test thus assesses the most important aspect of a security program, i.e. whether it will prevent malware from compromising the system. The test allows all available protection features to come into play, and products are able to download updates before each test case. Thus, it shows how well each product protects the system under optimal conditions. Static online multi-scanner services have their uses, but they cannot replicate the protection features of full security products. Firstly, there are limitations to the online scanning process. Hence its results may not even be identical to those of on-demand scans performed by full products. Secondly, online scanners do not employ all the features used by full security products, such as behavioural detection. It is very likely that in real life, a full security program would be able to protect against a malware sample not detected by an online multi-scanner service.
