Real-World Protection Test May 2018 – Factsheet
Real-World Protection Test on Consumer Products
|Test Period||May 2018|
|Number of Testcases||227|
|Online with cloud connectivity|
|False Alarm Test included|
Our Real-World Protection Test is currently the most comprehensive and complex test available, using a large number of test cases. Currently, we are running this test under Microsoft Windows 10 Pro RS3 64 Bit with up-to-date third-party software (such as Adobe Flash, Adobe Acrobat Reader, Java, etc.). Due to this, finding in-the-field working exploits and running malware is much more challenging than e.g. under an non-up-to-date system with unpatched/vulnerable third-party applications.
This fact sheet is a short overview of the Whole-Product Dynamic Real-World Protection Test results of May 2018. The detailed overall result consumer product reports (covering five months each) are released in July and December. Each of the overall result reports will also contain a false-alarm test and will contain the awards the products reached based on their overall scores during the respective five-month period. For more information about this Real-World Protection Test, please read the details and previous test reports available on https://www.av-comparatives.org
Over the year we evaluate several tens of thousands malicious URLs. Unfortunately, many of these have to be discarded for various reasons. We remove duplicates such as the same malware hosted on different domains or IP addresses, sites already tested, “grey” or non-malicious sites/files, and malware/sites disappearing during the test. Many malicious URLs carrying exploits were not able to compromise the chosen system/applications because of the patch level. This means that the vulnerabilities in the third-party applications on the system were already patched and the exploits could therefore not deliver their malicious payload. Users should be aware that by always keeping their system and third-party applications up-to-date/patched, they can dramatically decrease the risk posed by exploits.
The results are based on the test set of 227 live test cases (malicious URLs found in the field), consisting of working exploits (i.e. drive-by downloads) and URLs pointing directly to malware. Thus exactly the same infection vectors are used as a typical user would experience in everyday life. The test-cases used cover a wide range of current malicious sitesand provide insights into the protection given by the various products (using all their protection features) while surfing the web.
Every month (from February to June and from July to November) we update the charts on our website showing the protection rates of the various tested products over the various months. The interactive charts can be found on our website. The chart below shows only the protection scores for the month of MAY 2018 (227test cases). The results of the false-positives test are also shown in the monthly factsheets/graph below.
Note: During May testing, Panda’s protection rate fell below its usual standard. After the test, when we provided Panda with the relevant test data, they discovered a bug in their products caused by a recent update. This meant that some few malware samples were reported in the logs, but not blocked or warned about. Panda rapidly fixed the bug, and Panda products now protect fully against all their missed test cases.
We would like to point out that while some products may sometimes be able to reach 100% protection rates in a test, it does not mean that these products will always protect against all threats on the web. It just means that they were able to block 100% of the widespread malicious samples used in a test.
Copyright and Disclaimer
This publication is Copyright © 2018 by AV-Comparatives ®. Any use of the results, etc. in whole or in part, is ONLY permitted after the explicit written agreement of the management board of AV-Comparatives prior to any publication. AV-Comparatives and its testers cannot be held liable for any damage or loss, which might occur as result of, or in connection with, the use of the information provided in this paper. We take every possible care to ensure the correctness of the basic data, but a liability for the correctness of the test results cannot be taken by any representative of AV-Comparatives. We do not give any guarantee of the correctness, completeness, or suitability for a specific purpose of any of the information/content provided at any given time. No one else involved in creating, producing or delivering test results shall be liable for any indirect, special or consequential damage, or loss of profits, arising out of, or related to, the use or inability to use, the services provided by the website, test documents or any related data.
For more information about AV-Comparatives and the testing methodologies, please visit our website.