AV-Test.org presented at the VirusBulletin conference an interesting paper about the current desolate state of the WildList and made suggestions on how to improve it. Already at the AV Testing Workshop in Rekjavik 2007 most of the technical staff of the AV vendors admitted that the WildList is well-accepted and loved because it is easy to pass tests based on the WildList and because it is good for the marketing (100% detection*). So you may ask, why – if it is easy to pass – some vendors fail at detecting all samples from the WildList? The reasons could be either errors by the testers or temporary bugs in the software, but more often and likely it is because a) more variables than just detecting all samples are needed to pass (e.g. no false positives in case of VB100), b) sometimes also very old threats that were on the wildlist 10 years ago (e.g. boot sector viruses) are still included, and probably also because not all vendors receive the WildCore collection and therefore are not tested under same circumstances. So, who wants to keep the WildList alive? Of course (beside marketing** peoples and certification bodies which get lot of money for quite easy to do [and for av vendors to pass paid] tests) all those vendors that know that their product would not score well in tests using larger test-sets.
Summary to keep in mind
For peoples which are too lazy to read the information on the website and the reports:
1) The products tested by AV-Comparatives are already a selection of very good scanners. E.g. is a minimum requirement a detection rate of at least 85%. There are some big vendors (and many small vendors) which do not reach this requirement and are therefore not included in our tests due that. Some (relativly unknown) products do not even detect 20% of the test-set.
2) STANDARD rating is a good rating. It means that a product provides a good detection rate also of malware which is not on the Wildlist. As long a product scores at least STANDARD and is able to pass regularly the tests of VirusBulletin or ICSA, you can feel pretty safe with them. ADVANCED and ADVANCED+ are higher ratings, depending on your surfing habits and needs, you may feel to be in the need of using a scanner with such a rating.
3) Do not look just at the percentages or placing orders. A product belonging to one category (STANDARD, ADVANCED, ADVANCED+) can be considered as good as the other products in the same category.
4) The detection rate of an Anti-Virus product is just one factor you have to consider when choosing an AV. Other important factors are e.g.: impact on system performance, support, compatibility, price, GUI, easy of management, other protection features offered by the product, etc. – in other words: do not base the decision based on detection results alone and do not let other peoples decide what is best for you; try the various anti-virus products by yourself on your PC by downloading an evaluation version.
5) Do not annoy or bash a vendor if it scores lower than you would have expected in one test (and see point 1 & 2). You can be sure that they will do their best to improve their product and that their first goal is to protect their customers from the malware which is submitted to them by their customers (which of course has higher priority than samples submitted by other sources). Only in case your product e.g. often failed to protect you or the support you needed did not help you, you may consider to change your AV. If you are happy with your actual product and feel comfortable with it, there is probably no need to change it. Remember that AV-Comparatives does not recommend any specific product to you to use, what you get is just data results, all the rest remains up to you.
6) Look at various tests, possibly from as many different (professional) testers you find, see how the AV’s perform in those in long terms and . Some other testing institutions are listed in our links sections.
7) There is no AV which offers 100% detection against all malware. So it may be good if you from time to time check your PC with some online scanners of other vendors than the one you have installed. It may find something that your AV missed to detect – even if in tests it scores e.g. 99,9%.
Old blog entries removed
During the blog migration some blog posts got lost or removed. We will try to recover at least the most interesting posts. If you are missing an interesting topic, please let us know.