This website uses cookies to ensure you get the best experience on our website.
Please note that by continuing to use this site you consent to the terms of our Privacy and Data Protection Policy.

Why independent testing of anti-virus software is important – a blogpost by AV-Comparatives

If you’ve ever considered doing your own tests of antivirus programs, you might be surprised to find that it’s much, much more difficult than you think.

Have you ever been curious as to how good your antivirus program is at protecting your computers and servers? How it scores in an real av-test? If you’ve ever considered doing your own tests of antivirus programs, you might be surprised to find that it’s much, much more difficult than you think.

Here’s why:

There are literally millions of cyberthreats out there, so testing only against a few will not have any statistical validity. You need a threat-set that represents all the different types of threats(viruses, worms, ransomware, 0-days, exploits, trojans, backdoors, botnets and so on), AND you need to be sure that these are genuinely malicious and life on the internet.

Even if you had thousands of relevant cyberthreats, how would you test them? Well, you could scan them with an AV program installed on your own computer, or use an online scanning service. However, neither method lets you test against a large number of threats with multiple AV programs simultaneously.

There’s another, very important disadvantage to both. The on-demand scanning feature of a modern antivirus program is just one of many ways it can protect your computer. If in real life you try to browse to a cyberthreat, your security program checks the URL to see if it’s on a blacklist; uses file reputation services; unpacks the file to see through any obfuscation techniques; employs heuristics and machine learning to detect new malware programs; and monitor its behaviour in real time when you execute it, to see if it does anything suspicious.

If you look at AV-Comparatives Real-World Protection Test, you can see a perfect example, testing all the techniques an anti-virus software offers.

Professional labs have the resources to use large amounts of relevant malware, and test all the security features of multiple antivirus programs simultaneously, with equal conditions for all the tested products. They can also investigate usability of the tested products, that is whether they create a lot of false alarms or slow your computer down in daily use
Some of the test labs even undergo strict audits by ISO, to prove that they are independent. This ensures that they understand the interests of security vendors whilst remaining impartial and unaffected by commercial lobbying organisations. Independent av-test labs also work with universities on scientific research, which means that their test methodologies are state of the art.

Now, are you sure you want to rely on the results of DIY testing to protect your computer? Or do you prefer to rely on the results of an independent testing lab?

Have a look at the latest consumer and enterprise IT-Security and Antivirus software tests of one of the most well-trusted antivirus-software lab worldwide:

latest AV-Comparatives Consumer Security Tests
Real-World Protection Test February-May 2020 –
Speed-Impact Test –
Malware Protection Test –
Mac Security Test & Review 2020 –

latest AV-Comparatives Enterprise Security Tests
Business Security Test March-April 2020 – Factsheet
Mac Security Test & Review 2020 –

Peter Stelzhammer, AV-Comparatives, CBDO and co-founder:
“Finding a good antivirus program or IT security suite is a big challenge. Testing it yourself against thousands of threats is nearly impossible. You need a trustworthy, unbiased source of reliable information. Independent AV-test labs like AV-Comparatives have years of experience, plus the facilities and test methods to deliver meaningful results you can trust.”