List of AV Testing Labs

AV-Comparatives conducts a variety of malware protection tests, to ensure that all aspects of protection are covered. We test against various types of malicious programs, not only viruses, and use large sample sizes in each test to ensure statistical relevance. We also run false-positive tests to ensure that an antivirus program does not simply detect any unknown program as malicious. Each test report contains details of the aim of the test, methods and sample sizes used. We also intend to publish an overview of our test results, with guidelines for readers to help them understand which tests may be most relevant to their own particular requirements.

We would suggest that readers should read the description of each test (by whichever testing institute) carefully and try to understand which aspects are being evaluated, and also bear in mind things like the type of malware being used in the test, the sample size, the used product versions, etc. This may help to explain why a product gets good marks on one test by a particular institute, but bad marks on a different test by the same institute; also why a product scores well in a test by one institute, but badly in an apparently similar test by a different institute.

Listed below are some noteworthy institutes that provide tests of security software. The list is not meant as an endorsement, but more as a reference to some alternative sites.

Independent Testing Organizations

There are several independent testing organizations in the world. They all do their best to provide consumers with an unbiased view and assessment of the efficacy of security solutions. The services provided are useful not only for the users of the software, but also for the manufacturers. Very often, independent testing labs are used by security software vendors as an additional, external quality assessment lab, in order to find bugs and see in which areas further improvements are needed. We advise users to have a look at a variety of tests (covering different aspects of products) and testing labs (to get a broad overview over time from various sources).

AV-Comparatives

AV-Comparatives (Austria) is an independent security software testing organization, which provides a multitude of top-quality and state-of-the-art tests, like the “Real-World Protection Test”, which evaluates the protection provided by the security software as a whole. The detailed test reports are provided for free to the public. AV-Comparatives aims to provide valuable and unbiased information to consumers (such as this list of further testing labs), as well as some free public services like the Android APK Analyser (AVC UnDroid).

AV-Test

AV-Test (Germany) is an independent supplier of services in the fields of IT security and antivirus research, focusing on the detection and analysis of the latest malicious software and its use in comprehensive testing of security products. AV-Test, led by Andreas Marx, provides a variety of tests on various platforms and releases them almost every month on their website. AV-Test focusses more on certifications than on detailed comparative tests.

MRG Effitas

MRG Effitas (UK) is an independent IT security research organisation, which focuses mainly on browser and online banking security tests, but provides from time to time also other various evaluations.

NSS Labs

NSS Labs (USA) is an independent information security research and advisory company. NSS Labs delivers various research and expert analysis to provide useful information needed to make good security decisions. NSS Labs is known mainly for their thoughtful testing of appliances and hardware security solutions.

SE Labs

SE Labs (UK) is an independent testing facility that evaluates personal and business technology, including hardware and software. SE Labs (previously known as DennisTechnologyLabs) is a testing lab lead by Simon Edwards (SE).

Virus Bulletin

Virus Bulletin (UK) focuses its efforts in two main areas: an annual conference and a bimonthly product certification (VB100). Virus Bulletin’s VB100 logo award is de facto a very simple product certification, as the criterion is proving whether a product can detect 100% of malware samples listed on the WildList without generating any false positives.

Other Certification Labs

Certification labs are not really intended to provide comparative test results. They exist to certify whether a product has reached a given standard. It does not usually tell the public how many trials/cycles a particular vendor’s product had to go through to reach the standard. Certifications are very useful for establishing whether a vendor is reliable (has enough money to be in the business and get certified) and if its product is able to reach a minimum standard (usually covering all files included in the WildList) at one point in time. For users, the given information is usually too limited to be useful, but for companies, certifications are very meaningful and necessary. A good paper on why product certification matters can be found here. The most well-known certification labs are ICSA Labs and West Coast Labs (both located in USA).

ICSA Labs

West Coast Labs

Some more information and documents about test methods, guidelines and best practices for tests can be found e.g. on the AMTSO website. All the above-mentioned testing labs follow the best industry practices, so users have some valuable sources for security software tests and reviews without having to rely on “recommendations” of sales staff, marketing departments or opinions of unknown “advisors” on forums. A further interesting article about some tests and how to interpret them can be found here.