List of AV Testing Labs

AV-Comparatives conducts a variety of malware protection tests, to ensure that all aspects of protection are covered. We test against various types of malicious programs, not only viruses, and use large sample sizes in each test to ensure statistical relevance. We also run false-positive tests to ensure that an antivirus program does not simply detect any unknown program as malicious. Each test report contains details of the aim of the test, methods and sample sizes used. We also intend to publish an overview of our test results, with guidelines for readers to help them understand which tests may be most relevant to their own particular requirements.
We would suggest that readers should read the description of each test (by whichever testing institute) carefully and try to understand which aspects are being evaluated, and also bear in mind things like the type of malware being used in the test, the sample size, the used product versions, etc. This may help to explain why a product gets good marks on one test by a particular institute, but bad marks on a different test by the same institute;  also why a product scores well in a test by one institute, but badly in an apparently similar test by a different institute.
Listed below are some noteworthy institutes that provide tests of security software. The list is not meant as an endorsement, but more as a reference to some alternative sites.

Independent Testing Organizations
There are several independent testing organizations in the world. They all do their best to provide consumers with an unbiased view and assessment of the efficacy of security solutions. The services provided are useful not only for the users of the software, but also for the manufacturers. Very often, independent testing labs are used by security software vendors as an additional, external quality assessment lab, in order to find bugs and see in which areas further improvements are needed. We advise users to have a look at a variety of tests (covering different aspects of products) and testing labs (to get a broad overview over time from various sources).

AV-Comparatives

AV-Comparatives

AV-Comparatives (Austria) is an independent security software testing organization, which provides a multitude of top-quality and state-of-the-art tests, like the “Whole Product Real World Dynamic Protection Test”, which evaluates the protection provided by the security software as a whole. The detailed test reports are provided for free to the public. AV-Comparatives aims to provide valuable and unbiased information to consumers (such as this list of further testing labs), as well as some free public services like the Android APK Analyser (AVC UnDroid).

AV-Test

AV-Test

AV-Test (Germany) is an independent supplier of services in the fields of IT security and antivirus research, focusing on the detection and analysis of the latest malicious software and its use in comprehensive testing of security products. AV-Test, led by Andreas Marx, provides a variety of tests on various platforms and releases them almost every month for free on their website. Recently AV-Test has focused more on certifications than on detailed comparative tests.

Dennis Technology Labs

Dennis Technology Labs

Dennis Technology Labs (UK) is an independent testing facility that evaluates personal and business technology, including hardware and software. Dennis Technology Labs (aka DTL) is a small testing lab from Dennis Publishing, and provides interesting tests.

MRG Effitas

MRG Effitas

MRG Effitas (UK) is an independent IT security research organisation, which focuses mainly on browser and online banking security tests, but provides from time to time also other various evaluations.

NSS Labs

NSS Labs

NSS Labs (USA) is an independent information security research and advisory company. NSS Labs delivers various research and expert analysis to provide useful information needed to make good security decisions. NSS Labs is known mainly for their thoughtful testing of appliances and hardware security solutions.

PC Security Labs

PC Security Labs

PC Security Labs (China) is an independent Chinese testing agency providing various security software evaluations. PC Security Labs (aka PCSL) is currently mainly focussing on Mobile (Android) Security Software Tests.

Virus Bulletin

Virus Bulletin

Virus Bulletin (UK) focuses its efforts in three main areas: a monthly magazine, an annual conference and a bimonthly product certification (VB100). Virus Bulletin’s VB100 logo award of is de facto a product certification, as the criterion is proving whether a product can detect 100% of malware samples listed on the WildList without generating any false positives. The website only lists Pass/Fail for each tested product. Current up-to-date comparative reports including detailed results and comments are only available for a fee (~20 USD).

Certification Labs

Certification labs are not really intended to provide comparative test results. They exist to certify whether a product has reached a given standard. It does not usually tell the public how many trials/cycles a particular vendor’s product had to go through to reach the standard. Certifications are very useful for establishing whether a vendor is reliable (has enough money to be in the business and get certified) and if its product is able to reach a minimum standard (usually covering all files included in the WildList) at one point in time. For users, the given information is usually too limited to be useful, but for companies, certifications are very meaningful and necessary. A good paper on why product certification matters can be found here. The most well-known certification labs are ICSA Labs and West Coast Labs (both located in USA).

 

ICSA Labs		West Coast Labs

Some more information and documents about test methods, guidelines and best practices for tests can be found e.g. on the AMTSO website. All the above-mentioned testing labs follow the best industry practices, so users have some valuable sources for security software tests and reviews without having to rely on “recommendations” of sales staff, marketing departments or opinions of unknown “advisors” on forums. A further interesting article about some tests and how to interpret them can be found here.

Comparative Testing Labs	Real-World Protection Test	Number of test cases per month	File Detection Test	Number of test cases	Behavioral Test	Performance Test	Malware Removal Test	Included products
AV-Comparatives		~500 (~2000 per report)		~150000				~25
AV-Test		~50 (~100 per report)		~20000				~25
Dennis Technology Labs		~30 (100 per report)		N/A				~10
PC Security Labs		N/A		~20000				~25
VirusBulletin		N/A		~3000				~50

Data as of 10th of December 2013