This website uses cookies to ensure you get the best experience on our website.
Please note that by continuing to use this site you consent to the terms of our Privacy and Data Protection Policy.
Accept

List of AV Testing Labs

AV-Comparatives conducts a variety of tests, to ensure that all aspects of security solutions are covered. Each test report contains details of the aim of the test, methods and sample sizes used. We also intend to publish an overview of our test results, with guidelines for readers to help them understand which tests may be most relevant to their own particular requirements.

We would suggest that readers should read the description of each test (by whichever testing institute) carefully and try to understand which aspects are being evaluated, and also bear in mind things like the type of malware being used in the test, the sample size, the used product versions, etc. This may help to explain why a product gets good marks on one test by a particular institute, but bad marks on a different test by the same institute; also why a product scores well in a test by one institute, but badly in an apparently similar test by a different institute.

Independent Testing Organizations

There are several security-product testing organizations around the world. The services they provide are useful not only for the users of the software, but also for the manufacturers. Very often, independent testing labs are used by security software vendors as an additional, external quality assessment lab, in order to find bugs and see in which areas further improvements are needed. Below we have provided a (non-exhaustive) list of testing labs that we consider to be noteworthy. The list is not meant as an endorsement, but more as a reference to alternative sites.

AV-Comparatives

AV-Comparatives (Austria) is an independent security software testing organization, led by Andreas Clementi. It provides a multitude of tests for various platforms (such as Windows, Mac, Android) and scenarios. It is ISO certified for “Independent Tests of Anti-Virus Software”. It regularly conducts comparative tests of both enterprise endpoint protection products and consumer antivirus programs. AV-Comparatives also runs tests of other security-related products, such as phishing protection, parental control, anti-spam, anti-stalkerware, IoT products, and VPNs. In recent years, AV-Comparatives has put more emphasis on testing enterprise products and protection against advanced targeted attacks. Among the additional enterprise-focussed tests is the new Endpoint Prevention and Response Test. To accommodate the additional workload, AV-Comparatives has recently expanded its testing facilities with a new cloud-security validation lab located in Austin, Texas, USA.

AV-Test

AV-Test (Germany) is an independent supplier of services in the fields of IT security and antivirus research, led by Andreas Marx. It provides testing of security products, focusing on the detection and analysis of the latest malicious software. AV-Test provides a variety of tests on various platforms (predominantly consumer products) and provides bi-monthly reports of these on its website. Each test includes protection, performance and usability components. AV-Test also runs tests of Internet-of-Things devices, and provides a data feed service. The lab was acquired by the Swiss IT Security Group in 2021.

MRG Effitas

MRG Effitas (UK) is an IT-security research organisation, which focuses on comparative tests of endpoint security, online banking security and Android security products. It also provides other security-related services, such as commissioned case studies and malware feeds.

SE Labs

SE Labs (UK) is a testing facility that evaluates personal and business technology, including hardware and software. It is led by Simon Edwards (SE), and was previously known as Dennis Technology Labs. Its tests are run quarterly and cover enterprise, small business, and consumer products. Each test usually includes around eight products, from well-known manufacturers.

Virus Bulletin

Virus Bulletin (UK) focuses its efforts in three main areas: an annual conference, a news bulletin, and a bimonthly product certification test (VB100). To receive the VB100 award, a tested product simply needs to detect 99.5% of well-known and prevalent WildList malware samples, with no more than 1 false positive for every 10,000 clean files. The test methodology is limited to on-access and on-demand scans, without execution, and so does not test some protection features such as URL blockers or behavioural detection. Tests include an unusually high number of products, although many of these are relatively unknown, and many of the bigger AV vendors are not included.

Other Certification and Testing Labs

Certification labs are not really intended to provide comparative test results. They exist to certify whether a product has reached a given standard, usually involving detection of well-known malicious files (e.g. from the WildList). Whilst this can be a valuable service to some companies, users should be aware of the limitations of certification testing. The tests do not provide comparative results for different products, and so may be of very limited use when deciding which security program to buy. Certification reports do not usually state how many times a product had to be tested before reaching the required standard. Some certifications may only prove that a vendor has the financial resources to pay for the process, and that its product is able to reach a minimum standard at one point in time. A paper with useful information about certification testing can be found here.

Two US-based labs, West Coast Labs and NSS Labs, have both ceased operating in recent years. However, another American lab, ICSA Labs, continues to provide certification for various security-related products.

Another tech organisation which is noteworthy and emerged in the last few years is MITRE Engenuity (USA). The foundation supports several U.S. government agencies in the fields of IT security, 5G, health, and transportation. In its cybersecurity work, MITRE Engenuity uses the ATT&CK® framework, which describes in detail the different stages of advanced persistent threats (APTs). It provides evaluations of enterprise solutions against such attacks. These may be very valuable to participating security vendors and some IT-security specialists. However, their reports do not present the data so as to allow easy comparison of the effectiveness of different products.

The above-mentioned testing labs follow the best industry practices, so users have some valuable sources for security software tests and reviews without having to rely on “recommendations” of sales staff, marketing departments or opinions of unknown “advisors” on forums. A further interesting article about some tests and how to interpret them can be found here.