Retrospective / Proactive Test May 2010

Tested Products

• Avast Free Antivirus 5.0
• AVG Anti-Virus 9.0
• Avira Antivirus Pro 9.0
• Bitdefender Antivirus+ 2010
• eScan Anti-Virus 10.0
• ESET NOD32 Antivirus 4.0
• F-Secure Anti-Virus 2010
• G DATA AntiVirus 2010
• K7 Total Security 10.0
• Kaspersky Anti-Virus 2010
• Kingsoft Antivirus 2010
• McAfee AntiVirus Plus 2010
• Microsoft Security Essentials 1.0
• Norman Antivirus & Anti-Spyware 7.30
• Panda Antivirus Pro 2010
• PC Tools Spyware Doctor with AV 7.0
• Sophos Anti-Virus 9.0
• Symantec Norton Anti-Virus 2010
• Trend Micro Antivirus+ 2010
• Trustport Antivirus 2010

Test Procedure

Anti-Virus products often claim to have high proactive detection capabilities – far higher than those reached in this test. This is not just a self-promotional statement; it is possible that products reach the stated percentages, but this depends on the duration of the test-period, the size of the sample set and the used samples. The data shows how good the proactive detection capabilities of the scanners were in detecting new threats. Users should not be afraid if products have, in a retrospective test, low percentages. If the anti-virus software is always kept up-to-date, it will be able to detect more samples. For understanding how the detection rates of the Anti-Virus products look with updated signatures and programs, have a look at our regular on-demand detection tests. Only the on-demand detection capability was tested. Some products may be had the ability to detect some samples e.g. on-execution or by other monitoring tools, like behaviour-blocker, etc. Those kinds of additional protection technologies are considered by AV-Comparatives in e.g. dynamic tests.

This test report is the second part of the February 2010 test. The report is delivered begin of June due the high-required work, deeper analysis and preparation of the retrospective test-set. Many new viruses and other types of malware appear every day, this is why it’s important that Anti-Virus products not only provide new updates, as often and as fast as possible, but also that they are able to detect such threats in advance (also without executing them) with generic and/or heuristic techniques. Even if nowadays most Anti-Virus products provide daily, hourly or cloud updates, without heuristic/generic methods there is always a time-frame where the user is not reliably protected.

The products used the same updates and signatures they had the 10^th February, and the same highest detection settings were used as in February. This test shows the proactive detection capabilities that the products had at that time. We used new malware appeared between the 11^th and 18^th February 2010. The following 20 products were tested:

Testcases

We tried to include in the test-set only prevalent real-world malware that has not been seen before the 10^th February 2010 by consulting telemetry / cloud data collected and shared within the AV industry. Consulting that data was quite interesting for us, as it showed that, while some vendors had seen some malware already many months or even years ago, the same malware hashes appeared in some other vendors clouds only recently.

Ranking System

The awards are given by the testers after consulting a number of statistical methods, including hierarchical clustering. We based our decisions on the following scheme:

	Proactive Protection Rates
	Under 50%	3	2	1
None - Few FPs	TESTED	STANDARD	ADVANCED	ADVANCED+
Many FPs	TESTED	TESTED	STANDARD	ADVANCED
Very many FPs	TESTED	TESTED	TESTED	STANDARD
Crazy many FPs	TESTED	TESTED	TESTED	TESTED

Test Results

The results show the proactive (generic/heuristic) on-demand detection capabilities of the scan engines against new malware. This Test is performed on-demand, it is NOT an on-execution/behavioral test. The percentages are rounded to the nearest whole number. Do not take the results as an absolute assessment of quality – they just give an idea of who detected more, and who less, in this specific test. To know how these anti-virus products perform with updated signatures, please have a look at our on-demand tests of February and August. Readers should look at the results and build an opinion based on their needs. All the tested products are already selected from a group of very good scanners and if used correctly and kept up-to-date, users can feel safe with any of them.

False Positive (False Alarm) Test Result

To better evaluate the quality of the detection capabilities, the false alarm rate has to be taken into account too. A false alarm (or false positive) is when an Anti-Virus product flags an innocent file to be infected when it is not. False alarms can sometimes cause as much troubles like a real infection.

The false-alarm test results were already included in the February test report. For details, please read the report, False Alarm Test February 2010.

1.	eScan	1	very few FPs (0-3)
2.	F-Secure	2
3.	Bitdefender, ESET, Microsoft	3
4.	Sophos	4	few FPs (4-15)
5.	G DATA, Kaspersky	5
6.	PC Tools	8
7.	Trustport	9
8.	AVG	10
9.	Avast, Avira, Symantec	11
10.	Trend Micro	38	many FPs (over 15)
11.	Panda	47
12.	McAfee	61
13.	Norman	64
14.	Kingsoft	67
15.	K7	193

Summary Result

The results show the proactive protection capabilities of the various products against new malware. The percentages are rounded to the nearest whole number.

Below you can see the proactive protection results over our set of new and prevalent malware files/families appeared in-the-field (27,271 malware samples):

	Blocked	Compromised	Proactive / Protection Rate	False Alarms	Cluster
TrustPort	17181	10090	63%	few	1
G DATA	16635	10636	61%	few	1
Microsoft	16090	11181	59%	very few	1
Kaspersky	16090	11181	59%	few	1
AVIRA	14454	12817	53%	few	1
ESET NOD32, F-Secure	14181	13090	52%	very few	1
BitDefender, eScan	13636	13636	50%	very few	1
Panda	17181	10090	63%	many	2
K7	13636	13636	50%	many	2
Symantec	11727	15544	43%	few	2
AVG	9272	17999	34%	few	2
Sophos	8727	18544	32%	few	2
Avast	7909	19362	29%	few	2
McAfee	10363	16908	38%	many	3
Norman	7363	19908	27%	many	3
Trend Micro	7090	20181	26%	many	3
PC Tools	4636	22635	17%	few	3
Kingsoft	3000	24271	11%	many	–

Notes

To avoid some frequent questions, below are some notes about the used settings (scan of all files etc. is always enabled) of some products, whereas highest settings were not used on vendors request:

• F-Secure, Sophos: asked to get tested and awarded based on their default settings (i.e. without using their advanced heuristics / suspicious detections setting).
• AVG, AVIRA: asked to do not enable/consider the informational warnings of packers as detections.

Copyright and Disclaimer

This publication is Copyright © 2010 by AV-Comparatives ®. Any use of the results, etc. in whole or in part, is ONLY permitted after the explicit written agreement of the management board of AV-Comparatives prior to any publication. AV-Comparatives and its testers cannot be held liable for any damage or loss, which might occur as result of, or in connection with, the use of the information provided in this paper. We take every possible care to ensure the correctness of the basic data, but a liability for the correctness of the test results cannot be taken by any representative of AV-Comparatives. We do not give any guarantee of the correctness, completeness, or suitability for a specific purpose of any of the information/content provided at any given time. No one else involved in creating, producing or delivering test results shall be liable for any indirect, special or consequential damage, or loss of profits, arising out of, or related to, the use or inability to use, the services provided by the website, test documents or any related data.

For more information about AV-Comparatives and the testing methodologies, please visit our website.

AV-Comparatives
(June 2010)