Potentially-Unwanted-Application Test December 2010
incl. Adware, Spyware and Rogue Software
|Test Period||December 2010|
|Number of Testcases||82036|
|Online with cloud connectivity|
|False Alarm Test included|
The amount of adware, spyware and especially of fraudulent software circulating on the Internet has increased a lot over the past few years. Such applications are not typical malware and their classification is sometimes not an easy task; they are usually described using the term “potentially unwanted application” (PUA). Under some circumstances, certain “potentially unwanted applications” are accepted/wanted in some countries, depending on cultural background or legal system, due to which legal disputes sometimes come up as to whether a program can be considered to be malware or not. The term “potentially unwanted” covers this grey area. Usually our malware test sets do not include this kind of threat, but users may want to know how well their Anti-Virus program detects potentially unwanted software.
- Kingsoft Antivirus 2011
- PC Tools Spyware Doctor with AV 8.0
The PUA (Potentially Unwanted Applications) test set used for this test contains 82036 samples. It includes only program executable files and covers mainly rogue software (e.g. fake antivirus and other misleading or unwanted/unsafe applications), adware (e.g. Virtumonde, browser hijackers) and spyware (e.g. keyloggers) gathered or re-seen between July 2010 and November 2010. Some products may classify some PUAs as Trojans, while some other products may not want to add detection for some potentially unwanted applications as their company policy or due possible legal reasons. The test-set is so small/limited because we tried to remove in advance certain software types that could be disputed or do not belong to this testset (e.g. CasinoClients, Games, Toolbars, Utilities, etc.). Vendors of which products score high in this test may only exclude a detection when the legal implications become serious, otherwise they continue to detect it if it warrants detection from a technical perspective. Especially if products have special checkboxes that the user has to activate manually to get informed about “potentially” unwanted or unsafe stuff, such stuff should be detected, otherwise it should be in the normal detections or not reported at all. If some vendors are more reluctant to inform the user about potentially unwanted files, it may be observed in those test results. If some vendors simply add detection for everything (including for such files which are in the grey area), it may be possible to see in the false alarm tests1 that can be found on our website.
The PUA sets were frozen on the 6th November 2010. The AV products were last updated on the 1st December 2010. We tested all the products with highest settings (except F-Secure and Sophos on their own request).
Graph of missed samples (lower is better)
Detection rates for “potentially unwanted software”:
|5.||G DATA, PC Tools||99.3%|
|6.||Bitdefender, eScan, F-Secure, McAfee||98.7%|
Users that would like to be informed about potentially unwanted software of which they would be concerned about having on their PC without their knowledge may prefer the higher scoring products, while users which prefer that their AV products does not inform them about such potentially unwanted software may prefer products which are more conservative about what they report.
Award levels reached in this PUA Test
AV-Comparatives provides a 4-level ranking system: Tested, STANDARD, ADVANCED and ADVANCED+. The groups have been defined by consulting/using the hierarchical clustering method.
The Awards are based only on detection rates for potentially unwanted/unsafe programs. To see detection rates for malware like Trojans, backdoors, viruses, etc., as well as for false alarm rates of the products, please refer to the other test reports available on our website.
Due to the challenge of testing PUA in that every AV vendor undergoes its own criteria to define the PUA bar, we are not planning to conduct a separate PUA test next year.
AVG and Sophos decided to not get included in this report and to renounce to get awarded. As those products are included in our yearly public test-series, they are listed even if those vendors decided to do not get included.
Copyright and Disclaimer
This publication is Copyright © 2010 by AV-Comparatives ®. Any use of the results, etc. in whole or in part, is ONLY permitted after the explicit written agreement of the management board of AV-Comparatives prior to any publication. AV-Comparatives and its testers cannot be held liable for any damage or loss, which might occur as result of, or in connection with, the use of the information provided in this paper. We take every possible care to ensure the correctness of the basic data, but a liability for the correctness of the test results cannot be taken by any representative of AV-Comparatives. We do not give any guarantee of the correctness, completeness, or suitability for a specific purpose of any of the information/content provided at any given time. No one else involved in creating, producing or delivering test results shall be liable for any indirect, special or consequential damage, or loss of profits, arising out of, or related to, the use or inability to use, the services provided by the website, test documents or any related data.
For more information about AV-Comparatives and the testing methodologies, please visit our website.