Malware Removal Test Methodology

Operating System

Microsoft Windows; the exact details of the version used are noted in the individual test reports.

Aim of the test

This test aims to determine how effectively and easily different antivirus products remove malware that has already infected a system.

Target Audience

Anyone who is likely to try removing a malware infection from a PC will benefit from reading the results of this test. It will be of particular interest to IT staff and computer technicians who carry out malware removal as part of their jobs, but will also be applicable to computer users who undertake their own computer maintenance. It is assumed that the user does not have any specialist technical knowledge (other than the ability to boot Windows into Safe Mode), and will rely on intuitive use of the antivirus software concerned to remove the malware from the system.

Definition of the threat

Malware can be defined as computer programs that have a clear and significant malicious purpose. This excludes some programs such as commercial keyloggers which can be used legitimately, e.g. in computer training, and also excludes “potentially unwanted programs” such as some browser toolbars, which are irritating but cannot be classed as malicious.

Scope of the test

The test is solely concerned with each product’s ability to remove malware from an already infected system. It does not measure detection or protection.

Test Setup

The operating system is installed on each test PC and fully patched.

Settings

Products are tested with default settings. In the event that a scan fails to remove the malware, settings may then be changed for subsequent scans in individual cases. This is noted in the test report where relevant.

Sources and numbers of test cases

The samples have been selected according to the following criteria: All anti-virus products must be able to detect the malware dropper used when inactive. The sample must have been prevalent (according to metadata) and/or seen in the field on at least two PC’s of our local customers in the last 6 months. The malware must be non-destructive (in other words, it should be possible for an anti-virus product to repair/clean the system without the need for replacing Windows system files etc.). It must also show common malware behavior under the operating system used, in order to represent also behaviors observed by many other malware samples. Around one dozen randomly picked malware samples are taken from the pool of samples matching the above criteria.

Test procedure

Thorough malware analysis is done for each sample, to see exactly what changes are made. The physical machine is infected with one threat, rebooted, and a check done to ensure that threat is fully running. The anti-virus product is installed and updated. If this is not possible, the PC is rebooted into Safe Mode; if Safe Mode is not possible and in event that a rescue disk for the relevant AV-Product is available, this is used to run a full system scan before installing. A thorough/full system scan is run and instructions given by the anti-virus product are followed to remove the malware, as a typical home-user would do. The machine is rebooted, and manual inspection/analysis of the system is made, to check for malware removal and remnants.

Ratings

We allowed certain negligible/unimportant traces to be left behind, mainly because a perfect score can’t be reached due to the behaviour/system-modifications made by some of the malware samples used. The “removal of malware” and “removal of remnants” are combined into one dimension and we took into consideration also the convenience. The ratings are given as follows, whereby A is the highest mark and D the lowest:

1. Removal of malware/traces

• Malware removed, only negligible traces left (A)
• Malware removed, but some executable files, MBR and/or registry changes (e.g. loading points, etc.) remaining (B)
• Malware removed, but annoying or potentially dangerous problems (e.g. error messages, compromised hosts file, disabled task manager, disabled folder options, disabled registry editor, detection loop, etc.) remaining (C)
• Only the malware dropper has been neutralized and/or most other dropped malicious files/changes were not removed, or system is no longer normally usable; dropped malicious files are still on the system; removal failed (D)

1. Convenience:

• Removal could be done in normal mode (A)
• Removal requires booting in Safe Mode or other built-in utilities and manual actions (B)
• Removal requires Rescue Disk (C)
• Removal or install requires contacting support or similar; removal failed (D)

False positives

“Aggressive cleaning” – i.e. a program deleting more than it should – is regarded as a false positive in this test.

Summary

Related Posts