The aim of the user-experience review is to give readers an idea of what each tested product is like to use in everyday situations. For each of the tested products, we have looked at the following points (where applicable).
About the program
To start off with, we state whether the program is free or has to be paid for. We don’t list individual protection components (e.g. signatures, heuristics, behavioural protection), for the following reasons. Our protection tests verify how well each program protects the system, whereby it is not important which component(s) are involved. It is not the number of features that is important, but how effectively they work. Also, different vendors may have different names for individual functions, or combine multiple types of functionality under one name. This could make it misleading to compare products using the vendors’ component names. For readers’ convenience, we do note any non-malware-related features, such as parental controls or spam filtering. With the exception of a replacement firewall (see below), we do not check the functionality of these additional features.
We note any options available, whether you have to make any decisions, and any other points of interest, such as introductory wizards that explain the program’s features. We suggest that there should be a simple installation option for non-expert users. If at any stage the user has to make a decision in order to proceed, the options should be explained simply and clearly.
System Tray icon
Here we state what functionality is available from the program’s System Tray icon. This can be a convenient way of accessing commonly-used functions, such as scans and updates. A System Tray icon is a standard feature for modern security programs for consumers. We regard it as a very useful means of showing that the program is running. However, we note that by default, Windows 10 hides the System Tray icons of third-party programs, so many non-expert users will probably not see the icon for a non-Microsoft AV app.
Security status alert
Here, we disable the program’s real-time protection, and check to see what alerts are shown in the program window or elsewhere. We also look for a quick and easy means of reactivating the protection. An effective status display in the main program window, which shows a clear warning if protection is disabled, is a very standard feature, as is a “Fix-All” button/link with which the user can easily re-enable protection if it is not active. We regard both of these as very important, especially for non-expert users. We suggest that additional pop-up alerts, which the user would see even if the program window were not open, are a desirable bonus.
Malware detection alert
We check what sort of alert each program shows when malware is encountered. To do this, we try to copy some malware samples from a network share to the Windows Desktop of our test PC. If the AV product does not detect the copied malware, we then execute one of the samples (by this stage at the latest, all the tested programs will detect the malware samples used).
At whichever point the malware is detected, we look to see what sort of alert is shown, if the user has to take any action, and how long the alert is shown for. If the message box provides a link to more details, we click on this to see what information is provided. We also note whether multiple alerts are shown when multiple malicious files are detected at the same time.
We regard it as ideal if the malware is deleted or quarantined automatically, without the user having to make a decision on what to do with it. We would definitely recommend that any alert box should NOT include an option to instantly whitelist the file (i.e. allow it to be executed there and then). A much safer option is to quarantine the file, after which power users could go into the program’s settings to whitelist and restore it if they wanted.
We suggest that persistent alerts, which are displayed until the user closes them, are ideal, as they ensure the user has time to read them. If a separate alert box is shown for every malicious file discovered, it can be a nuisance to have to close them all when multiple detections are made at once. We would say that a single alert box that lets you browse through detections, but can be closed with a single click, is optimal.
Malware detection scenarios
As part of our review, we check to see how each AV program handles malicious programs – at which stage they are detected, what action is taken, and what alerts are shown – in four different scenarios. These are: execution; copying from a USB drive to the system; copying from a shared network folder to the system; on-demand scan of a USB drive via Windows Explorer’s right-click menu. For all of these, we use the same set of files, made up as follows. We take 5 highly prevalent malware samples, and 5 clean files (current installers for popular Windows programs). We then copy all 10 files into a sub-folder, to see if these are handled differently by the AV program from those in the root directory. The entire set of 20 files is then copied to a USB flash drive or network share, as applicable.
The USB copy and LAN copy checks allow us to see if the AV product has on-access protection (meaning the copied malware will be detected during or shortly after the copy process), or on-execution protection (meaning that malicious files can be copied to the system, but will be detected as soon as they are run). Regarding on-access versus on-execution protection, we suggest that for most people, the former is the better option. Whilst it may have a somewhat higher effect on system performance, it helps ensure that users cannot inadvertently pass on malware to other people, e.g. by copying it to a flash drive or network share, or sending it as an email attachment. For the execution check, we disable any automatic USB-scanning function in the AV program, and connect the USB drive (ignoring any prompts to scan it). We then open the drive in Windows File Explorer, and attempt to run in turn each of the five malware samples on the root of the drive. We have Windows Task Manager running during this check, so that we can observe whether any of the malicious programs is able to start a process. We note that some security programs with very sensitive on-access protection will delete the malware before it can be executed – an ideal action, which renders an actual execution check redundant. In our USB copy check, we attempt to copy the entire set of files from a USB flash drive to the Windows Desktop. Again, we disable/ignore any attempts by the program to scan the USB drive, which we connect to the system and open in File Explorer. To simulate the realistic action/speed of a non-expert user, we allow 10 seconds between opening the drive and starting the copy process, which we perform with Explorer itself. We then note the following: if the malware is deleted from the USB drive, and at which stage if so; if it is possible to copy any of the (remaining) malware to the Desktop; and if the latter is the case, whether the malicious files are later deleted from the Desktop by the AV program.
For the LAN copy check, we follow the same procedure as for USB copy, except that the files are on a writeable network share rather than a flash drive. In many cases, the results – in terms of whether the malicious files can be copied, and how they are then handled if so – are identical to the USB copy check. We therefore only report on this check for programs that either the handle the malware copy differently, or also delete the source malicious files in the shared folder.
In our on-demand scan test, we again disable/ignore any attempts by the program to scan the USB drive, which we connect to the system. Without opening the drive itself, we use the AV program’s entry in Windows Explorer’s right-click menu to run a scan of the drive. We note how the results are displayed by the security program at the end of the scan, whether any further action is needed by the user, and how easy it is to take this where applicable. We also check to see whether all the malware samples have been deleted from the USB drive.
Here we look at the different types of on-demand scan provided by each program, how to access and configure them, set scan exclusions, schedule scans, and what options are provided for PUA detection.
In the program’s quarantine function, we look to see what information it provides about the detection location/time and the malware itself, and what options are available for processing it, e.g. delete, restore, etc. .
For users who do not share their computer with anyone, this section is not relevant. However, if you share a computer, e.g. with your family at home, or colleagues in a small business, you might want to read it. Here we check, if it is possible to prevent other users of the computer from disabling the security program’s protection features or uninstall it altogether. There are two ways of doing this. Firstly, access can be limited using Windows User Accounts: users with Administrator Accounts can change settings and thus disable protection, whereas those with Standard User Accounts can’t. Alternatively, a program can provide password protection, so that any user – regardless of account type – must enter a password to change settings. Some programs provide both methods, which we regard as ideal. When testing access control, we try to find all possible means of disabling protection, to ensure that any restrictions apply to all of them.
In this section, we take a quick look at whatever help features can be directly accessed from the program itself. Some vendors will have additional online resources, such as manuals and FAQ pages, that can be found by visiting their respective websites.
Here we note what information is provided in the program’s log function.
Some of the products in this year’s tests have a replacement firewall. That is to say, they include their own firewall, which is used in place of Windows Firewall. For these products, we perform a very simple functionality test, to check that basic functions of their replacement firewalls work as expected. In essence, this just verifies that network discovery, file sharing and incoming Remote Desktop access are allowed on private networks, but blocked on public ones.
For this check, we use a laptop PC with a wireless network adapter, running a clean installation of Windows 10 Professional. It is initially connected to a wireless network that is defined as Private in Windows’ network status settings. We share the Documents folder, with read and write permissions for “Everyone”, and enable Remote Desktop access. In the Windows settings, we turn on network discovery, file sharing, and incoming Remote Desktop access for Private networks, but turn them all off for Public networks. We then verify that all three forms of network access are working as expected, i.e. allowed for Private networks but blocked for Public ones. We then install the security product with default settings, and reboot the computer. If during installation the third-party firewall in the security product were to prompt us to define the current network as public or private, we would designate it as private at that point. We would also note and report this. After the reboot, we check to see if we can still ping the PC, open and edit a document in its shared folder, and gain Remote Desktop access. We would expect the third-party firewall to allow all these types of access. We then connect the laptop to a new, unknown wireless network, which Windows will automatically define as Public in its own settings. If the third-party firewall were to display its own network-status prompt, we would also choose the public/untrusted option here. Next, we attempt to ping the test laptop (using IPv4) from another computer on the same network, access its file share, and log in with Remote Desktop. We would expect the third-party firewall to block all these forms of access, as Windows Firewall would do.
In our opinion, a third-party replacement firewall in a security program should either adopt Windows’ network status and firewall settings automatically, or warn the user that they will need to configure it themselves. This would allow laptop users to e.g. share files when at home, but keep intruders out when using public networks.
We recognise that some users may like to use Windows Firewall – which is a known standard – rather than the third-party firewall in their security product. For such users, it is ideal if the security product’s own firewall can be cleanly disabled (i.e. permanently disabled, without security alerts being constantly shown), and Windows Firewall can be activated instead. We check to see if this is possible.
Other points of interest
Here we note anything we observe or find out about a product that we think is relevant. This may include privacy-related items, descriptions of the product on the vendor’s website, unusual places to find features, customisation options, prompts to install additional features, upselling, bugs, explanations of functions, and out-of-the-ordinary features and notifications.
Support for Windows 11
All the tests in the 2023 Consumer Main-Test Series were performed using Windows 10. We also used Windows 10 for the review functionality checks described in this section. However, all of the tested/reviewed products are fully compatible/supported with Windows 11.