Introduction
This is the first half-year report of our Business Main-Test Series of 2023, containing the results of the Business Real-World Protection Test (March-June), Business Malware Protection Test (March), Business Performance Test (June), as well as the Product Reviews.
Please note that the results of the Business Main-Test Series cannot be compared with the results of the Consumer Main-Test Series, as the tests are done at different times, with different sets, different settings, etc.
AV security software caters to businesses of all sizes and types. However, the suitability of a particular software solution varies depending on the scale of operations. Before selecting an appropriate software, it is crucial to understand the business environment in which it will be deployed, enabling informed decision-making.
Let’s focus on the smaller end of the market. These environments typically emerge from micro businesses where consumer-grade AV products might have sufficed. However, as the business expands beyond a few machines, the importance of AV management becomes evident. This is particularly critical when considering the potential business and reputational damage that can result from a significant, uncontained malware outbreak.
In the smaller SME segment, on-site IT managers or professionals are often absent. Instead, the responsibility of “computer maintenance” falls on an interested non-expert, usually a senior partner with other primary roles in the business. This model is commonly found in retail, accountancy, and legal professions. In such cases, it is essential to have a centralized overview of all computing assets and instant clarity regarding the protection status in a straightforward manner. If necessary, remediation can involve temporarily disconnecting a machine, transferring the user to a spare device, and waiting for an IT professional to arrive on-site for cleanup and integrity checks. While users may be kept informed about the status, managing the platform remains the responsibility of one or a few senior individuals within the organization. These decisions are often driven by the company’s overriding need for data confidentiality.
In larger organizations, having dedicated on-site IT specialists, including network security professionals, is expected. The Chief Technology Officer (CTO) in such organizations seeks straightforward, real-time statistics and a management overview that allows for detailed analysis of data to address emerging issues. Software installation engineers play a vital role in ensuring correct and appropriate deployment of the AV package on new machines. It is crucial to monitor and detect when machines become disconnected from the network to prevent the presence of rogue and unprotected devices on the LAN. Additionally, a help desk role serves as the first line of defense, responsible for monitoring and tracking malware activity and taking appropriate actions, such as initiating a wipe-and-restart process on compromised computers.
In this larger organizational structure with multiple layers, remediation and tracking become key tasks. Identifying a malware infection is only the beginning; effectively handling and tracing the infection back to its original point are essential functions in larger organizations. If weaknesses in network security and operational procedures cannot be clearly identified, the risk of future breaches remains high. To fulfil this role, comprehensive analysis and forensic tools are required, with a focus on understanding the timeline of an attack or infection originating from a compromised computer. However, presenting this information coherently is challenging, as it involves processing vast amounts of data and employing tools to filter, categorize, and highlight unfolding issues, often in real time.
Due to these significant differences, it is crucial to accurately assess the organization’s needs and risk profile to identify the appropriate security tool. Under-specifying can lead to breaches that are difficult to manage, while over-specifying results in a system so complex that it becomes challenging to deploy, use, and maintain effectively. The business becomes vulnerable to attacks due to the confusion and lack of compliance resulting from an overly complex system.
One crucial consideration for businesses is choosing between a cloud-based or server-based console. Cloud-based consoles are quick to set up and generally do not require additional configuration of client devices. On the other hand, server-based consoles require more initial setup work, including configuring clients and the company firewall. However, they provide the advantage of having the entire setup on the company’s premises and under the direct control of the administrator. For smaller businesses with limited IT staff, cloud-based consoles may be a more accessible option. It’s important to note that manufacturers often offer both cloud-based and server-based options for managing their products. The console types mentioned here refer specifically to the product used in our tests. It is recommended to consult the respective vendor to explore other console types that may be available.
Avast and VIPRE offer user-friendly cloud consoles that are well-suited for smaller businesses without dedicated IT staff. These solutions are also suitable for larger companies, allowing for business growth. G Data and K7 utilize server-based consoles that are straightforward for experienced Windows professionals and can be used by SMEs and beyond.
For businesses of the same size seeking cloud-based management solutions, Bitdefender, ESET, Kaspersky, Microsoft, Sophos, and WatchGuard provide robust and comprehensive options. Cybereason and VMware may require a slightly steeper learning curve but are also suitable for this category of business.
At the larger end of the market, CISCO, CrowdStrike, Elastic, and Trellix offer exceptionally powerful tools. However, their suitability for your organization, both in its current state and future growth plans over the next five years, should be carefully planned. Seeking external expertise and consultancy is recommended during the planning and deployment stages, as these tools require significant training and ongoing support. Nonetheless, they offer capabilities that surpass those of smaller packages.
Tested Products
The following business products were tested under Microsoft Windows 10 64-bit:
In business environments, and with business products in general, it is usual for products to be configured by the system administrator, in accordance with vendor’s guidelines, and so we invited all vendors to configure their respective products.
Only a few vendors provide their products with optimal default settings which are ready to use, and did therefore not change any settings.
Please keep in mind that the results reached in the Enterprise Main-Test Series were only achieved by applying the respective product configurations described here. Any setting listed here as enabled might be disabled in your environment, and vice versa. This influences the protection rates, false alarm rates and system impact. The applied settings are used across all our Enterprise Tests over the year. That is to say, we do not allow a vendor to change settings depending on the test. Otherwise, vendors could e.g. configure their respective products for maximum protection in the protection tests (which would reduce performance and increase false alarms), and maximum speed in the performance tests (thus reducing protection and false alarms). Please not that some enterprise products have all their protection features disabled by default, so the admin has to configure the product to get any protection.
Below we have listed relevant deviations from default settings (i.e. setting changes applied by the vendors):
Bitdefender: “Sandbox Analyzer” (for Applications and Documents) enabled. “Analysis mode” set to “Monitoring”. “Scan SSL” enabled for HTTP and RDP. “HyperDetect” and “Device Control” disabled. “Update ring” changed to “Fast ring”. “Web Traffic Scan” and “Email Traffic Scan” enabled for Incoming emails (POP3). “Ransomware Mitigation” enabled. “Process memory Scan” for “On-Access scanning” enabled. All “AMSI Command-Line Scanner” settings enabled for “Fileless Attack Protection”.
CISCO: “On Execute File and Process Scan” set to Active; “Exploit Prevention: Script Control” set to “Block”; “TETRA Deep Scan File” disabled; “Exclusions” set to “Microsoft Windows Default”; Engines “ETHIS”, “ETHOS”, “SPERO” and “Step-Up” disabled. “MaxScanFileSize” increased to 500 MB.
CrowdStrike: everything enabled and set to maximum, i.e. “Extra Aggressive”. “On-demand Scans” and Uploading of “Unknown Detection-Related Executables” and “Unknown Executables” disabled.
Cybereason: “Anti-Malware” enabled; “Signatures mode” set to “Quarantine”; “Artificial intelligence” set to “Moderate”; “Fileless protection” enabled and set to “Prevent”; Update interval set to 1 minute.
Elastic: MalwareScore (“windows.advanced.malware.threshold”) set to “aggressive”, and Rollback-SelfHealing (“windows.advanced.alerts.rollback.self_healing.enabled”) enabled. “Credential hardening” enabled.
ESET: All “Real-Time & Machine Learning Protection” settings set to “Aggressive”.
G Data: “BEAST Behavior Monitoring” set to “Halt program and move to quarantine”. “BEST Automatic Whitelisting” deactivated. “G DATA WebProtection” add-on for Google Chrome installed and activated. “Malware Information Initiative” enabled.
Kaspersky: “Adaptive Anomaly Control” disabled; “Detect other software that can be used by criminals to damage your computer or personal data” enabled;
Microsoft: “CloudExtendedTimeOut” set to 55; “PuaMode” enabled.
Sophos: “Threat Graph creation”, “Web Control” and “Event logging” disabled.
Trellix: “Web Control” add-on for Google Chrome enabled. “Firewall” and “Exploit Prevention” disabled.
VIPRE: “IDS” enabled and set to “Block With Notify”. “Firewall” enabled.
VMware: policy set to “Advanced”.
Avast, K7, WatchGuard: default settings.
Information about additional third-party engines/signatures used by some of the products: CISCO, Cybereason, G Data and VIPRE use the Bitdefender engine (in addition to their own protection features). CISCO uses also the ClamAV engine. VMware uses the Avira engine (in addition to their own protection features). G Data’s OutbreakShield is based on Cyren.
The “ENS” version of Trellix in this test uses the erstwhile McAfee engine (now owned by Trellix), opposed to the “HX” version which uses the FireEye engine (McAfee Enterprise and FireEye were merged into Trellix in 2022).
We congratulate the vendors who are participating in the Business Main-Test Series for having their business products publicly tested by an independent lab, showing their commitment to improving their products, being transparent to their customers and having confidence in their product quality.
Test Procedure
The test series consists of three main parts:
The Real-World Protection Test mimics online malware attacks that a typical business user might encounter when surfing the Internet.
The Malware Protection Test considers a scenario in which the malware pre-exists on the disk or enters the test system via e.g. the local area network or removable device, rather than directly from the Internet.
In addition to each of the protection tests, a False-Positives Test is conducted, to check whether any products falsely identify legitimate software as harmful.
The Performance Test looks at the impact each product has on the system’s performance, i.e. how much it slows down normal use of the PC while performing certain tasks.
To complete the picture of each product’s key capabilities, there is a product description included in the report as well.
Some of the products in the test are clearly aimed at larger enterprises and organisations, while others are more applicable to smaller businesses. Please see each product’s review section for further details.
Kindly note that some of the included vendors provide more than one business product. In such cases, other products in the range may have a different type of management console (server-based as opposed to cloud-based, or vice-versa); they may also include additional features not included in the tested product, such as endpoint detection and response (EDR). Readers should not assume that the test results for one product in a vendor’s business range will necessarily be the same for another product from the same vendor.
Test Results
Real-World Protection Test (March-June)
The results below are based on a test set consisting of 526 test cases (such as malicious URLs), tested from the beginning of March 2023 till the end of June 2023.
|
Blocked |
User dependent |
Compromised |
PROTECTION RATE
[Blocked % + (User dependent %)/2]* |
False Alarms |
Kaspersky
|
526 |
– |
– |
100% |
0 |
Bitdefender |
526 |
– |
– |
100% |
2 |
VIPRE
|
525 |
– |
1 |
99.8% |
6 |
Avast |
525 |
– |
1 |
99.8% |
15 |
ESET, Microsoft |
524 |
– |
2 |
99.6% |
1 |
G Data |
524 |
– |
2 |
99.6% |
4 |
Elastic |
524 |
– |
2 |
99.6% |
6 |
K7 |
523 |
– |
3 |
99.4% |
2 |
Trellix |
523 |
– |
3 |
99.4% |
10 |
CrowdStrike
|
523 |
– |
3 |
99.4% |
30 |
Watchguard
|
520 |
– |
6 |
98.9%
|
23 |
CISCO
|
519 |
– |
7 |
98.7%
|
8 |
Sophos
|
515 |
1 |
10 |
98.0%
|
5 |
Cybereason |
506 |
– |
20 |
96.2% |
15 |
VMware |
503 |
– |
23 |
95.6% |
1 |
User-dependent cases are given half credit. For example, if a program blocks 80% by itself, and another 20% of cases are user-dependent, we give half credit for the 20%, i.e. 10%, so it gets 90% altogether.
Malware Protection Test (March)
The following chart shows the results of the Business Malware Protection Test:
False positive (false alarm) test with common business software
A false alarm test done with common business software was also performed. All tested products had zero false alarms on common business software.
|
Malware Protection Rate |
False Alarms on common business software |
Microsoft, Trellix |
99.9% |
0 |
Watchguard |
99.8% |
0 |
Avast, CrowdStrike, Elastic, VMware |
99.7% |
0 |
Cisco, Kaspersky |
99.6% |
0 |
G Data |
99.5% |
0 |
Bitdefender, ESET, VIPRE |
99.4% |
0 |
Cybereason |
98.9% |
0 |
Sophos |
98.8% |
0 |
K7 |
98.6% |
0 |
In order to better evaluate the products’ detection accuracy and file detection capabilities (ability to distinguish benign files from malicious files), we also performed a false alarm test on non-business software and uncommon files. Results are shown in the tables below; the false alarms found were promptly fixed by the respective vendors. However, organisations which often use uncommon or non-business software, or their own self-developed software, might like to consider these results. Products are required to have an FP rate on non-business files below the Remarkably High threshold in order to be approved. This is to ensure that tested products do not achieve higher protection scores by using settings that might cause excessive levels of false positives.
FP rate |
Number of FPs on
non-business software |
Very low |
0 – 5 |
Low |
6 – 15 |
Medium/Average |
16 – 35 |
High |
36 – 75 |
Very high |
76 – 125 |
Remarkably high |
> 125 |
|
FP rate on non-business software |
Bitdefender, ESET, G Data, Kaspersky, Trellix, VIPRE, VMware |
Very low |
– |
Low |
Avast, Microsoft |
Medium/Average |
K7 |
High |
Cisco, CrowdStrike, Sophos, WatchGuard |
Very high |
Cybereason, Elastic |
Remarkably high |
It should be noted that Cybereason and Elastic had Remarkably High levels of false positives on non-business files. Administrators should consider whether this might create problems in their respective organisations’ specific environments.
These specific test results show the impact on system performance that a security product has, compared to the other tested security products. The reported data just gives an indication and is not necessarily applicable in all circumstances, as too many factors can play an additional part. The testers defined the categories Slow, Mediocre, Fast and Very Fast by consulting statistical methods and taking into consideration what would be noticed from the user’s perspective, or compared to the impact of the other security products. If some products are faster/slower than others in a single subtest, this is reflected in the results.
Overview of single AV-C performance scores
Vendor |
File copying |
Archiving /
Unarchiving |
Installing /
Uninstalling
Applications |
Launching Applications |
Downloading Files |
Browsing Webites |
First Run |
Subsequent Run |
First Run |
Subsequent Run |
Avast |
|
|
|
|
|
|
|
|
Bitdefender |
|
|
|
|
|
|
|
|
CISCO |
|
|
|
|
|
|
|
|
CrowdStrike |
|
|
|
|
|
|
|
|
Cybereason |
|
|
|
|
|
|
|
|
Elastic |
|
|
|
|
|
|
|
|
ESET |
|
|
|
|
|
|
|
|
G Data |
|
|
|
|
|
|
|
|
K7 |
|
|
|
|
|
|
|
|
Kaspersky |
|
|
|
|
|
|
|
|
Microsoft |
|
|
|
|
|
|
|
|
Sophos |
|
|
|
|
|
|
|
|
Trellix |
|
|
|
|
|
|
|
|
VIPRE |
|
|
|
|
|
|
|
|
VMware |
|
|
|
|
|
|
|
|
WatchGuard |
|
|
|
|
|
|
|
|
Key |
|
Slow |
|
Medicore |
|
Fast |
|
Very fast |
PC Mark Tests
In order to provide an industry-recognized performance test, we used the PC Mark 10 Professional Edition testing suite (for more information, see https://benchmarks.ul.com). Users using PC Mark 10 benchmark should take care to minimize all external factors that could affect the testing suite, and strictly follow at least the suggestions documented inside the PC Mark manual, to get consistent and valid/useful results. Furthermore, the tests should be repeated several times to verify them. For more information about the various consumer scenarios tests included in PC Mark, please read the whitepaper on their website.
“No security software” is tested on a baseline system without any security software installed, which scores 100 points in the PC Mark 10 benchmark.
Baseline system: Intel Core i3 machine with 4GB RAM and SSD drive
PC Mark® is a registered trademark of Futuremark Corporation / UL.
Summarized results
Users should weight the various subtests according to their needs. We applied a scoring system to sum up the various results. Please note that for the File Copying and Launching Applications subtests, we noted separately the results for the first run and for subsequent runs. For the AV-C score, we took the rounded mean values of first and subsequent runs for File Copying, whilst for Launching Applications we considered only the subsequent runs. “Very fast” gets 15 points, “fast” gets 10 points, “mediocre” gets 5 points and “slow” gets 0 points. This leads to the following results:
| | AVC Score | PC Mark Score | Impact Score |
1. | WatchGuard | 90 | 97.2 | 2.8 |
2. | Avast | 90 | 97.0 | 3.0 |
3. | Bitdefender | 90 | 96.8 | 3.2 |
4. | K7 | 90 | 96.4 | 3.6 |
5. | ESET | 88 | 98.1 | 3.9 |
6. | Kaspersky | 85 | 97.8 | 7.2 |
7. | G DATA | 85 | 97.6 | 7.4 |
8. | VIPRE | 83 | 96.5 | 10.5 |
9. | Trellix | 83 | 96.1 | 10.9 |
10. | Cybereason | 80 | 97.3 | 12.7 |
11. | CrowdStrike | 78 | 95.2 | 16.8 |
12. | Microsoft | 75 | 96.9 | 18.1 |
13. | Elastic | 75 | 95.9 | 19.1 |
14. | VMware | 73 | 95.1 | 21.9 |
15. | Sophos | 65 | 92.6 | 32.4 |
16. | Cisco | 63 | 91.2 | 35.8 |
|
Product Reviews
Bellow, you will find product descriptions of the tested enterprise products. Please note that the product descriptions are based on information provided by vendors. For more detailed and current information, please visit the vendors’ websites.
Avast Ultimate Business Security includes a next-gen antivirus with online privacy tools and patch management automation software to help keep business devices, data, and applications updated and secure.
Key Features
Online Management Platform: Get real-time visibility of cyberthreats, comprehensive reporting, and administrative capabilities – right from your web browser. A cloud-based console lets you centrally manage your Avast Business security services and their subscriptions.
Next-gen Antivirus: Next-gen endpoint protection with File Shield, Web Shield, Mail Shield, real-time Behaviour Monitoring, and Cloud Sandbox help secure users’ devices against malware infections and zero-day threats.
Advanced Firewall: Monitor network traffic between your employees’ devices and the internet. Improve blocking of dangerous or superfluous data transmissions for better protection of your business against malicious data manipulation.
Ransomware Shield: Reinforce the protection of your sensitive data and other critical business documents against modification, deletion, or encryption by ransomware attacks. Choose which applications have permission to access your protected folders and block the rest.
Real Site: Real Site supports safer web browsing and banking by helping your employees avoid fake websites created to steal sensitive data such as usernames, passwords, and credit card details. It is designed to secure users against DNS (Domain Name System) hijacking.
Password Protection: Help safeguard your employees’ login information that is stored in web browsers from being stolen and misused. Password Protection is designed to prevent applications and malware from tampering with passwords that are saved in Google Chrome, Mozilla Firefox, Microsoft Edge, and Avast Secure Browser browsers.
VPN: Built-in personal VPN with no data limits encrypts your data traffic over the internet to help protect your employees’ data, making them also private when using public Wi-Fi networks, such as those in cafes or the airport.
USB Protection: Prevent employees from using unauthorized removable storage devices, including flash drives, external drives, and memory cards to avoid data theft, data loss, and malware infections.
Patch Management: Automatically fix vulnerabilities in Windows and third-party applications that are susceptible to cyberattacks by remotely patching devices, no matter where they are. Patch Management helps you distribute tested patches to hundreds of devices in minutes, with minimal impact on your network.
GravityZone Business Security Premium is designed to protect small to medium organizations, covering any number of file servers, desktops, laptops, physical or virtual machines. It is based on a layered next-gen endpoint protection platform with prevention, detection and blocking capabilities, using machine learning techniques, behavioural analysis, and continuous monitoring of running processes.
Key Features
Machine Learning Anti-Malware: Bitdefender’s machine learning models utilize 40,000 features and billions of file samples to predict and block advanced attacks effectively, improving malware detection accuracy while minimizing false positives.
Process Inspector: Operating in zero-trust mode, Process Inspector continuously monitors all processes in the system, detecting suspicious activities and anomalous behaviours. It effectively identifies unknown advanced malware, including ransomware, and takes remediation actions such as termination and undoing changes.
Advanced Anti-Exploit: This technology protects memory and vulnerable applications by detecting and blocking exploit techniques like API caller verification, stack pivot, and return-oriented-programming (ROP).
Endpoint Control and Hardening: Policy-based controls include firewall management, USB scanning for device control, and web content filtering with URL categorization.
Anti-Phishing and Web Security Filtering: Real-time scanning of web traffic, including SSL, http, and https, prevents the download of malware. Anti-phishing protection automatically blocks fraudulent web pages.
Response and Containment: GravityZone automatically blocks and contains threats, terminates malicious processes, and rolls back unauthorized changes.
Ransomware Protection: Bitdefender can detect new ransomware patterns, offering robust protection against evolving threats.
Automate Threat Remediation and Response: GravityZone neutralizes threats through actions such as process terminations, quarantine, removal, and rollback. Real-time threat information sharing with Bitdefender’s cloud-based threat intelligence service prevents similar attacks globally.
GravityZone Control Center: GravityZone Control Center is an integrated and centralized management console that provides a view for all security management components. It can be cloud-hosted or deployed locally. GravityZone management center incorporates multiple roles and contains the database server, communication server, update server and web console.
Cisco Secure Endpoint Essentials is a comprehensive endpoint security solution that provides advanced protection, threat detection and response capabilities in a single agent that offers Endpoint Detection and Response and integrated Extended Detection and Response (XDR) capabilities.
Key Features
Advanced Protection: Cisco Secure Endpoint uses a layered approach consisting of reputation, application, process and command monitoring, machine learning and behavioural analysis to detect and prevent advanced attacks.
Next-Generation Antivirus (NGAV): Preventative technologies to stop malware by leveraging file reputation, exploit prevention, script protections, and signature detection techniques to stop known and unknown threats.
Endpoint Detection and Response (EDR): Real-time visibility and control of endpoint activities to enable threat hunting and accelerate incident response.
Threat Intelligence: Cisco Talos Intelligence provides the latest threat intelligence to identify and prevent emerging threats.
Dynamic analysis: Produces detailed runtime insight and analysis, including the severity of behaviours, the original file name, screenshots of the malware executing, and packet captures.
Device Control: Visibility and control over USB mass storage devices.
Secure Endpoint: This prevents breaches, blocks malware at the point of entry, and continuously monitors and analyses file and process activity to rapidly detect, contain, and remediate threats that can evade front-line defences.
Prevention and Detection: Identify and stop threats before compromise. Reduce the attack surface with prevention techniques, risk-based vulnerability management, and posture assessments. Enable hunts for hidden threats, detect malware, and perform advanced investigations.
Rapid Response: The Cisco Secure portfolio provides automatic global outbreak control. Endpoint response ranging from file, application and network control to automated actions and isolation help automate endpoint triage and threat containment to reduce time to respond.
Extended Detection and Response (XDR): Reduce incident detection and response times with Cisco Extended Detection and Response (XDR). Built-in integration with the Cisco Secure portfolio and 3rd party solutions to provide a unified view to simplify and orchestrate incident response across your security control points, for a layered defence against threats.
Flexible Deployment and Simplified Management: The solution is easy to deploy, manage, and scale. It can be deployed on-premises or in the cloud, providing flexibility to meet different organizational needs.
Single Agent: Cisco Secure Endpoint Essentials combines Endpoint Prevention, Detection and Response in a single agent.
Management Console: The solution provides a centralized management console to manage and monitor endpoints and can be deployed on-premises or in the cloud.
Scalability: management console can scale to support businesses as they grow.
CrowdStrike Falcon Pro offers cloud-native capabilities through a lightweight agent and a centralized command center. In addition to threat protection, it provides investigative functions and threat intelligence for analysis and remediation of attacks. The solution is scalable, making it suitable for managing networks with thousands of devices.
Key Features
Easy to deploy: The Falcon agent is easy to deploy at scale, offering instant protection without the need for a reboot or tuning processes.
Advanced Threat Detection: Falcon Pro is designed to detect advanced and unknown threats, including fileless attacks, ransomware, adware, and potentially unwanted programs.
Full Attack Visibility: The solution provides attack visibility through a process tree. It unravels complete attack scenarios, enriches them with contextual threat intelligence, and maps adversary behaviours using MITRE ATT&CK® terminology.
Falcon Fusion: Falcon Pro includes Falcon Fusion, an integrated Security Orchestration, Automation, and Response (SOAR) framework. This enables IT and security teams to streamline workflow orchestration and automation.
Signatureless Approach: Falcon Pro does not rely on signatures, eliminating the need for daily virus definition updates. This reduces the administrative overhead and ensures protection against emerging threats.
Exploit Blocking: The solution proactively blocks the execution and spread of threats through unpatched vulnerabilities, preventing potential exploitation.
On-Write Quarantine: Falcon Pro detects and isolates malicious files as soon as they appear on a host, ensuring they are contained and unable to cause harm.
Custom Indicators of Attack (IOAs): Teams can utilize custom IOAs to create behaviour-based blocking rules tailored to their specific organizational needs, providing enhanced protection against targeted attacks.
Advanced Memory Scanning: Automated memory scans are performed using behavioural triggers to prevent fileless and memory-based attacks, such as ransomware and the use of dual-purpose tools like Cobalt Strike, earlier in the kill chain.
Quarantine Functionality: Blocked files are quarantined, allowing analysts to access and investigate them for deeper analysis and understanding of the threat landscape.
Script-Based Execution Monitoring: Falcon Pro inspects and blocks malicious office macros, preventing script-based attacks.
Incident Response Acceleration: The solution accelerates incident response workflows by offering automated, scripted, and manual response capabilities. This streamlines the incident management process and enables faster resolution.
Built-in Threat Intelligence: Falcon Pro integrates comprehensive threat intelligence, strengthening detection capabilities and enhancing the efficiency of Security Operations Centers (SOCs). From automatic sandbox submissions of blocked files to actor profiles, analysts can gain valuable insights into threats and adversaries without exposing their local systems and network infrastructure.
Cybereason NGAV: Multiple layers of unparalleled attack protection. Cybereason brings a unique approach of multi-layered NGAV defence, with multiple layers purpose-built to prevent unique attacker techniques. Designed to stop everything from the simplest to the most novel Malware that exists today, even those never before seen. When these independent, yet complimentary, layers are combined, unparalleled attack protection is achieved.
During AV-Comparatives testing, a base configuration of Cybereason NGAV is used where many of these unique layers are enabled. The most unique layers in the Cybereason NGAV product enabled during the testing are AI-Based Anti-Malware and Fileless Malware Prevention.
Key Features
Anti-Malware: Designed to block malware, the AI-Based anti-malware layer leverages artificial Intelligence to evaluate behaviour occurring across the enterprise as a whole to stop actors in their tracks, even when they’re using never before seen malware.
Fileless Malware Prevention: Purpose-built to block in-memory command line and script-based attacks, the Fileless Malware Prevention layer examines the behaviour of the PowerShell engine, .Net, JScript, and VBScript to ensure that attackers are not able to slip by defences by loading malicious code into memory.
Elastic Security for endpoint prevents ransomware and malware, detects advanced threats, and arms responders with vital investigative context. Elastic Security provides organizations with prevention, detection, and response capabilities across running on both traditional endpoints and public, private, and hybrid cloud environments.
Elastic Security combines SIEM threat detection features with endpoint prevention and response capabilities in one solution. These analytical and protection capabilities, leveraged by the speed and extensibility of Elasticsearch, enable analysts to defend their organization from threats before damage and loss occur.
Key Features
Prevent complex attacks: Prevent malware and ransomware from executing, and stop advanced threats with malicious behaviour, memory threat, and provides credential hardening protections. All powered by Elastic Labs and the global community.
Detect threats in high fidelity: Elastic Defend facilitates deep visibility by instrumenting the process, file, and network data in users’ environments with minimal data collection overhead.
Triage and rapid response: Elastic Security allows for detailed analysis of data across hosts and examining of host-based activity with interactive visualizations. It allows users to invoke remote response actions across distributed endpoints. The investigation capabilities can be further extended with the OSquery integration, fully integrated into Elastic Security workflows.
Secure cloud workloads: This allows stopping threats targeting cloud workloads and cloud-native applications. The lightweight user-space agent, powered by eBPF, allows for real-time visibility and control. Automates identification of cloud threats with detection rules and machine learning (ML). MITRE ATT&CK-aligned detections honed by Elastic Security Labs enable a rapid time-to-value.
View terminal sessions: This gives security teams an investigative tool for digital forensics and incident response (DFIR), reducing the mean time to respond (MTTR).
Continuous Monitoring: Including both user and network activity monitoring but also custom security monitoring. This allows the protection of platforms like AWS, GCP, and Azure from data theft, resource hijacking, and sabotage. Allowing users to observe container security and health and to safeguard distributed workplaces by tracking IT and security applications from Azure AD to Zoom.
ESET PROTECT is powered by ESET LiveSense, ESET’s multi-layered technology that combines machine learning and ESET LiveGrid, ESET’s global, cloud-based reputation system.
Key Features
Combines cybersecurity needs: ESET PROTECT Platform integrates multiple cybersecurity capabilities under one roof so customers can choose which are most effective for protecting their organization. It is simple, modular, adaptable, and continuously innovated – across all operating systems.
Modern endpoint capabilities and protection tools: ESET uses multi-layered technologies that go far beyond the capabilities of basic antivirus or antimalware. ESET PROTECT Entry provides ESET’s multi-layered protection and threat intelligence information, which protects against ransomware and botnets, blocks targeted attacks, prevents data breaches, and detects zero-day threats, fileless attacks, advanced persistent threats and more.
In-house research and development: ESET’s teams not only develop its products but also publish research. ESET is also currently among the top 5 contributors and top 10 referenced sources in the MITRE Enterprise Matrix, thus providing much-needed intelligence into TTPs exploited by diverse APT groups.
Local language support for users in every corner of the globe: The enterprise management consoles are available in 23 languages, and the endpoint security solution in 37 languages, making ESET’s solution one of the most accessible.
Network management with one-click actions: Actions such as isolating the device from the network, creating an exclusion, or initiating a scan are available with a single click in ESET PROTECT console.
Deep-dive insights into the network: ESET PROTECT Platform provides over 120 built-in reports and allows you to create custom reports from over 1000 data points.
Real-time alerts about incidents in your organization: Use pre-defined notifications or create your own. The notification system features a full “what you see is what you get” editor.
Effortless and quick installation: Deploy pre-configured live installers that automatically activate and connect your endpoints to the management console.
G DATA Endpoint Protection Business is a long-standing product line that has developed from a static scanning engine only product into incorporating next generation scanning and heuristic technologies. These technologies help us detect and prevent malware even when normal scanning approaches fail.
Key Features
Privacy by design: G Data’s development only happens in Germany, which had very strict data privacy laws even before the GDPR, employing strict privacy by design and by default rules in the development of their software.
Online and offline protection: G Data’s products offer very strong offline and local protection by design. Protection modules work offline and do not require a cloud connection, although the cloud connection does improve detection against latest and unknown threats.
BehaviorStorage (BEAST) module: This module runs locally on the client and does not transmit user behaviour data into a cloud. BEAST is able to run completely independent of Internet connectivity and can still classify suspicious or malicious activity.
In house support: Support is not outsourced, being involved in the development processes which enables G Data to fix errors reported by customers.
MMC style admin: Allowing for easy use by Windows administrators.
K7 Security simplifies deployment and management, protecting client workstations and critical servers. The Centralised Management Server consolidates threats, implements endpoint security policies, and manages them with fewer IT resources. The web-based console handles K7 software installation on multiple endpoints, user group creation, policy enforcement, task scheduling, updates, and remote management of core capabilities such as Antivirus, Firewall, Application Control, and Web Content Filtering.
Key Features
Admin Console: The web-based interface enables complete security settings management, including client installation, group and policy management, task scheduling, updates, and control over Antivirus, Firewall, Application Control, Web Filtering, and Notifications.
Advanced Malware Detection and Remediation: The Host Intrusion Prevention System collates, analyses and triages various events to effectively detect and deal with malware. This feature deals with analysis of both pre-execution and runtime behaviour of monitored objects in the host.
Anti-Ransomware Protection: Monitors secured devices for ransomware, employing signature-less, behaviour-based detection mechanisms. K7 Ecosystem Threat Intelligence enhances protection against known and new ransomware variants. Real-time security defends against ransomware distribution through shared files and folders on the network.
K7 Device Control: This prevents USB and storage media infections by blocking unauthorized access to unknown devices. Host-level policies enforce device password access, file execution control, and on-demand/automatic device scanning.
K7 SafeSurf: This ensures secure online browsing by identifying and blocking malicious websites through URL analysis and cloud-based reputation services.
K7 Firewall / HIPS: The K7 Firewall, working with the integrated Host Intrusion Prevention System (HIPS), stealths system ports and protects against direct attacks. The Intrusion Detection System (IDS) blocks known malicious network-based exploits before processing.
System Security and Performance: K7 Security prioritizes system performance by utilizing a proprietary lean data-loading algorithm and ordering mechanism, minimizing RAM and CPU usage.
Web Categorisation: Web Categorization allows administrators to define website and content access for company devices, limiting access to unproductive or inappropriate sites.
Groups and Policies: Endpoint security is managed through groups and policies, controlling malware detection, and user settings. Default settings provide optimum security, and end-users are limited to updates and scans.
Application control: This enables automatic reporting and blocking of applications, including version-based blocking.
Fine control of administrative privileges: Administrative privileges can be fine-tuned with custom roles and group-based administration.
Scans: Options include Quick Scan, Full System Scan, and Vulnerability Scan, with patch links. Scans can be scheduled and deployed to desired endpoints.
Kaspersky Endpoint Security for Business is a next-gen endpoint security solution which can secure organizations against a wide range of threats, from BIOS-related to fileless threats. The solution provides crucial endpoint management and security tools to IT administrators and cybersecurity specialists in organizations of any size and type.
Key Features
Protect user data: Kaspersky Endpoint Security for Business protects all endpoints against widespread and emerging threats, thanks to Kaspersky technologies like behaviour-based protection from advanced threats including fileless ones, ML-based analysis, and specific protection against exploits, ransomware, miners and financial spyware. Recognizing threat behaviour patterns, allows for the neutralizing of unknown threats.
Proactive protection: Stops attacks before they start. System hardening by Adaptive Anomaly Control combines the simplicity of blocking rules with the smartness of automatic tuning, based on behaviour analysis.
Reduced attack surface: This is achieved by controlling what applications, websites and devices can interact with endpoints and users.
Complete ecosystem: Users can grow their IT security maturity. Automated response and analysis leverages integrations with EDR and SIEM solutions
Single solution for any platform: Security for every workstation, server and mobile device that carries user data, regardless of location and ownership.
Cross platform support: A single solution, working from a single console covers every OS in a mixed environment.
High levels of automation: Particularly for essential but routine tasks such as patching and OS deployment.
Remote management capabilities: Covering different scenarios, like setting up workstations in home offices or securing data with encryption options.
Centralization: Integrated single-screen management, either at the user’s perimeter or in the cloud.
Futureproofing: Upgrading is seamless, allowing users to move through the tiers. The fully scalable solution is ready to support thousands of managed devices as companies grow.
Flexibility: Users can choose their preferred deployment option: in the cloud, on-premises, air gapped and in hybrid deployments. Then they can allocate different levels of security systems access to different team members with granular role-based access control (RBAC).
Microsoft Defender Antivirus is pre-installed on Windows 10/11 systems. In business environments, it can be managed e.g. with Microsoft Defender for Endpoint’s P1 plan. Microsoft Defender for Endpoint is an enterprise security product designed to help organizations prevent, detect, and respond to evolving threats across operating systems and network devices. Its antivirus capability combines machine learning models trained on cloud-scale data and behaviour-based detection to protect in real-time against malware and malicious activity.
Key Features
Defender for Endpoint’s P1 plan allows security teams to do the following:
Eliminate blind spots in their environment: Discover unmanaged and unauthorized endpoints and network devices. Secure these assets using integrated workflows.
Block sophisticated threats and malware: Examples include novel polymorphic and metamorphic malware, and fileless and file-based threats. With cloud-delivered, next-generation protection, analysts benefit from near-instant detection and blocking of these threats.
Apply manual response actions: Security teams can act on devices or files when threats are detected, such as quarantining them.
Harness attack surface reduction capabilities: Harden devices, prevent zero-day attacks, and take granular control over endpoint access and behaviours. These capabilities include rules, ransomware mitigation, device control, web protection, network protection, network firewall, and application control.
Access unified security tools and centralized management: Security administrators can use role-based access control from the Microsoft 365 Defender customizable portal to manage which users have access to which assets.
Management console: The Microsoft 365 Defender portal provides security teams access to unified security tools and centralized management. This can be used to monitor and respond to alerts of potential threats and can go beyond protecting endpoints to securing across identities, data, apps, and infrastructure.
Customizable home page: The landing page provides a customizable view that shows at-risk devices, threats detected, alerts/incidents and actionable information depending on which Microsoft Defender capabilities the organization is using. Examples of what you can see:
- Incidents & alerts: Lists incidents that were created as a result of triggered alerts generated as threats are detected across devices.
- Action center: This lists remediation actions taken. Analysts can see details like investigation package collection, antivirus scan, app restriction, and device isolation.
- Reports section: This section includes reports that show threats and their status.
- Device Inventory: A list of the devices in the user’s network that triggered alerts. This shows domain, risk level, OS platform, and other details for easy identification of devices most at risk.
Sophos Intercept X Advanced is an endpoint security solution designed to minimize the attack surface and prevent attacks. It combines multiple technologies, including anti-exploit, anti-ransomware, deep learning AI, and control technology to detect and block threats before they can impact users’ systems.
Key Features
Stop Unknown Threats: Intercept X utilizes deep learning AI to identify and block malware that hasn’t been seen before. It analyses file attributes to detect threats without relying on signatures.
Block Ransomware: Intercept X incorporates anti-ransomware capabilities that identify and block the encryption processes used in ransomware attacks. Encrypted files can be rolled back to a safe state, minimizing the potential impact.
Prevent Exploits: The anti-exploit technology in Intercept X prevents attackers from leveraging exploit techniques to compromise devices, steal credentials, and distribute malware. This protection extends to file-less attacks and zero-day exploits.
Reduce the Attack Surface: Users have control over the apps and devices allowed to run in their environment. Intercept X enables blocking of malicious websites and potentially unwanted apps (PUAs).
Synchronized Security: Sophos solutions work together seamlessly. For instance, Intercept X and Sophos Firewall share data to isolate compromised devices during cleanup, restoring network access once the threat is neutralized, all without requiring admin intervention.
Straightforward Management: Intercept X is managed through Sophos Central, the cloud-based management platform for all Sophos solutions. This centralized management approach simplifies deployment, configuration, and management, including remote working setups.
AI and Expert Powered Data: Intercept X combines the power of deep learning AI with the expertise of SophosLabs cybersecurity professionals to provide robust protection and accurate threat detection.
Trellix Endpoint Security (ENS) is a comprehensive security solution designed for enterprise networks of all sizes. The ePolicy Orchestrator management console offers flexible options, including both cloud-based and on-premises consoles, for efficient management of the endpoint protection software.
Key Features
Customizable Dashboard: The dashboard and reporting can be tailored to display relevant endpoint status information for each user.
Deployment Flexibility: The console offers a variety of deployment options, including cloud-based, on-premises hosting, and Amazon hosting.
Management Console: The ePolicy Orchestrator console is easily accessed through the primary navigation menu located at the top left of the main dashboard. It provides access to different sections and pages, such as Dashboard, Reporting, Policy Management, Automation, and Software and Systems Administration. Integration of additional components like DLP, Mobile Security, and Insights Threat Intelligence and EDR is also available.
Real Protect: Through machine learning classification, threats are detected in real time, and behavior classification continually evolves to identify future attacks. Endpoints are restored to the last known good state, preventing infections and reducing administrative burdens.
Adaptive Scanning: The system intelligently skips scanning trusted processes and gives priority to suspicious processes and applications during scanning.
Endpoint Client Deployment: Client agent packages can be created on the Product Deployment page. The installer file can be distributed via a web link, manually executed, or deployed through a systems management product. After installation, the agent downloads the necessary protection engine before full protection becomes active. The client interface displays the installed and enabled protection components.
Proactive web security: This feature ensures safe browsing by providing web protection and filtering for endpoints.
Hostile network attack blocking: The integrated firewall utilizes reputation scores based on GTI to safeguard endpoints against botnets, DDoS attacks, advanced persistent threats, and suspicious web connections. During system startup, the firewall only allows outbound traffic, providing protection when endpoints are not connected to the corporate network.
Antimalware protection: Trellix protects, detects, and corrects malware quickly with an antimalware engine that works across multiple devices and operating systems.
VIPRE Endpoint Detection & Response (EDR) provides comprehensive endpoint protection with next-gen antivirus (NGAV) and EDR features combined into a seamless platform. Designed to automatically block the vast majority of threats, and to provide for quick and efficient containment and investigation of potential threats, VIPRE provides everything you need to keep your endpoints and users safe.
Key Features
Detailed network protection: This includes a full IDS, DNS Protection, and browser exploit prevention. The core NGAV components scan for and remove any latent malware, and behavioural process monitoring ensures that apps and users behave. The EDR layer on top of these core components orchestrates response to zero-day and persistent threats that can’t be immediately identified as malicious, but that represent a possible threat.
Supports investigation: EDR bundles in endpoint vulnerability scanning, raw event telemetry, and detailed root cause analysis. VIPRE Endpoint Detection & Response (EDR) includes access to cloud-based malware analysis sandboxes to investigate suspicious files and URLs, with detailed results presented right in the console. It also includes a simple method to isolate endpoints that are misbehaving, to prevent attack spread and give you time to understand what is happening on the endpoint.
Remediate threats on endpoints: EDR will help patch vulnerable applications automatically and provides for integrated remote access to the endpoint to clean up files, processes, registry keys, and more. Any files corrupted by zero-day ransomware will be restored. Any security gaps identified by your investigation can be closed quickly.
Single Interface: VIPRE EDR combines all these tools into a clean, easy to use interface that helps speed response times and reduce confusion. Mobile responders can access everything from their smartphones, avoiding the expense, annoyance, and delays of having to rush into the office. And with transparent delegated access via VIPRE Site Manager, MSPs, MSSPs, and MDR providers can assist in incident response and investigation with zero friction.
VMware Carbon Black Cloud™ Endpoint Standard is a cloud native endpoint, workload, and container protection platform that combines the intelligent system hardening and behavioural prevention needed to keep emerging threats at bay. The cloud native protection platform enables customers to utilize different modular capabilities to identify risk, prevent, detect and respond to known and unknown threats using a single lightweight agent and an easy-to-use console. Its sensor serves as both a continuous event recorder and preventive action agent. For detection and response purposes, the VMware Carbon Black Cloud captures all process executions and associated metadata, file modifications, registry modifications, network connections, authentication events, module loads, fileless script executions, and cross-process behaviours (i.e., Process injection). All this behavioural activity is captured and streamed live to your cloud instance for visualization, searching, alerting, and blocking. This allows for both real-time and historical threat hunting across your environment. The VMware Carbon Black Cloud also keeps track of every application executed in your environment and its metadata, including a copy of that binary for forensics purposes.
Key Features
Threat prevention updates: Carbon Black deploys updates to prevent the latest attack techniques focused on behavioural attributes quickly without additional effort required by users.
Custom detections: Rapidly deploy custom detections in the form of threat intelligence indicators focusing on the same behavioural attributes.
Alert and detections mapping: Alerts and detection techniques can be directly mapped to MITRE ATT&CK®.
Post analysis tools: Search for binary prevalence, process masquerading, binary signing issuers, and forensic capture for post analysis
Robust and extensible API: Some examples of 3rd party API integrations are:
- YARA
- Out of the box SIEM, SOAR and ITSM API integrations
- Binary Detonation and Sandboxing Uploads
- Network security/service appliances (DNS, IDS, IPS, DHCP)
- File integrity monitoring – VMware Carbon Black Cloud can alert any time files, file paths, registry keys, and registry hives are
WatchGuard EPP is a cloud-native security solution that centralizes next-gen antivirus with advanced technologies to protect against threats. It offers real-time monitoring, behaviour analysis, and blocking of malware. WatchGuard EPP defends against ransomware attacks with contextual detections, anti-phishing, decoy files, and shadow copies.
Key Features
Multiplatform Security: cross-platform security for various systems. Management of licenses belonging to both persistent and non-persistent virtualization infrastructure (VDI).
Management and Installation: Multiple deployment methods available, with automatic uninstallers for other products allowing rapid migration from third-party solutions. Deployment can be done via email and download URL, or silently to selected endpoints via the solution’s distribution tool. The MSI installer is compatible with third-party tools (Active Directory, Tivoli, SMS, etc.).
Performance: all operations are performed on the Cloud. WatchGuard EPP requires no installation, management, or maintenance of new hardware resources in the organization’s infrastructure.
Centralize Device Security: centralized management from a single web-based administration console for all workstations and servers on the corporate network.
Malware and Ransomware Protection: WatchGuard EPP analyses behaviours and hacking techniques to detect and block both known and unknown malware, as well as ransomware, trojans and phishing.
Advanced Disinfection: in the event of a security breach, affected computers can be restored to the state before infection with advanced disinfection tools. Quarantine stores suspicious and deleted items. Administrators can remotely restart workstations and servers to ensure the latest product updates are installed.
Real-time Monitoring and Reports: detailed, real-time security monitoring is delivered via comprehensive dashboards and easy-to-interpret graphs. Reports are automatically generated and delivered on protection status, detections, and improper use of devices.
Granular Configuration of Profiles: Assign user profile-based protection policies, ensuring appropriate policies for every user group.
Centralized Device Control: Stop malware and information leaks by blocking device categories (flash drives, USB modems, webcams, DVD/CD, etc.), allowlisting devices or configuring read-only, write-only, and read-and-write access permissions.
Vulnerability Assessment: Vulnerability assessment helps IT teams to identify, evaluate, and prioritize security weaknesses and vulnerabilities in applications and systems.
Malware Freezer: Quarantines malware for seven days and, in the event of a false positive, automatically restores the affected file to the system.
Ransomware Remediation and Recovery: Besides encrypting files, adversaries try to delete backup and VSS files and turn off services designed to help recovery. Files are protected using shadow copies, which can be used to recover ransomware encrypted files.
Award levels reached in this Business Security Tests and Review
As in previous years, we are giving our “Approved Business Product” award to qualifying products. As we are conducting two tests for business products per year, separate awards will be given to qualifying products in July (for March-June tests), and December (for August-November tests).
To be certified in July 2023 as an “Approved Business Product” by AV-Comparatives, the tested products must score at least 90% in the Malware Protection Test, with zero false alarms on common business software, and an FP rate on non-business files below the Remarkably High threshold. Additionally, products must score at least 90% in the overall Real-World Protection Test (i.e. over the course of four months), with less than fifty false alarms on any clean software/websites, and zero false alarms on common business software. Tested products must also avoid major performance issues (impact score must be below 40) and have fixed all reported bugs in order to gain certification.
We congratulate the vendors shown below, whose products met the certification criteria, and are thus given the AV-Comparatives Approved Business Security Product Award for July 2023:
Although Cybereason and Elastic achieved good malware protection scores, they unfortunately did not reach all the requirements for the July 2023 Approved Award. This was due to the level of false positives on non-business files. We hope to see these issues resolved in the second half year of the 2023 tests.
Copyright and Disclaimer
This publication is Copyright © 2023 by AV-Comparatives ®. Any use of the results, etc. in whole or in part, is ONLY permitted after the explicit written agreement of the management board of AV-Comparatives prior to any publication. AV-Comparatives and its testers cannot be held liable for any damage or loss, which might occur as result of, or in connection with, the use of the information provided in this paper. We take every possible care to ensure the correctness of the basic data, but a liability for the correctness of the test results cannot be taken by any representative of AV-Comparatives. We do not give any guarantee of the correctness, completeness, or suitability for a specific purpose of any of the information/content provided at any given time. No one else involved in creating, producing or delivering test results shall be liable for any indirect, special or consequential damage, or loss of profits, arising out of, or related to, the use or inability to use, the services provided by the website, test documents or any related data.
For more information about AV-Comparatives and the testing methodologies, please visit our website.
AV-Comparatives
(July 2023)