This website uses cookies to ensure you get the best experience on our website.
Please note that by continuing to use this site you consent to the terms of our Privacy and Data Protection Policy .
Some of our partner services are located in the United States. According to the case law of the European Court of Justice, there is currently no adequate data protection in the USA. There is a risk that your data will be controlled and monitored by US authorities. You cannot bring any effective legal remedies against this.

The difference between AV-Comparatives’ EPR Test and MITRE ATT&CK Engenuity

Both the AV-Comparatives EPR Test and MITRE Engenuity have their merits, each providing useful insights into endpoint security solutions. Understanding the differences between these two tests is essential for IT managers, CISOs, and other tech-savvy professionals looking to select endpoint security solutions that will effectively protect their environments.

Test Scenarios

While MITRE Engenuity evaluates techniques from a single attack chain, carried out by a preselected APT announced in advance of the assessment, AV-Comparatives’ EPR Test involves 50 separate attack scenarios from undisclosed APTs. AV-Comparatives refrains from disclosing the attack methods and techniques in advance, mirroring real-world scenarios. This approach aims to showcase a solution’s ability to prevent, detect, and remediate attacks while providing passive response to users.

Traditionally, MITRE Engenuity participants are aware of the adversary groups chosen for upcoming evaluations through Calls for Participation. However, the Call for Participation in Managed Services Round 2023-2024 represents a departure from this practice. This latest Call for Contribution also allows participants to potentially impact test complexity by submitting data about APT tactics, techniques, and procedures known only to them.

Protection Capabilities

Historically, MITRE assessed solutions in Detect-Only mode, examining product responses to individual techniques within attack chains. However, MITRE only began testing Protection scenarios, meaning the attack was blocked or disrupted, in Round 3 (2020-2021). On the other hand, AV-Comparatives has been dedicated to validating Protection capabilities since the test’s inception in 2020.

Product Settings and Real-World Implementation

MITRE Engenuity permits the use of customized product settings and allows vendors to list these configurations on dedicated product pages. While vendors might use highly specific settings to enhance test results, these settings may not be practical for real-world use due to potential false positives, performance issues, and alert fatigue for real-world EDR/XDR operators. AV-Comparatives maintains control over setting changes, reporting them in the test results, so as to better inform users.

Scoring and Performance Comparisons

MITRE Engenuity lacks a straightforward scoring system to compare products’ effectiveness against threats and lacks comprehensive incident telemetry. AV-Comparatives addresses this gap by introducing a simple comparison scoring system, aiding customers in evaluating product efficiencies. Additionally, AV-Comparatives introduced a Total Cost of Ownership metric for product comparison, providing better insight into the numbers. MITRE Engenuity participants all claim to be the winner at the end. In contrast, AV-Comparatives testing poses greater challenges, allowing participants to remain anonymous. Achieving certification signifies exceptional proficiency, even for Strong Challengers.

Operational Accuracy and False Positives

Unlike AV-Comparatives’ EPR-Test, the MITRE Engenuity assessment does not consider False Positive scenarios (operational accuracy). This approach, combined with the flexibility to modify product configurations, introduces a risk of misinterpreting final results. In contrast, AV-Comparatives’ EPR-Test assesses operational accuracy and emphasizes the importance of balancing false negatives and operational accuracy.

Test Schedules

MITRE Engenuity testing occurs over varying timeframes, with several months potentially separating evaluations. Those who join early may have the option to be tested later. In AV-Comparatives’ EPR-Test the solutions are tested almost in parallel.

Telemetry and Threat Hunting

In devising its Engenuity tests, MITRE employs telemetry, heavily relying on data interpretation skills to uncover insights. Manufacturers with knowledge of what to search for are better equipped to uncover valuable findings in the data. Alternatively, AV-Comparatives devises its EPR tests based on its research of attack scenarios its specialists are aware of and have analyzed in depth themselves and also trained by the vendors before the test in order to operate their solutions efficiently.


In conclusion, the AV-Comparatives EPR Test and MITRE Engenuity tests offer complementary insights, each with a unique focus. It is essential for IT managers and other cybersecurity professionals to understand the differences between these tests and weigh these factors when selecting endpoint security solutions.

P.S.: AV-Comparatives’ EPR Test 2023 will be released in a few weeks (end of October 2023).