This website uses cookies to ensure you get the best experience on our website.
Please note that by continuing to use this site you consent to the terms of our Privacy and Data Protection Policy.
Accept

RTTL Certification Test March 2016

Date March 2016
Language English
Last Revision February 24th 2016

Release date 2016-04-08
Revision date 2016-02-24
Test Period March 2016
Number of Testcases 253
Online with cloud connectivity checkbox-checked
Update allowed checkbox-checked
False Alarm Test included checkbox-unchecked
Platform/OS Microsoft Windows

Introduction

1st AMTSO Real Time Threat List (RTTL) based certification test.

What is the RTTL

The Real-Time Threat List (RTTL) is a repository of malware samples collected by experts from around the world. The repository is managed, maintained and secured by the Anti-Malware Testing Standards Organization (AMTSO).

Why is there a need for the RTTL?

As malware now travels the globe in real-time with the majority of infections happening through websites then a real-time system was needed to provide testers of Anti-Malware solution a repository of malware they can use to validate that Anti-Malware products are working in real-time to protect users. The end result of this being that published test results allows business and consumer to make informed decisions on what Anti-Malware solution best meets their requirements.

Who Submits samples to the RTTL?

Anti-Malware companies and Anti-Malware experts from around the world submit identified and validated samples to the RTTL, which include prevalence data that includes the distribution and source of the malware.

Who uses the samples from RTTL?

Testers looking at the efficacy of Anti-Malware products use the samples to validate their own collected samples that they test with to check for prevalence to ensure that what they are testing with are real world examples that threaten users a businesses. Academics researching or analysing trends in the Anti-malware industry can also use the RTTL to allow them to have a rich data source to work from.

Source: http://www.amtso.org/rttl/

Tested Products

Test Procedure

AV-Comparatives used their Real-World Protection Framework to execute those 253 samples. Each sample was executed simultaneously under Windows 7 SP1 64bit with the respective security product installed to see if it is detected and blocked. The security products used default settings and had full-Internet (cloud) access.

For this first test, we included available endpoint security products of current AMTSO members.

Testcases

The Top500 samples (tagged as “malware” in RTTL) were queried and tested on the 9th March 2016. As the RTTL contains currently a lot of misclassified clean and PUA files, we took only those samples which showed malicious behaviour in our sandboxes. At the end, 253 malicious samples remained and were taken for the test.

Test Details

Test Period  9th March 2016
Number of Test cases  253
Online with cloud connection  Yes 
Update allowed  Yes 
Platform/OS  Microsoft Windows 7 SP1 64bit english 
Query 

The API Function “getTopFiles” was used with the following parameters according to the RTTL manual:

parameters = {
 “resultsLimit” 	: 500,
 “resultsPage” 		: 1,
 “prevalenceFrom” 	: 1,
 “prevalenceTo” 	: 4294967295,
 “sortBy” 		: “prevalence”,
 “sortOrder” 		: “desc”,
 “fileTypesArr[]” 	: “1”
}
Query Timestamp  9th March 2016 11:38 AM

 

Test Results

29 product have been put to the test. A protection rate over 98% is required to get certified.

Award levels reached in this Certification Test

CERTIFIED
AhnLabCERTIFIED
AvastCERTIFIED
AVGCERTIFIED
AviraCERTIFIED
BitdefenderCERTIFIED
BullGuardCERTIFIED
EmsisoftCERTIFIED
eScanCERTIFIED
ESETCERTIFIED
F-SecureCERTIFIED
FortinetCERTIFIED
G DATACERTIFIED
K7CERTIFIED
Kaspersky LabCERTIFIED
LavasoftCERTIFIED
McAfeeCERTIFIED
MicrosoftCERTIFIED
NanoNOT CERTIFIED
PandaCERTIFIED
PC PitstopNOT CERTIFIED
Quick Heal Total SecurityNOT CERTIFIED
Quick Heal Total SecurityCERTIFIED
SophosCERTIFIED
SymantecCERTIFIED
TencentCERTIFIED
TG SoftNOT CERTIFIED
ThreatTrackCERTIFIED
Trend MicroCERTIFIED
WebrootNOT CERTIFIED

Copyright and Disclaimer

This publication is Copyright © 2016 by AV-Comparatives ®. Any use of the results, etc. in whole or in part, is ONLY permitted after the explicit written agreement of the management board of AV-Comparatives prior to any publication. AV-Comparatives and its testers cannot be held liable for any damage or loss, which might occur as result of, or in connection with, the use of the information provided in this paper. We take every possible care to ensure the correctness of the basic data, but a liability for the correctness of the test results cannot be taken by any representative of AV-Comparatives. We do not give any guarantee of the correctness, completeness, or suitability for a specific purpose of any of the information/content provided at any given time. No one else involved in creating, producing or delivering test results shall be liable for any indirect, special or consequential damage, or loss of profits, arising out of, or related to, the use or inability to use, the services provided by the website, test documents or any related data.

For more information about AV-Comparatives and the testing methodologies, please visit our website.

AV-Comparatives
(April 2016)