We had a closer look at the vendors’ reaction regarding the samples they missed in the online file-detection test of September.
100 days later, many of them had added detection for all missed malicious files we had sent to them after the test.
This means they now detect all threats in the test-set, resulting in a 100% detection rate with the September test-set. On the other hand, six vendors only added detection for about 90% of their misses, which results in a detection rate of between 99.2% and 99.9% of the September test-set.
To compare the reaction with that of other vendors not in our public main-test series, and who therefore do not get missed samples, we looked at how much one well-known vendor (with a similar number of misses) added in the past 100 days; it was also around 90%.
This shows that some vendors are faster at adding detection for missed malware files, and some are slower or reactive (i.e. wait till one of their users reports an infection / a missed sample to them) in adding detection for malicious samples, even if these are prevalent and confirmed as malicious. Some vendors sometimes claim that they get low scores in tests because they do not detect “non-malicious” or “non-prevalent” samples, which is thus shown not to be accurate, as they and even other vendors not taking part in tests add (albeit with much delay) missed malicious files which have been found in the field.