AV-Comparatives weblog

This AMTSO meeting was great: interesting discussions on how tests can be improved etc. The AMTSO members finalized two papers, one about fundamental testing principles and one about best practices for dynamic testing. These drafts were approved unanimously by all the different vendors and testers/revieweres/publishers present, and will probably be published on the AMTSO web site in the next week or so.

I suggest that anyone involved in testing (testers, reviewers, etc.) read and follow those guidelines, and that consumers will also benefit from reading the documents in order to get a better understanding of how tests should be done properly and so on. AMTSO is currently working on many topics and draft documents which will be of considerable interest and will contribute to raising the quality of testing etc. AV-Comparatives supports the goals of AMTSO, as it is for the benefit of the consumers if they can build informed opinions about AV products based on tests that are properly conducted and without bias.

Therefore, please spread the word about the existence of the Anti-Malware Testing Standards Organization: many people have wanted such an organization to exist for years, and now that it does, it should get the attention it deserves.

AMTSO membership is open to academics, reviewers, publications, testers and vendors, subject to guidelines determined by AMTSO.

Peter and I are going to the next AMTSO* meeting. We will return on Sunday - all emails will be answered on return.

* AMTSO (The Anti-Malware Testing Standards Organization ) is an international non-profit association that focuses on the addressing the global need for improvement in the objectivity, quality and relevance of anti-malware testing methodologies.

P.S.: last weeks we were and still are busy with some new test (beside the retrospective test which has also to be done). I guess we will release it in some weeks, so check our site for news from time to time.

I (ac) just came back to IBK from Ottawa. The VB conference is IMO the best of all, mainly because almost everyone goes there, and after all, meeting peoples is the most interesting part. I enjoyed to meet and talk with peoples e.g. from (in no specific order): K7 Computing, Kingsoft, WildList, ESET, Sunbelt Software, AVG Technologies, TrustPort, NSS Labs, VirusBuster, Symantec, Avira, Sophos, KasperskyLab, VirusBulletin, Alwil Software, G DATA Software, McAfee, F-Secure, BitDefender, Microsoft, MessageLabs, etc. (sorry if I forgot someone, I am tired).

There were a lot more peoples there, but time was limited and to be frankly, this time I decided to went to VB in last minute mainly for some personal reason and not due the presentations (which I prefer to read in the book), so my thoughts were all the time somewhere else. Anyway, the two presentations I remember and I liked most were:

- Rebuilding testing for the future
- The AV industry: Quo Vadis?

I guess during the next weeks the presentations (as well as pictures from the conference) will be available on the VirusBulletin website, so keep watching their site.

Next year the VirusBulletin conference will be in Switzerland - I will be there. And you?

(ac)

P.S.: my mailbox exploded in my absence, so if you got the message ~”mailbox is full” or you do not get a reply within this week, please resend your mail.

The Chairman of AV-Comparatives left for Canada (VirusBulletin conference in Ottawa, etc.) and will be back in about 2 weeks. All emails etc. will be answered after his return..

The updated August 2008 test report has now been upload. Some minor changes were applied, as well as the noncompetitive iclusion of the enterprise version of McAfee (with Artemis); the detailed report about Artemis (and some other reports) will be published in some weeks.

Please download this updated report from our website.

AV-Comparatives set up a newsletter subscription site. We plan to send out in future newsletters containing e.g. useful informations and update notifications. If you are interested in receiving our newsletters, please sign up on http://www.av-comparatives.info

AV-Comparatives released an updated document containing the FAQ’s and methodology. You can find the PDF document on this page.

Some weeks ago we started moving office and now the new PC’s etc. are at the new location. We already started the August test some days ago, but due some unexpected required upgrades at the infrastructure (e.g. intensity of amperage, air cooling system, etc.) and the expanded tests (the August test will include also a false alarm test etc.), we will probably delay the release of the report to around mid September (instead the 1st September), as due the upgrades the tests need to be partially interrupted for some days.

available here (simple and traditional chinese):

http://www.av-comparatives.org/seiten/ergebnisse/report18chinese.pdf
http://www.av-comparatives.org/seiten/ergebnisse/report18sc.pdf

(translated by a third party = we do not take any responsability in case of wrong translation)

We just released the technology preview test report of McAfee “Artemis”. You can find it in the comparatives section of our website. You can access the PDF report directly by clicking here.

A test of Kaspersky Internet Security 2009 has been released. You can find it in the Comparatives section of AV-Comparatives.org

The results of the AV-Comparatives website Survey of March 2008 (”How satisfied are you with your current antivirus?”) are now available. You can download the PDF from here.

We have just released the February 2008 test report translated into Simple Chinese. You can download it from here.

As it was stated in the press release, we wanted to start the collaboration by the end of 2008. After reconsideration and doubts from several respected people and vendors I withdraw herewith our partnership idea with Sergey Ilyin in order to keep our neutral position. Not much will change, except that AV-Comparatives will not use third party results in its reports. AV-Comparatives would prefer to use only results that we can 100% stand by and reproduce, to ensure that our reputation and independence is unaffected by any possible issues with third party results. We will continue to work with and discuss issues of testing with all testers and vendors in the same spirit of openness that has led us to join AMTSO.

A test of IKARUS virus.utilities T3 has been released (March 2008). You can find it in the Comparatives section of AV-Comparatives.org

Our colleagues of ProtectStar TestLab - which used some of our test-results of February - reviewed various well-known Security Suites, evaluating various aspects like the firewall, the user-friendliness, performance and the price/equipment ratio, etc. and awarding some products based on a rating-score. The english report will be released in the first week of April on their website.

The QA report of the August 2007 test set has been released. It can be found on the website in the Comparatives section.

As announced publicly several times already last year, AV-Comparatives provides since 2008 its services for a fee. The fee is NOT for the tests itself, but for the various services we provide (bug reports, usage of our material/logos for reprint and marketing material, getting sample lists, getting false alarms after the public tests, internal services, etc.). Vendors are not obligated to pay for the FULL-service-package, and if a vendor e.g. prefers to do not get all services, the fee is lower. All major testers get paid for the services they provide, this is a common practice. Of course, the fee has no influence at all on the results and is to cover our expenses and time involved.
For users and magazines the public results are free of charge, as long the source is given. If we have time and e.g. a magazine wants us to make an additional work/test of e.g. products that are not already tested, this costs some money which has to be covered by the magazine (only for the service provided to the magazine; the vendor of the tested product will not get any service).