Virus Stoppers: How anti-virus software is tested

Twenty years after the first one appeared, PC viruses are more diabolical than ever. How do you keep them off your system?


The rating for stand-alone anti-virus software is based on separately calculated scores in each of four categories: Performance, Specifications, Design, and Price.

The Performance score, heavily weighted in the rating, measures not only system speed tests but also the critically important malware detection and disinfection tests. The detection tests are heavily weighted within the performance score.

Detection and disinfection tests are done in conjunction with AV-Test.org, a security research company in Germany. AV-Test.org puts programs through a rigorous analysis; its overall malware detection test pits each app against an almost 900,000-sample "zoo" of viruses, Trojan horses, back doors, and other malware types. Some of these samples are commonly used in Internet attacks; others are far less well known and may have been used in small, targeted attacks. For these on-demand zoo detection tests, the antivirus programs are set at their best detection settings.


Outbreak response time tests rate how quickly each antivirus company delivers the malware signatures used in these detection tests, as measured by AV-Test.org. The reported time ranges are based on when the first company begins releasing signatures, since determining when a given piece of new malware first surfaces in an attack can be difficult or nearly impossible.

To perform proactive tests that simulate how well the programs can detect unknown malware, AV-Test.org scans a set of new malware with each app using one- and two-month-old signature files. The detection tests also examine how well each program handles different document types, such as whether it can find malware hidden within various types of archived files.

The disinfection tests measure how well an antivirus app can detect and then clean an existing infection that has installed itself on a PC. AV-Test.org checks whether the antivirus software has removed malware files, changes to the Hosts file, and Registry changes. Cleanup of Registry entries is the least important of the three areas, and is weighted less heavily.

To round out the Performance score, the impact of the antivirus software on a PC's speed is tested. This is done with PC World's WorldBench 6 Beta 2 benchmarking application. These tests measure how long a computer takes to run a set of automated tasks with a variety of programs, including Firefox, Microsoft Office, WinZip, and other apps. WorldBench 6 Beta 2 is run on a test PC multiple times with the antivirus software installed and then without. By comparing the results with and without the software installed, the system drag or slowdown for each antivirus program can be calculated.

The Specifications score gauges each program's basic feature set. While it's important, it's not nearly as heavily weighted as the Performance score. Things such as whether the program scans e-mail and Web traffic to catch attacks before they hit the hard drive and whether the program can manually delete a file via a right-click menu option in Windows Explorer are checked. Also the type of support (phone and e-mail) a program offers and any costs associated with it is researched.

The Design score evaluates each program's interface and ease of use. If a program makes it easy to find and understand program settings, looks good, and installs with default options appropriate for the average user, it scores well here.

Finally, the price is rated. For the sake of consistency, the download price rather than the price of a retail box (where there was a difference) is used. The purchase price of a license for one computer for one year (except for products whose lowest price covers multiple computers), as well as the second-year renewal cost is used. Most companies offer different rates for multiple computers and/or multiple years, so check the options for the best deal for your situation.