Tuesday, June 15, 2010

“Antivirus tests should measure speed, says group” plus 3 more

“Antivirus tests should measure speed, says group” plus 3 more

Antivirus tests should measure speed, says group

Posted: 14 Jun 2010 09:47 AM PDT

The way antivirus programs are rated could be set for a much-needed overhaul after the organisation set up by security vendors to influence test design urged researchers to spend more time on basic issues such as performance.

The new testing methodology recommended by the Anti-Malware Testing Standards Organisation (AMTSO) notes that today's tests put too much emphasis on isolating individual layers of security within a product and fixate on detection rates.

The organisation's new Performance Testing Guidelines recommends that testers also look at measurements that matter to users such as the effect antivirus products have on issues such as boot speed, application loading, memory usage, network overhead and battery drain.

The document even suggests that testing should measure the impact on everyday tasks such as Internet browsing, opening popular types of files such as Word and PDF, and downloading email.

For an industry that has tended until recently to focus on comparative detection tests against a limited and not necessarily especially demanding family of malware examples, this represents a bit of a sea change.

"The Performance Testing Guidelines examines the myriad - and often subtle - complexities in conducting speed tests," said, Chief Research Officer of F-Secure. 

"It is very tempting to take a simplistic approach to measuring speed and footprint of an anti-virus program," said F-Secure's chief research officer,   Mikko Hypponen. "This document will help testers understand these issues and allow them to take the necessary steps to minimize them and take them into account," he said.

A second paper, Whole Product Testing, advocates that testers assess each program as a whole rather than simply measuring how well individual bits perform on their own.

"Too many current tests focus on individual technologies, such as 'on demand scans.' Only by testing all of a product's protection capabilities in a comprehensive test, can one provide a more realistic view of the security offered to computer users by contemporary security suites," said McAfee's Dr. Igor Muttik.

Some of this will sound obvious to the buyers of antivirus products, but the deeper issue is not only how to make antivirus testing more representative of real-world use, but how to do so without sacrificing consistency. It has also been argued that tests should aim to measure whether products reach baseline standards rather than worry about comparative results.

AMTSO was formed two years ago by a range of well-known antivirus vendors and security organisations as an influential talking shop for testing geeks. Since then, it has published a number of interesting papers on how antivirus testing could be improved.

Beyond its meetings, however, progress has been slow. Antivirus companies still routinely quote the detection rates of their products against static collections of malware such as the well-known Wildlist.

Five Filters featured article: Headshot - Propaganda, State Religion and the Attack On the Gaza Peace Flotilla. Available tools: PDF Newspaper, Full Text RSS, Term Extraction.

New standards could guide antivirus testing

Posted: 15 Jun 2010 01:19 AM PDT

A coalition of security companies and researchers has agreed on guidelines for how security software products should be tested, which may help put an end to long-running disputes about different testing methodologies.

Two sets of guidelines covering principles for testing security software for performance and testing entire security suites were adopted by the Anti-Malware Testing Standards Organization (AMTSO) at its latest meeting in Helsinki.

The guidelines are part of a series of documents on AMTSO's website aimed at introducing a set of standards broadly agreed through the industry as appropriate for ranking the effectiveness of security software.

Security companies have bickered over tests run by organisations such as AV-Test.org and Virus Bulletin and others, which regularly test and rank security software.

Among the contentious issues are the age of the malicious software samples used to test antivirus programs. Companies that failed to detect certain kinds of malware have argued in the past that the samples weren't threats any more and that they had removed the signatures from their databases.

Other companies have countered that their failed test wasn't accurate since other technologies in their software would have stopped the particular threat. Some antivirus testing programs perform static testing, where an antivirus engine is run against a set of samples.

But many security companies have incorporated other complex ways to detect malware. One such method, known as behavioural detection, checks to see how malware behaves on a computer.

The new guidelines are voluntary, but many testing organisations have agreed to go along with them, such as Virus Bulletin, AV-Comparatives, West Coast Labs and ICSA Labs, said David Harley, an AMTSO board member and director of malware intelligence for ESET. AV-Test.org will also use the guidelines, according to Andreas Marx, who founded the company.

"We're just trying to get people to think harder their methodologies so that they actually make sense," Harley said. "It doesn't mean you can't do things different ways, it just means you have to try and conform to a rationality."

Virus Bulletin has contributed to the guidelines covering performance testing, said John Hawes, technical consultant and test team director.

"We've already started implementing some of the ideas developed while discussing and designing this document, with some major expansions to the performance data we report in our comparatives in recent months and more improvements on the way," Hawes said. "We're also hard at work developing a new style of test which will allow us to measure the full range of features in many of today's security solutions."

The guidelines may not entirely end the feuding between security companies and testing organisations but will better inform those who write reviews of security software, said Stuart Taylor, manager of Sophos' threat lab and chairman of the board for AMTSO.

"What we are trying to do is get everyone involved to think more widely about decisions made in designing tests," Taylor said. "In this case we are talking about testing a whole product and not just part of a product, which can be very misleading, and also about measuring performance where it is so easy to create a performance test that is really just meaningless."

AMTSO has reached agreement with many security companies, but the nonprofit organisation is still viewed with some suspicion by other consumer organisations outside of the security industry that do security software testing, Harley said.

AMTSO's membership comprises mostly vendors, but the organization will work to communicate that the standards that have been developed are sound and can be trusted to provide more accurate test results, Harley said. "That's something we are going to have to work on," Harley said.

See also:

PC security advice

Five Filters featured article: Headshot - Propaganda, State Religion and the Attack On the Gaza Peace Flotilla. Available tools: PDF Newspaper, Full Text RSS, Term Extraction.

Guidelines released for antivirus tests

Posted: 14 Jun 2010 12:10 PM PDT

IDG News Service - A coalition of security companies and researchers have agreed on guidelines for how security software products should be tested, which may help put an end to long-running disputes about different testing methodologies.

Two sets of guidelines covering principles for testing security software for performance and testing entire security suites were adopted by the Anti-Malware Testing Standards Organization (AMTSO) at its latest meeting in Helsinki.

The guidelines are part of a series of documents on AMTSO's Web site aimed at introducing a set of standards broadly agreed through the industry as appropriate for ranking the effectiveness of security software.

Security companies have bickered over tests run by organizations such as AV-Test.org and Virus Bulletin and others, which regularly test and rank security software.

Among the contentious issues are the age of the malicious software samples used to test antivirus programs. Companies that failed to detect certain kinds of malware have argued in the past that the samples weren't threats any more and that they had removed the signatures from their databases.

Other companies have countered that their failed test wasn't accurate since other technologies in their software would have stopped the particular threat. Some antivirus testing programs perform static testing, where an antivirus engine is run against a set of samples.

But many security companies have incorporated other complex ways to detect malware. One such method, known as behavioral detection, checks to see how malware behaves on a computer.

The new guidelines are voluntary, but many testing organizations have agreed to go along with them, such as Virus Bulletin, AV-Comparatives, West Coast Labs and ICSA Labs, said David Harley, an AMTSO board member and director of malware intelligence for ESET. AV-Test.org will also use the guidelines, according to Andreas Marx, who founded the company.

"We're just trying to get people to think harder their methodologies so that they actually make sense," Harley said. "It doesn't mean you can't do things different ways, it just means you have to try and conform to a rationality."

Virus Bulletin has contributed to the guidelines covering performance testing, said John Hawes, technical consultant and test team director.

"We've already started implementing some of the ideas developed while discussing and designing this document, with some major expansions to the performance data we report in our comparatives in recent months and more improvements on the way," Hawes said. "We're also hard at work developing a new style of test which will allow us to measure the full range of features in many of today's security solutions."

AMTSO has reached agreement with many security companies, but the nonprofit organization is still viewed with some suspicion by other consumer organizations outside of the security industry that do security software testing, Harley said.

AMTSO's membership comprises mostly vendors, but the organization will work to communicate that the standards that have been developed are sound and can be trusted to provide more accurate test results, Harley said. "That's something we are going to have to work on," Harley said.

Send news tips and comments to jeremy_kirk@idg.com

Five Filters featured article: Headshot - Propaganda, State Religion and the Attack On the Gaza Peace Flotilla. Available tools: PDF Newspaper, Full Text RSS, Term Extraction.

AVG brings LinkScanner malware detector to the Mac

Posted: 14 Jun 2010 12:05 AM PDT

A company that provides antivirus offerings for PC users is bringing a free link checker to the Mac.

AVG Technologies plans to unveil a Mac version of AVG LinkScanner, a free download that scans Website links for potential threats. AVG says its LinkScanner application checks Web pages in real-time, posting a warning to users if the software finds a Website that could pose potential problems.

"Every single time you click on a link, any time you're accessing information on the Internet, we scan it, looking for malware and phishing attempts," J.R. Smith, CEO of AVG, told Macworld.

AVG's software doesn't rely on blacklisted sites that have been flagged for potential harmful content. Rather, the application uses real-time checking to alert users to potentially problematic sites. The company relies heavily on a large user base—tens of millions of users, according to AVG—to ensure users are surfing to secure Web pages.

Mac users have lived largely free of malware and viruses, as malware creators have focused on attacking more widely used platforms. That figures to change as the Mac becomes a more widely used system, thanks to the growth of both OS X and Apple's mobile iOS. But the real threat is posed by the growth of social networking sites that increase the risk Web surfers could fall prey to phishing and other malicious attacks.

"We're not here screaming that Macs are vulnerable," said Smith, adding that like any platform the Mac OS has its vulnerabilities. Rather, Smith continues, the threats posed by attacks via social networks threaten users regardless of platform. "We're protecting people in some cases from themselves."

Look at the increasing prominence of shortened URLs, Smith said. Savvy Web surfers used to be able to look at a URL and spot a potential phishing attempt; condensed URLs make that harder to do. "You can't see [the full URL]," Smith added. "You can't be your own policeman."

The free download of AVG LinkScanner is designed for Mac OS X 10.5 and 10.6; it runs on Mozilla Firefox 3.x and later as well as Safari 3.x and later, including the newly released version 5 of Apple's Web browser.

Five Filters featured article: Headshot - Propaganda, State Religion and the Attack On the Gaza Peace Flotilla. Available tools: PDF Newspaper, Full Text RSS, Term Extraction.
Posted by at

0 comments:

Post a Comment

Subscribe to: Post Comments (Atom)