Last week, the Internet Storm Center, a group of security professionals that track threats on the Net, flagged a flaw in how a common Microsoft Windows utility and several anti-spyware utilities detect system changes made by malicious software. By using long names for registry keys, spyware programs could, in a simple way, hide from such utilities yet still force the system to run the malicious program every time the compromised computer starts up.
Already, some spyware authors seem to be playing with the rudimentary technique to try and hide their programs, said Tom Liston, a handler for the Internet Storm Center and a network security consultant for Intelguardians.
"We have seen indications that someone is trying this technique out," Liston said. "Basically, we have seen code that is stuffing a key in the registry with a huge length. Yet, the author still doesn't have it working."
A Microsoft representative said that the company is investigating the report, but does not consider the problem an operating system flaw.
"Our early analysis indicates that this attempt to bypass these features is not a software security vulnerability, but a function within the operating system that could be misused," the company said in a statement. "Microsoft is reviewing the report to determine further details and whether there is any potential impact for customers and will provide appropriate customer guidance if necessary."
The potential threat comes as more malicious software has started to use various techniques to attempt to escape detection. Some attackers have merely used targeted Trojan horses and customized spyware to evade defensive software. Such techniques are believed to be the reason that a sustained attack on US and UK government agencies and industry has largely gone unnoticed.
The creators of more advanced rootkits - software designed to stealthily and completely compromise a system -are starting to add memory-hiding to their bag of tricks, said Greg Hoglund, CEO of software analysis firm HBGary and author of the recently published ROOTKITS: Subverting the Windows Kernel. Hoglund discussed the technique at the Black Hat Security Briefings and DEF CON hacker convention in July.
"Spyware is the biggest problem right now, and the people that are writing it are starting to get a clue, and that's a scary trend," Hoglund said.
The potential for hiding the execution of programs using overly long registry keys, on the other hand, is much smaller, because Microsoft and affected security software vendors will likely fix the affected utilities soon, he said.
"None of the people that I know who are writing rootkits would not use this method to hide the key," he said.
The technique involves using a registry key whose name is longer than 256 bytes. The Windows Registry holds important system data, including what programs to run at startup. The long key and any of its subkeys are not seen by the affected utilities, but can be read by the system just fine. By using the technique, a malicious program could run every time a computer is started, but keep its execution a secret from the utilities, the Internet Storm Center said.
Programs that apparently cannot detect malicious software using the registry technique include AdAware, Microsoft's Anti-spyware Beta, Norton SystemWorks 2003 Pro, Registry Explorer and WinDoctor, according to an ISC posting. The Internet Storm Center could not create a definitive list, because the programs apparently acted differently on non-English versions of Windows.
Symantec, the creator of the Norton brand of system utilities, is the owner of SecurityFocus.
The technique works against Microsoft's RegEdit utility, but other system utilities, such as Reg.exe and the Microsoft Configuration Editor, are not affected, the software giant stated.
The developers of the affected programs are already working on fixes for their products. If Microsoft fixes the RegEdit issue, it may also solve the issue for other vendors, ISC's Liston said.
"It should be something that Microsoft should be able to address in the next monthly update," he said. "There are a lot of programs out there that do things like look at the registry that are affected by this."
While the technique may only be useful for a limited time, spyware authors will likely incorporate it into their programs, said Joe Stewart, senior researcher for security firm Lurhq. Another major threat, bot software, will likely not use the technique, he said.
"Spyware usually does a much better job of hiding itself in the registry than bot software," Stewart said. "Even though bots are often used for spyware, adware or other financially motivated activity, they are programmed as if they were just general-purpose utilities - for some reason they almost always go with the tried-and-true 'Run' registry key."
System integrity checkers and security software should attempt to detect more surreptitious techniques like registry hiding, added HBGary's Hoglund.
Hoglund and two other researchers have modified a common rootkit using techniques, ironically, taken from a way of protecting against buffer overflows, a common software flaw. The memory cloaking allows a rootkit to run its own code while hiding that code from detection by the operating system.
Such techniques will likely become common in malicious software in the near future, he said. Hoglund stressed that security software makers have to start thinking more like attackers and adding more advanced detection capabilities to their products.
"If your security tools aren't also using rootkit-like techniques, then they can be subverted easier," he said.