Any admin worth his salt knows messing around with an operating system kernel is a bad thing to do. It's bad for system reliability, it's bad for security, and it's bad for application compatibility. The good news is the days of malware writers and security software vendors patching the kernel to their hearts' content are numbered.
Consider for a moment what patching the kernel really means. The kernel is the lowest level of the operating system. It enables software and hardware to communicate and is responsible for memory management, application and process launching, and other basic tasks. Its integrity is vital for a server running applications on which the organization relies.
In spite of this, patching the kernel modifying pieces of kernel code is a practice used by many software vendors, most notably (but not exclusively) vendors of security products, to make their products work.
Unfortunately, what is good for the goose is also good for the gander, and malware writers also use it as a way to get deep into a system, sometimes to create rootkits that can unlink themselves from the file system and effectively make themselves undetectable.
In the 32-bit world, Microsoft has always frowned on kernel patching, but in practice it has turned a blind eye because it would be impossible to forbid it without breaking countless important existing applications.
But the company has taken a firmer line with its 64-bit operating systems, introducing Kernel Patch Protection (also known as PatchGuard) in Windows Server 2003 SP1. PatchGuard doesn't actually prevent kernel patching, but rather caches known, good copies or checksums of critical tables and files (e.g., ntoskrnl.exe, ndis.sys and hal.dll), checks every five to 10 minutes to see if any of these parts of the kernel have been modified, and shuts down the system if it detects that they have.
However, PatchGuard isn't particularly useful at the moment because very few organizations run 64-bit server operating systems. This is partly because of the lack of 64-bit driver and application support, which in turn is due to lack of adoption. It's a bit of a vicious circle in other words.
That PatchGuard is hackable as it stands is unimportant. It's a signal that kernel patching is unacceptable, and vendors must get serious and find other ways to make their software work.
But this will change fairly rapidly. SQL Server 2005 was the first major application that Microsoft released with a 64-bit version, and Server Longhorn will ship later this year in both 32- and 64-bit versions. Bob Muglia, Microsoft's server and tools vice president, is on record as saying that this will be the last time Windows Server ships a 32-bit version future releases will be 64-bit, only. And Exchange Server 2007 will be available only as a 64-bit application from the beginning.
So it's pretty clear 64-bit server computing is going to become the norm in the near future, and as that happens, Microsoft will take the opportunity to rid itself of the patch-happy 32-bit kernel and actively enforce its policy that the kernel should not be patched using PatchGuard as its policeman.
Great news for anyone operating 64-bit servers as it ushers in a new era of secure and reliable computing? Sadly not. Because, inevitably, there's a catch. PatchGuard relies on security through obscurity, using functions that have been deliberately misnamed and general code obfuscation. Thus, if you can figure out how it works which inevitably people have then you can bypass it. In fact, sample code is already on the Internet that claims to stop PatchGuard's timer so the kernel is no longer regularly checked for modifications, effectively turning PatchGuard off. And researchers at the security software company Authentium also claim to have found a way to circumvent PatchGuard.
Whether PatchGuard is itself patched to prevent these workarounds is irrelevant. The point is that PatchGuard is bound to be cracked from time to time. So whereas before everyone had access to the kernel, it's now principally the bad guys that will try to modify it: Responsible vendors that previously patched it will refrain from doing so, as Microsoft prohibits it and because there's a risk that the applications will break whenever Microsoft itself modifies the kernel.
The real problem here is much deeper, and one Microsoft is aware of and cannot get around. It's simply not possible to protect the kernel using code running at the same privilege level as the code trying to subvert it. The only way to monitor and supervise the kernel effectively and ensure its integrity is to do so at a more privileged level. What this would entail (and has been hinted at by Microsoft staff working on kernel security) is locking down the kernel using hardware-assisted security technology: using a virtual machine hypervisor to run the kernel, and presumably the rest of the operating system.
Cast in this light, PatchGuard could be interpreted as an intermediate step between Microsoft's "patch the kernel as much as you like, but without our approval" approach to the 32-bit platform, and a kernel that is locked down and rock solid. That PatchGuard is hackable as it stands is unimportant. It's a signal that kernel patching is unacceptable, and vendors must get serious and find other ways to make their software work.
Microsoft has promised a set of APIs to help security vendors do this. When they will appear is anyone's guess.
If and when a move to a hypervisor-based kernel security system is put in place, legitimate software vendors will have had ample time to make their products work, and malware writers will have a much tougher time writing rootkits and other exploits to subvert 64-bit systems. The kernel, at last, may be secure. And not ahead of time.