The biggest problem with host-based security has always been what happens when your protection fails. And yes, all traditional host-based protections will have the potential for failure, especially when you consider that it's generally easy to trick users into installing bad stuff.
But when your protection fails, and the bad guy has a foothold on your machine, you're in a very bad spot. The bad guy generally can, if he puts in the effort, disable any security product you are running on your machine. So even if your product of choice eventually does protect against a threat, it could be too late for you.
Often, when bad guys disable things, they don't disable all security products. So if you're using a good product that isn't as popular, that can leave you better off than running something by a brand name that every bad guy in the world will want to target.
The security industry hasn't been able to overcome this problem in the past 15 years. But, I expect to see that change, because there is a relatively easy solution to this problem... virtualization technology.
With virtualization, you can run one operating system on another. But, it doesn't have to be running Windows on MacOS X. I mean, you don't have to be running two graphical user interfaces on top of each other.
Instead, you could have a very small operating system that the user typically doesn't see. Just for the sake of discussion, let's call it SecureOS. This operating system would have your desktop operating system of choice running in it (again, for discussion, I'll assume you're running Windows). In an ideal world, your security software could run inside SecureOS and be able to protect Windows. And, if Windows got infected, only Windows would suffer. That's because, from the Windows point of view, it is running on some machine, and doesn't see that it really at the mercy of SecureOS.
If a bad guy breaks onto your machine without getting detected by your anti-virus or other host security software, he hasn't broken on to SecureOS. So the AV software can still make a network connection to get updates that would allow it to clean up your infected Windows installation after the fact. Right now, once Windows is infected, the only real surefire way to make sure your AV (or other host security product) is doing its job is to download a boot-time scanner on a known safe machine, and then run it when rebooting the potentially infected computer. Even just rebooting is too much of a PITA (pain in the neck) for users. With virtualization, the user doesn't have to do anything, it can all happen automatically in the background.
This scheme is technically feasible, though it is complicated to build. Particularly, moving all of your host security out Windows would require a lot of work, because host security products generally rely on parts of Windows to run well, even if just the file system. However, there is a middle ground, where some security code can run inside the OS if necessary, talking to the SecureOS. And, the SecureOS would monitor the integrity of the security code within Windows, so that it could detect when the bad guys are tampering with it. Plus, the communications and update channel would always want to live outside of Windows.
Additionally, there's the problem of what happens when SecureOS is itself not so secure. That is, if the virtualization platform has a security problem, it could be possible for a bad guy to break onto it if they break onto Windows. However, there's far less of an "attack surface" here. That means there are far fewer doors and windows to secure than normal, which generally would make it less risky.
The same kind of virtualization technology can provide great protection for your personal data. Your personal data (particularly credit card numbers, social security numbers and things like your mother's maiden name) need never leave SecureOS, unless it is explicitly encrypted for the company you want to do business with. Your Windows machine would just be relaying some encrypted data, and wouldn't be able to see your personal information. Clearly, there has to be a way to enter and change your personal data, in such a way that the user won't be confused as to whether they are secure or not. At the end of the day, that's a big usability problem, but there is no real underlying technological issue.
One of the key requirements for a system like this is true boundaries between operating systems, with limited communication interfaces (i.e., a small attack service). There are solutions where things run "semi-virtualized" (meaning, inside the operating system, but with a lot of trickery to try to make programs unable to see other programs running). I've said before that such solutions have some usability problems that will make it extremely difficult for these solutions to be any kind of silver bullet. But, they are also going to have a much bigger attack surface, since they really are running inside the operating system.
But, you could do this kind of thing at the hardware or BIOS level. Hardware level support for virtualization helps bring this kind of technology closer to reality. And, if Apple wants a real leg up over Microsoft, they should put this kind of technology into the Open Firmware, which they control (in contrast, Microsoft does not control the firmware for the machines on which it runs, which makes its virtualization efforts even more important).
The major issue with this application of virtualization technology is the cost. Host security vendors would have to do a lot of engineering work to retool their technologies. Then, you'd have to get customers to consent to the virtualization. On new hardware with direct virtualization support, there wouldn't necessarily be much or any performance impact. But, legacy computers would certainly have a big issue. Plus, even for existing hardware that does support virtualization technology, we'd have to wrestle with people migrating their non-virtualized OS to a virtualized setup.
Nonetheless, I think virtualization is the long-term future of host security. Your main OS will eventually be virtualized as a "guest" operating system. Security services will start migrating into the "host" OS, which, I hope, will be a small, dedicated piece of technology.
If we can make it this far, the advantage in the never-ending war between security vendors and the bad guys will, for the first time, shift to the security vendors. We'll be able to make pretty reasonable security guarantees under reasonable assumptions. No longer will we have to cross our fingers and hope the bad guys don't get administrative privileges. We'll just have to hope they don't find a way to break onto SecureOS, something that, as a specific and constrained piece of technology, will be fundamentally easier to secure than an entire operating system.