The other thing to remember on IBL is to keep the intensity low. YMMV, but I've found that unless I keep the IBL intensity down in the 10-25% range, you end up with a very flat image w/ almost no shadow definition, but having that little IBL helps tremendously. Note, I do use AO on all my IBL lights and have had very little problems with it. Check Refuge to see a complex scene w/ IBL/AO enabled. I should say, however, that I'm running a 3.2GHz w/ 4GB RAM, and I'm willing to wait a day for a high-res render.

(Note, the 10-25% intensity if for a pure-white IBL. With a darker image, you can raise the intensity. Before I understood this, I generated this where the colors are thoroughly killed by IBL at 100% pure-white intensity.)