a lot of people will tell you to turn down shadows on lights to achieve realistic results.  i find that if you use IBL in conjunction with directional lights, you'll acheive much greater realism.  if your shadows are too dark, use IBL to raise your ambient light.