Thanks for your replies. I have changed shadows parameters and adjusted the "sun position" as it was advised. And I changed colour of the light imitating the sun so as to match warmer look of the rest of the photo. The main point is, of course, that the same photo is for background and for IBL and it does not reflect the whole environment so it must be corrected by additional lights and simultaneously it decreases usefulness of IBL in this case.