It almost sounds like the scene might be texture intensive and a possibility that the displacement setting might be slowing down the render.

Check your material room settings for your scene and determine if a displacement map is needed.  With such a detail level on your background, I wouldn't be surprised that there is one.  While in the material room, change the Texture Quality to None on the Image_Map.  In your render settings, uncheck displacement and re-render to determine how fast is your turnover.  If no difference, then systematically make each part of your scene invisible and again render.  Doing this will isolate where the problem is occurring.  A highly detailed texture map will cause your system to slow down considerably.  Happened to me once on a slower machine.