The textures will slow things down as will the light (though the intensity of them will make no difference)  Its not really a polygon issue if you are using bryce booleans as Bryster has pointed out, it takes some beating.

http://www.renderosity.com/mod/gallery/index.php?image_id=1638165&username=Rochr&member&np Rochr with another stunner.

 The bryce boolean works very well for distance, with a square edge. If you start trying to boolean chamfer, bevel, etc it gets somewhat complicated and this is where the mesh modeling should come in. Its about how much detail you need.