Displacement and bump maps can increase render time significantly.  Consider not using such mapping on distant objects or landscapes.  Medium distant objects, use bump maps, and anything close-up would need to be carefully evaluated.  If the scene contains full on normals, use bump; objects at perspective angles would require displacement.