Ah, I thought you meant the roughness of the actual edges of the displacement, not the banding on the side.

Sadly, thre's nothing that can be done about that... if you think about your UV map which the textures are being mapped onto, it doesn't have any allowance for those side pieces, since they're not actually geometry. So the colour texture map is sread right along the 'extruded' area of the displacement.

This also occurs at low levels of displacement, by the way, but it's just not so obvious because of the smaller areas of surface being distorted.

JonTheCelt