I think it's caused by "Smooth Polygons"

Turn it off in your properties for the background, or in your render settings.

 

I have no idea why that fixes it, but it does for me. I don't think "Smooth Polygons" is all it's cracked up to be.