The "square" prop are two polygons, the "one sided square" is only one polygon. The problem is that both polygons in the square prop are in exactly the same place (they share the four corner vertices), only facing in opposite directions (opposite normals). FireFly is trying to render both polygons, and due to the limited precision of 64bit floating point calculations, you get artifacts.