If you are using a Finite Sphere there is a setting "Visualize Finite Dome", try setting that to off. I had this problem before, but couldn't remember what I did to fix it, so just spent the last 45 minutes trying to recreate it. Turning on that setting was the only time I was able to recreate that effect so far.

Edited to add, the attached photo is the Finite Sphere set to a radius of 1.0, you can see how it gives the rainbow effect. I think it's so you have more control over the size of elements in the HDRI.