Sometimes it is the shadow map that does it.
Try to use only ray tracing shadows and not depth map shadows.
I don't know what's the difference in the results because I never could render a single image with the depth map option on. LOL
On the other hand, I have not tried again since SR2. May be they solved the problem ?
I'd be interested to know if it works.
Compiler