Update:

I played with shadow min bias and it helped a bit. But the sudden strange shadowing catches me at some frames. I try different things and it disappear sometimes but i can't pinpoint the direct problem and at an animation frame, i can't move everything for just some frames..