I can't remember where I read this now, but I understand that firefly just plain old doesn't like displacements in animations.  It recalculates the shadows at every frame and does so without consistency.  There is no known way around it from what I understand.

This shadow issue isn't only in firefly from what I remember reading aswell.  A number of rendering engines have similiar problems.  I think Carrara tries to get around it by allowing you to select for it to only calculate shadows once, but if your camera angle or light source changes, it will again have these troubles.

If I can find more indepth info on it I'll post a link.  It is really a huge let down.  It isn't relegated only to displacement maps, either.  A lot of shader node stuff has known bugs like that when used for animation.  It drives me insane!!