I would suggest breaking up your building into parts and applying the deformer to each part separately. That way you can time which part of the building is blown apart in an order that suggest good timing to you. I would break up the model in the meshmodeler into non-uniform pieces. perhaps the comet could blast through the buildings mid section first and then the top would crumble down after the comet goes through it. I saw something about the movie starship trooper in which the animator added extra rubble to make the explosion of a ship look more intense. This was done by having two spaceships existing in the same space explode instead of one. the timing for each explosion was slightly offsets. In the movie independence day they went further and had actual furniture and stuff flying out of the exploding white house. good luck