Actually, what you COULD do is make a static shot (locked down camera, still frame 3D matte - even animated environment but a still camera) - render it wider to give yourself some room on all four sides as mentioned before, composite your characters in the environment, and create the handheld jitter in AE. That'll save you the headache, and you still get the handheld feel.