Stop using the magnifying glass to zoom and the hand to pan. They are intended to be used with the orthogonal views (top, front, right, ect...) Use the camera controls to move the director and camera views.

When you use the hand to pan, you are not moving the camera. You are moving the viewpoint, like an out of body experience. The zoom button work by changing the viewpoint's scale, again, they don't actually do anything to the camera.

The giant arrows above the render button moves the camera, so use them when you want to move the camera.

{letting you do something wrong isn't a bug, it's a feature}
{The camera and the directors view are the same thing with two exceptions: the camera is a selectable object in a scene, and the director will let you rotate around an object.}