Not sure about rotate vs pan, but one common "gotcha" with point-at is the
real direction of pointing.   The ball's EndPoint is what does the pointing.
And the default ball has its EndPoint on top.  Thus, when you first set it
to point at another prop, the ball will try to flop over (rotate on X or Z) instead
of rotating around Y as you might expect.  

This "gotcha" comes up most often with a figure's head ...  the default
EndPoint of most heads is on top, not in front where where the eyes are. 

You can always move the EndPoint (red cross-hairs) with the Joint Editor.

My python page
My ShareCG freebies