Use "phantom" prop in the middle.
I had similar problems.

Loaded any low poly primitive (for example low res ball) set ball material full transparent. Set your prop (sword, gun or sth) as a balls parent. Fix ball in correct place (on the handle?) but not change any scale. Later set ball as a hand parent.

In case you changed figure scale but your prop is still 100% scale, set figure scaling parameter on ball.

On your render ball will be invisible.

Maybe not in English but I hope clear ;)