Have you "resize and center imported objects" off in the preferences? Maybe this simple thing can do the trick here. I fear the arrow keys are not precise enough. But you can try out the little arrows to the right of the numerical input. They are MUCH finer. A little trick maybe. Define a block in the modelling software that contains all objects of the scene. Then export always this block with the part you need in Vue and delete then this block. The position and size should be then correct for all parts. One of the tricks from ancient Vue times where there was no option to switch off center and resize.

One day your ship comes in - but you're at the airport.