I must agree you're right about photoshop handling huge layers better than Vue handling texture maps. There has been so much grief about this problem with Vue that I'm confident next release will be better in this field. But for now, that's all we can do.

I'm surprised that your scnee creates such problems. At first I thought you had 4 human figures in the scene, but only 1 with 3 kitties shouldn't lead to this problem. Did you check "collapse identical materials" when importing your pz3?

Also, I think you get the OOM because you want to render at such a big resolution. In this case, Vue is not the culprit, it's not enough RAM. I've had this a few times, and Vue told me that it required more ram than I had to render my scene. I said render anyway, but got the OOM. Is this what happens with your scene?

If you have another computer around, even an older one, use Hypervue network rendering.

If not, try and render to disk instead of screen. This works better, since Vue doesn't need to show the computation on screen, it needs less resources.