I seem to remember that PS has a 100 layer limit. If you aren't getting anywhere near that then more than likely it's a hardware issue such as limited memory. There are other programs that allow more than that but they don't have the animation capabilities of PS. One option would be to use Adobe's Premier Pro to create your animations. You can import your .psd files and still use the separate layers.