Hi

Over the past week or so, as debate rages over the potential and merits of P7, there have been a lot of comments about how poor at managing memory Poser is.

Can someone, in not-too-technical language, explain what the problem is and why it is so?

I've been playing (a little) with D|S and it seems to be able to load and navigate the menus much faster than P6, and its rendering is much faster too. Why is this?

TIA
Martin

Basically, the 1st problem is that Poser was initially written for the original Mac. Then Ported over to Win 3.1. So you have all the issues that any application had spanning those two very different operating environments. 2nd problem is that Poser was conceived as a virtual artist's woody; so from an architectural standpoint, it was never meant to do a fraction of the things that people try to do with it. 3rd Problem. With Poser 4, the actual hardware people had started having the resources to handle things like multiple figures...but the code base was firmly from the 'old days'. The P4 .bum file is actually a multicolor displacement map, not a grayscale jpg bump map (and ate memory accordingly). 4th problem. The .com collapse. P5 was -supposed- to include a rewrite of the core code, to take advantage of the newer hardware; but it was caught in the implosion, and a lot of things intended never came to be. So while it was tweaked a bit, the memory management system was still the same one from the Mac/Dos/Win era. Meshes were getting denser, skin textures and transmaps were getting larger and larger, more morphs (and a correspondingly larger cr2 file) were being added. For its initial purpose, the memory manager was sufficient. For the purposes it was being put to, a virtual artist studio and animation system, it was showing its age with every passing day. DS is newer code, as is the rendering engine they liscenced. It also doesn't have anywhere near the feature set of P6, and those features slow any app down (if someone ever writes a dynamic cloth and strand based hair plugin for DS, it will impact its performance).

(While I could be wrong on this) It seems to me the main reason Poser's memory management isn't that great is because the program was originally developed when there was far less memory computers could use (What was the average memory on a computer 10 years ago?).

And since the original developers probably didn't think they'd have to worry about running the program on a computer with 2 gigabytes of memory, they didn't really design the memory management portion to utilize that much, either not thinking there would be computers with that much memory, or figuring that it would be handled as soon as it became an issue.

Really, what use would it be for a program today built to have a memory access limit of 20 Gig? No home computer can come close to that at present, so better off keeping things within current memory limits.

I have written E-Frontier several times since the announcement of Poser 7 about the compatibility with Windows XP/Vista 64 bit. The main reason for this is the ability to utilize more than 4 gigabytes of ram.  Their response, silence, total Death Valley silence.  I am so sick of the "Out of Memory" message I continually get with Poser6/5/4.  E-Frontier if your are listening to your user base, Give me 64 bit O/S compatability!

Never argue with an idiot, they will only drag you down to their level and beat you with their expertise.

The poor memory management claims come from those people who don't understand that they can't load 20 GB of image, program and object data into an application that uses the regular Windows32 memory management. They want Poser to have its own memory handling - like that's something every application has... Oh, and the render speed difference you mention hasn't anything to do with memory management but rather the different renderers used.

I think Dale B. has given a great summary of the main problems. I don't think thever wrote a Windows version from scratch. In P4 at least, you can find traces of a cross-platform product they apparently used to translate the original Mac code to run under Windows. Such a conversion is never as effecient as a rewrite. As for 64 bit, there are plenty of 32 bit 3D applications that work fine. They could probably cobble something that would run 64 but unless they design it with modern memory management from the ground up, don't expect any miracles.

I used to get the Out of memory message all the time, but since having properly installed the updates(Befoer opening a Setup)  each version of Poser I recieved the messages in then worked beautifullly!

from the limited amount of training I've done with 64-bit OS (1 flinkin' course, that was actually a recording on line), 64 bit will handle 32 bit apps, but I seem to remember it doesn't handle 16-bit and donwn.

I was working on PC's 10 years ago, and if memory (mine) serves, 16-32 meg was what was going in servers, and 4 meg wasn't too shabby for PC's. Of course, Win 95 was brand new (called WinEver by most folks because it was delayed so many times), and a goodly number of folks were still using DOS..;)

Quote - Really, what use would it be for a program today built to have a memory access limit of 20 Gig? No home computer can come close to that at present, so better off keeping things within current memory limits.

Lots of use!  Windows XP Pro 64-bit (and other OSs) can address up to 15 Exa-bytes of memory (that's far bigger than a GB or TB).   Most home systems (hardware) can't handle 20GB, but there are quite a few that CAN handle 8 or 16 - NOW.  And I think that there might be a few that can handle 32GB.

Cinema 4D's 64-bit can handle as much memory as you can throw at it within the 64-bit range.  That means it will be 'limitless' for the next ten to twenty years.

Sorry, with the size of OSs and applications and the memory that they consume and the fact that you can get 500GB harddrives, it's about time that memory started ramping up and quickly.  Better to expand the current memory range and allow people to do things that currently result in "Out of Memory" errors.  2GB brick walls are becoming more and more frequent - most especially in the 3D CGI arena.

I think I have to debunk a few of these myths here :)

Quote - Basically, the 1st problem is that Poser was initially written for the original Mac. Then Ported over to Win 3.1. So you have all the issues that any application had spanning those two very different operating environments.

Poser was never ported to Windows 3.1. If you happen to have a Poser 1 box around, it states Windows 95 as minimum requirement. The problems of covering two different operating systems are not what one might think they are - they're mostly UI differences (like Window focus, menu handling etc) and have barely anything to do with memory management. Besides, we're in good company - most other 3d programs out there didn't even originate on Windows or the Mac to begin with. Cinema4D and Lightwave were originally Amiga programs, Maya comes from IRIX. > Quote - 2nd problem is that Poser was conceived as a virtual artist's woody; so from an architectural standpoint, it was never meant to do a fraction of the things that people try to do with it.

Be aware that programs are not buildings. While you can't change a building's fundaments without tearing the whole thing down, you can change a program's fundaments sometimes without even touching the upper levels. Look at Windows 95 vs Windows NT 4: Almost identical UI, many programs run on both systems, but fundamental layers of these systems are vastly different. > Quote - The P4 .bum file is actually a multicolor displacement map, not a grayscale jpg bump map (and ate memory accordingly).

.bum files are actually normal maps. Very hip right now for games, and are used there for the main reason why they were used in P4: They require less computation to render (no derivatives), compared to gray scale bumps. > Quote - So while it was tweaked a bit, the memory management system was still the same one from the Mac/Dos/Win era.

Now here's a fun fact: Memory management, as seen from an application, hasn't changed over the last 20 years or so, it's mostly the same as it was on the first Unix systems. You call malloc() or new when you need memory, you call free() or delete when you don't need the memory any more. That's all there is (garbage collectors or reference counters are just ways of automating those calls, and the old MacOS NewPtr() and DisposePtr() are just malloc/free wrappers on OS X). Where the operating system gets (or doesn't get) that memory from (real RAM, disk space, space alien mutant ninja turtles jelly beans) is none of the application's concern. So where do Poser 5 and 6 get the out of memory message from? Whenever it the OS rejects such a memory request. Usually because all of Poser's address space is taken by a dozen of SAT filtered textures, and that is something that happens to other, more epxensive applications as well when you try the same thing there. I won't tell you about Poser 7...yet.

LOL, Stewer! - You're right (with a couple of small exceptions..)

Exception 1. Poser's "Out of memory" error is (IIRC) the default error message given whenever some error occurs that the programmer hadn't counted on (Get a "Failure" return from a function? oops - no indication of what that means - better give an "out-of-memory" error...)

Exception 2. The malloc() / free() stuff works well as long as it is mainly stack-based (ie, on a last-in, first-out basis). But Poser tends to hang onto stuff that is no longer needed (for faster re-use, should it be asked for). Also, I am fully aware that the free() function will combine adjascent free areas. But it doesn't take too many large textures to be loaded/removed to get memory fragmentation - the memory is still there, but scattered around. And textures need contiguuous space.... so Poser ends up with no single area large enough to hold the texture...

The only way around the problem is to write a completely new memory-handling module that will not just garbage-collect, but de-fragment at the same time (which involves moving all the used stuff so that it is in one single area at the front, and all the unused stuff to the end - or vice-versa). And changing every reference within the entire data area to point to the new position of the moved data. Not a trivial task!. And that, alas, slows up response time......

But I expect you knew that anyway. I'm just showing off (again)...
Cheers anyway (about to run and hide under my bedclothes),

Diolma

I was trying to put it at least in the proximity of layman's terms ;) Of course we could start ranting about Windows' heap fragmentation, caching strategies, garbage collection, block allocation, memory pools etc but that would rather confuse than educate 99% of the 'rosity members.

....but that would rather confuse than educate 99% of the 'rosity members.

I think that already 99.5% is confused; malloc() and free() are common terms for professional C or C++, programmers but not for the everyday Poser-artist. I'm not sure that Poser1 was released for windows95, as far as I know it was released in 1995 and came on  2 floppy-disks. I do have the original box, but I have to dig to find it. I don't care if E-frontier solves the memory-problem, If Poser7 needs a dual-core, 4 gig computer I'll just have to buy a dual-core 4gig computer. That's the way it goes and it will always be going. I've bought once a game which I couldn't play because my computer wasn't fast enough, so I stored the game on my bookshelf. For Poser7  I will make an exception...

Quote - I don't care if E-frontier solves the memory-problem, If Poser7 needs a dual-core, 4 gig computer I'll just have to buy a dual-core 4gig computer.

I'm pretty sure that the remaining four reasons to get Poser 7 will bring some pleasant surprises for even those who don't have a dual-core 4 gig computer :)

I agree to that... And now we wait...(famous dutch Heineken commercial)

"Exception 2. The malloc() / free() stuff works well as long as it is mainly stack-based (ie, on a last-in, first-out basis). But Poser tends to hang onto stuff that is no longer needed (for faster re-use, should it be asked for). Also, I am fully aware that the free() function will combine adjascent free areas. But it doesn't take too many large textures to be loaded/removed to get memory fragmentation - the memory is still there, but scattered around. And textures need contiguuous space.... so Poser ends up with no single area large enough to hold the texture..."

The memory management is not done by Poser, Vue, Maya, Photoshop and whatever application.
Memory management is done by the Operational System (Windows, Linux, OSX, etc)
Nobody writes a memory management for a program, unless you are someone rare that wants to make your own accesing the root kit and CPU core, and better do it in Assembler and not in C.
When you need memory the program make a request to the Operational System by means of malloc() or Kernel variants and when you don't need more this memory you issue a free() request to the OS.
The OS is responsable for keeping the memory unfragmented and all memory tasks, and Windows is very bad in administrating the memory. Windows was bad in 3.11, continue bad in 95, the same in 98, is in XP and will be in Vista, nothing changed, the problems still are the same and never were resolved.
The only thing wrong that programmers do with memory is that they often forget the free the memory that is not needed anymore and when the program exit. If the program doesn't free the memory, when you exit this application your computer will have less available memory, starting again this program it will allocate more new memory and when you exit the program again you have even less memory for your computer.
Multiply this by the number of other programs that have the same errors, then multiply by the number of times you used this programs and you will have after some hours of work that your computer has no more memory, so it crash and you need to restart your computer.
A healthy  practice is to restart the computer after some hours of work so it always will have all the memory available.
All this programmer error would be not so dramatic for a well written OS, every time a program ends the OS should release all the memory used by the program, but Windows don't do this, if the memory used is not freed by the program when it exits, the memory continue to be allocated to a program that doesn't exist anymore.
Poser 4 is not bad, is acceptable and only time to time let your computer without memory, but 3DSMax 2.5 was dramatic!!!!

The problem with Poser is that it use too much memory, who knows why, I cannot find any reason for this.
The second problem with Poser is that every texture you load Poser allocates memory for it, if you load a lot of textures, the textures are big or huge the result will be a lot of memory used.
Until now this is normal, if your scene uses a lot of big textures you will need a lot of memory.
The problen is when your scene has not too much textures, or maybe only few ones and you runout of memory.
The problem is, as I said before, that every time you load a texture memory is allocated for it, when you are creating a scene, even a simple scene with only one texture, you load many textures to see which one is better, you try different backgrounds, etc.
Your scene use only one texture, but you have experimented 50 textures until you found what you needed, now you have 50 textures in memory and not only one!
The only workaround for this, is once you found the texture, save your scene and close Poser.
Then open Poser again, load the scene and now you will have only one texture in memory and not 50!
The easy solution for this problem would be a simple buttom or menu in Poser "free unused textures", something simple to be done, but never was done.
Poser 7 will have??????......., maybe not..........

I just hope people will start using some of the Poser 5 features which will let them use smaller texturemaps, and give up on P4 compatibility that requires a texturemap that fills UV-space. Stewer, Diolma, you might know if it makes a difference: I'm talking about the Scale and Offset parameters that can be applied to an image-map node in the Materials Room. Various tricks I've seen suggest that the Firefly render engine is internally 16-bit, possibly with one bit for a sign-bit. That includes greyscale colour depth, as used by stonemason for displacement maps. But you have to work around Poser's limits on reading a bitmap.

Firefly is REYES is it not ? In which case like all other REYES renderers it should be rendering internally in float (or 32 bit per channel)

As for Poser's RAM usage, i'm with kawecki on this, since i also don't know what the hell is going on in the render. Either its not freeing the RAM properly, doesn't have a proper scene BSP in the first place or has some mem leak somewhere. Cause in both Lightwave and XSI (the 2 other apps i use with poser) i can load up 20 James high rez meshes with full 3k textures and the render does not cross 700-800 MB in RAM. In poser even 4-5 such figs start running into double that.

Hope they work on this in 7, cause throwing RAM or 64 bits at the problem is no solution. Even if Vista lets you cross the 4GB barrier, most non server boards in the market have hard limits on the amount of RAM they let you add. Many wont even run in dual channel mode after 4GB or will slow the RAM down to 333 MHz or even lower.. So adding more and more RAM is not as yet a very good option IMO.

Have you checked these threads for help on the memory issue?

http://www.renderosity.com/mod/forumpro/showthread.php?thread_id=2315646

http://www.renderosity.com/mod/forumpro/showthread.php?thread_id=2516735

Other big problem is that today's computers have a lot of memory, it can look contradictory, you need more memory, the computer has more memory and this is bad. Why?
In old Win 3.1 days when you had only 4M memory and 50M hard-disk you were playing some card game and suddenly your screen became corrupted and the computer crashed.
The computer worked normaly, but every time playing this same game the same story happened, so the conclusion was obvious, this game has some bug.
If you investigated a little with some tools you have discovered that the game was using Windows resources and not freeing them, so after some time of playing Windows runout of resources and crashed.
Now you have upgraded your computer and you have 16M RAM and a disk of 512M and you play again the same buggy game. Surprise you discover that nothing happens and you reach wrongly the conclussion that the game is OK and the problem was in your previous computer.
What happened?, the game is still buggy, it uses resources without freeing, but as you have more memory available you will need more time of playing for the computer runout of memory and you only play half an hour, not time enough.
You live happy with your new computer, you can play the game without crashing, but what you don't know is when you exit the game your computer has only 4M instead of 16M that makes other applications to run more slowly, or even crash the computer. Then you blame this other application until you upgrade your computer again and all works normaly.
With a modern computer with 2G RAM, this game never will crash the computer, you will need to be playing for monthes until it crash, the program is buggy but there's no way to realise that this software is wrong.
Having a huge amount of resources makes all the bugs become hidden, specialy the tiny bugs.
But the bugs continue to exist and continue degrading the performance of your computer.
Something that was very visible with little resources becomes invisible.....
With today's computers the number of buggy programs is very much bigger and the program are more baddly done. If you make a software is easy to discover a bug if your computer has only 32M, but if your computer has 2G you never discover the bug and what is worst, people that make a software and discover that it crash with 32M and run OK with 512M,  don't do nothing and put in the system requirements for the software "minimal memory 512M"
Result, minimal requirements : "CTL-ALT-DEL keys and Power Switch".........

"The memory management is not done by Poser, Vue, Maya, Photoshop and whatever application.  Memory management is done by the Operational System (Windows, Linux, OSX, etc)
Nobody writes a memory management for a program..."

Photoshop > Edit > Preferances > Plugins & Scratch DIsks allows you set set up to 4 disks for Photoshop to use for Virtual Memory.  It's been this way for years.  I'll be surprissed to not find it in other applications soon!

I just tried a small experiment in P6 (same thing happened in P5)...

I copied a texture (selected at random)  from its original folder to the desktop. The I fired up Poser, loaded a cube and applied the texture (from the desktop) to the cube. The texture showed up in the preview.
Then I minimised Poser and re-named the texture on the desktop to be different. Restored Poser.
Moved the camera around in the Preview (Pose Room) - there was no problem - texture showed up fine in all views.

Hit render (Firefly), and guess what? It couldn't find the texture (and came up with a dialogue complaining about it).

Conclusion? Even tho Poser still has that texture sitting in memory (it must have, otherwise the preview changes wouldn't have worked), Firerfly reloads textures from disk..

Which leads to the inescapable conclusion: Every (bitmap-type) texture that you apply to anything gets re-loaded (thus duplicated) at render time....

(Not sure if this applies to the P4 renderer, that's for another experiment..)

No wonder Poser uses up memory fast...- consider all those 4096x4096 V3 textures being duplicated...

Umm. One thing I didn't do which I should have, was check that the preview mode still worked after the attempted (and failed) render. Oh, well - next time...

Cheers,
Diolma

I don't see the problem you're seeing here, diolma. The preview uses a sized down texture - 256x256 as stewer already pointed out in that other thread you participated in.

Diolma - Poser does'nt automatically reload textures at render time.  I often edit the V3 Head and Body textures that I have loaded into a scene(While they are loaded,) from Photoshop.  To view my progress from I need to; save over the file from Photoshop, then go into Poser and make sure that Render > Kepp Textures Loded is'nt checked, click Render > Reload Textures > and then Render.

"I don't see the problem you're seeing here, diolma. The preview uses a sized down texture - 256x256 as stewer already pointed out in that other thread you participated in."

Umm. OK, mea culpa - I missed that post (I regrettably didn't read all of the posts fully).

I stand corrected and blush!

But still, I live and learn..:-))
S-l-o-w-l-y...

Cheers,
Diolma

"Poser does'nt automatically reload textures at render time.  I often edit the V3 Head and Body textures that I have loaded into a scene(While they are loaded,) from Photoshop.  To view my progress from I need to; save over the file from Photoshop, then go into Poser and make sure that Render > Kepp Textures Loded is'nt checked, click Render > Reload Textures > and then Render."

I agree. That wasn't my point (which turned out to be wrong, anyway). I thought that the Preview and the renderer both loaded the texture at the same resolution. I was wrong.

Once the renderer has loaded a texture, then it sticks in memory (as long as the session is still in operation, IE, Poser still running), and if the "Keep Textures Loaded" is checked, then any changes won't be re-loaded during render..

I think I'll shaddup on this subject til I've got the details sorted in my one remaining brain cell (which is rapidly deteriorating - probably due to memory fragmentation..)

Cheers,
Diolma