FutureFantasyDesign opened this issue on Mar 12, 2007 · 31 posts
FutureFantasyDesign posted Mon, 12 March 2007 at 9:33 PM
**I have 4Gigs of Ram, and a good Radeon 1600 Video card...I keep having issues with large renders and P6 freezing up and making me go thru hoops to get a clean Hi Res / Hi textured render. I got a window that said allocate more memory to poser. Anyone know how to do this>???? Is this a Virtual Memory issue? Currently I am using 4603 and the max seems to be ****8184 in my VM advanced settings. All help is greatly appreciated. Any other ideas are also welcome.
Avg render size needed 4800 x 3600 to 7000 x 4600 both at 300 DPI
HuggerZ!
Ariana**
Is there water in your future or is
it being shipped away to be resold to you?
Water, the ultimate
weapon...
www.futurefantasydesign.com
kuroyume0161 posted Mon, 12 March 2007 at 9:58 PM
Again: VM won't give you more 'memory' if you already have more than a 32-bit process can support (2GB). Virtual Memory is only a HD swapping space when using multiple applications so as to be able to virtually use more than your available physical memory over the multiple applications. You can't teach an old dog a new trick and a 32-bit process cannot address more than 2GB of memory.
You already have the maximum amount of memory that any 32-bit OS will address (Windows XP SP2 with the 3GB switch only gets you to 3GB physical memory seen by the system). Unless the OS and application are 64-bit, even a 64-bit OS won't help a 32-bit app address more than 2GB. Poser is a 32-bit application.
You'll have to find alternatives to reduce the possibility of 'out of memory' errors during renders in Poser. Poser 7 supposedly does a better job at this with large renders. You'll get responses concerning Firefly renders with respect to bucket size and so on.
C makes it easy to shoot yourself in the
foot. C++ makes it harder, but when you do, you blow your whole leg
off.
-- Bjarne
Stroustrup
Contact Me | Kuroyume's DevelopmentZone
pjz99 posted Mon, 12 March 2007 at 10:03 PM
For Windows XP Pro 32-bit:
Process limit by default is 2GB. You can increase this to 3GB by changing Windows XP's config (requires a reboot):
http://www.microsoft.com/whdc/system/platform/server/PAE/PAEmem.mspx
This will not work for Windows XP Home Edition, or previous versions of Windows (e.g. Windows 2000).
For Windows XP Pro 64-bit Edition:
Process limit is 3GB by default, and cannot be further increased.
Note that this probably will not help you based on the size of the renders you are looking at (it never helped me) - but Poser 7 + SR1 is tremendously better about freeing up un-needed memory during the process of rendering. If you are willing to cough up the price of upgrade I'd strongly recommend it. There are other issues with P7 but whatever else, memory management is far better than previous versions (including unpatched P7).
My current gallery submission is an example, I could never have gotten this to render even with P7 un-patched, not at this quality:
http://www.renderosity.com/mod/gallery/index.php?image_id=1400873
17 lights, 4 raytrace bounces, irradiance caching 99 (and lots of ambient occlusion), pixel sampling 10, min shading rate .2.
pjz99 posted Mon, 12 March 2007 at 10:05 PM
Quote - Unless the OS and application are 64-bit, even a 64-bit OS won't help a 32-bit app address more than 2GB.
Well, not quite, as 32-bit apps are allowed to take up to 3GB under Windows XP 64, but that isn't anything you can get 32-bit XP to do so it's not a reason to rush out and upgrade.
Note that there are two resources consumed by a running app: Physical memory, and Virtual memory. I have seen many times that, despite taking great pains to hold Poser 6's hand during large or complex renders, and managing to keep Physical memory below 2GB, it happens that Virtual memory can get quite large (3.5GB or more) which will still cause the app to die. P7 SR1 is the first combination that I have had very safe results with so far regardless of compexity of the scene.
FutureFantasyDesign posted Mon, 12 March 2007 at 10:34 PM
**Hi First ThanX to you both!
pjz99, I own P7, I took it off due to the skin shredding issue with the program prior to the SR1. I have not reloaded it yet. I will go look at your pic in a sec. I have XP Pro and it is a high end system. So I will reload P7 if that particular issue is resolved. I do Hi Res renders for a commercial printer. I am lousey at PS7 and do not know how to resample the pixels to a higher res. OK I went and looked at your pic...you mention 18 hours to render (Excelent POV and look!!!) What bucket size do you use and is this an auto setting / time from P7??? Maybe we could IM and work out some of my flaws and glitches if you have the time. I render a bit differently in look and lighting (*more colored lights) but your style is very clean and life like! :)
Here is a link to a pic of mine that has 8 lights, 3 are AO's and one is Infinite.
http://www.renderosity.com/mod/gallery/index.php?image_id=1393633 I lowered my texture on my current headache to 1000 anhd I hate to do that. But I need to leave Texture Filtering on for the hair to render cleanly.
Talk soon.....
HuggerZ!
Ariana
PPS>>>ThanX for the reboot link, I'll check this out.**
Is there water in your future or is
it being shipped away to be resold to you?
Water, the ultimate
weapon...
www.futurefantasydesign.com
Acadia posted Mon, 12 March 2007 at 10:37 PM
http://www.renderosity.com/mod/forumpro/showthread.php?thread_id=2516735
Try that thread. Scroll down for screenshots on how to do it.
"It is good to see ourselves as
others see us. Try as we may, we are never
able to know ourselves fully as we
are, especially the evil side of us.
This we can do only if we are not
angry with our critics but will take in good
heart whatever they might have to
say." - Ghandi
pjz99 posted Mon, 12 March 2007 at 10:41 PM
The skin shredding bug appears to be fixed, I've never had it happen since initially turning off Use External Binary Morph Targets (which also seems to be fixed in SR1).
That image was done with a bucket size of 8 - bucket size won't affect image quality at all, only how safe the image will be to render (smaller number = safer memory management). As far as I can tell there is no impact on SPEED of render, so I don't see any reason not to set it pretty low (I usually set it to 8).
I have never bothered to down sample texture sizes and generally use very high-rez textures even for medium or long range shots. Maybe that is sloppy and maybe I could save render time and/or memory if I used lower rez texures but I think that is now a moot point with P7+SR1 (assuming one of the OTHER bugs doesn't give you problems). Since you already spent the money for P7 I think it will be worthwile to try it with SR1. Note that you can always just do the great majority of your work in P6 and only use P7 for rendering ...
kuroyume0161 posted Mon, 12 March 2007 at 10:42 PM
No, 32-bit apps aren't normally capable of this!! The 3GB switch gives the OS 3GB not the application. If the application can address 3GB, it can use it. 32-bit applications cannot work with anything greater than LONG (4 Byte) addresses - and it won't happen unless someone recompiles the application to work with VLONG.
The virtual address space of processes and applications is still limited to 2 GB unless the /3GB switch is used in the Boot.ini file. When the physical RAM in the system exceeds 16 GB and the /3GB switch is used, the operating system will ignore the additional RAM until the /3GB switch is removed. This is because of the increased size of the kernel required to support more Page Table Entries. The assumption is made that the administrator would rather not lose the /3GB functionality silently and automatically; therefore, this requires the administrator to explicitly change this setting.
The /3GB switch allocates 3 GB of virtual address space to an application that uses IMAGE_FILE_LARGE_ADDRESS_AWARE in the process header. This switch allows applications to address 1 GB of additional virtual address space above 2 GB.
The virtual address space of processes and applications is still limited to 2 GB, unless the /3GB switch is used in the Boot.ini file. The following example shows how to add the /3GB parameter in the Boot.ini file to enable application memory tuning:
And:
Executables that can use the 3-GB address space are required to have the bit IMAGE_FILE_LARGE_ADDRESS_AWARE set in their image header. If you are the developer of the executable, you can specify a linker flag (/LARGEADDRESSAWARE).
Very, very few 32-bit applications actually do this. This has to be set in the build (compile/link) for the application. Also note the limitation on VM for 32-bit processes. It's all there is red italics and white... ;)
C makes it easy to shoot yourself in the
foot. C++ makes it harder, but when you do, you blow your whole leg
off.
-- Bjarne
Stroustrup
Contact Me | Kuroyume's DevelopmentZone
pjz99 posted Mon, 12 March 2007 at 10:53 PM
Well, I don't know what to tell you, because many times I've seen Poser 7 take >2GB of memory under XP 64 with no special configuration change... I guess it was compiled with that option ^_^
Perhaps more strangely, I have frequently seen Poser 7 take >3GB of Virtual memory and still not crash (although not much beyond that, seems to die around 3.6-3.7GB).
kuroyume0161 posted Mon, 12 March 2007 at 11:00 PM
Maybe on 64-bit, the emulator allows greater VM storage - but I have never ever seen this for Windows XP Pro 32-bit (and I do have the /3GB switch set on my Dual Xeon system w/ 4GB).. ;P Poser usually gets to 1.7GB and croaks (as noted by more than me). Maybe Poser 7 has taken underlying steps towards 64-bit support (I have not installed P7 on my 64-bit system and probably won't for some time) - but this will only work on exactly two OSs: Windows XP Pro x64 and Windows Vista and that's it.
C makes it easy to shoot yourself in the
foot. C++ makes it harder, but when you do, you blow your whole leg
off.
-- Bjarne
Stroustrup
Contact Me | Kuroyume's DevelopmentZone
kawecki posted Tue, 13 March 2007 at 12:47 AM
I find very stupid that XP64 has a 2GB/3GB limit for 32 bits applications. Old 32 bit applications never can request more than 4 GB, but 4GB is larger than 2 or 3 GB it would be no problem of allowing 4GB.
Newly written 32 bit application are able to request more than 4GB using the new RAX, RBX, etc registers even if are runing in 32 bit mode, so why the reason for a 3GB limit?
It looks as XP64 is like Windows 3.1 that was supposed to be 32 bit and really only was 16 bits with some 32 bits addons.
Stupidity also evolves!
pjz99 posted Tue, 13 March 2007 at 1:56 AM
RAX, RBX etc. registers are part of the X64 register set:
http://msdn.microsoft.com/msdnmag/issues/06/05/x64/default.aspx
If you're running a "32-bit" app that makes use of the R* registers but isn't truly 64-bit, the fault is with the developer, not the OS - why design your app to be partially 64-bit aware but still leave most of the code compiled in 32-bit instructions? At that point why not make it fully 64-bit?
kuroyume0161 posted Tue, 13 March 2007 at 2:17 AM
Well, it would be great if addressing spaces could be arbitrarily expanded on existing applications without rebuilding them - but we all know that this is practically impossible. Unless the underlying system is doing some expert juggling, the addressing will never be expandable without a rebuild.
My question has always been this: if 32-bit addressing is capable of 4GB (2^32) why has it always been restricted to half that (2GB=2^31) except in special circumstances? It has always puzzled me why this was the case since there is no reason except if you are employing relative values (positive and negative) - otherwise you have 0-Max (of whatever is available).
Seems that the OS developers/hardware manufacturers would rather do transitory twiddling than get right to the full fledged support of wider address ranges. Heck, 64-bit can handle up to 16 EB or 17,179,869,184 gigabytes (yes, that's 17 BILLION BILLION - or 17 BILLION Giga - bytes) - why are manufacturers (e.g. cpus) and OS developers so curmudgeonly allotting the resouces for this vast-vast-vast amount of addressing? Seems almost conspiratorial to stretch to 8GB here or 32GB there when the 64-bit address space is MILLIONS of times larger than this. Can we say 'profit margins'? Why create a full architecture that would be forward expandable when you can have people throw away their current systems and next systems and next and next and next and next (for ten to twenty years hence) as they alot a little more and a little more. You are all being duped - demand full 64-BIT EXPANSION capabilities now (and a firm way to do this is to refuse to meet 0.0000001% of the way with these pathetic crumbs of 8GB etc. Yes, they are working within the framework of current hardware limitations - to support 1000 GB of memory would require either vast improvements in DIMM capacity or many more slots for accomodation - but is it at all surprising that after 32-bit has been around for over ten years that memory maximums are still only two or three powers beyond this on most systems?
Moore's law forgot one thing about technological leaps - whence the profit margin supports dolling out portions, technological advancement will decrease thereby. (please quote that)
C makes it easy to shoot yourself in the
foot. C++ makes it harder, but when you do, you blow your whole leg
off.
-- Bjarne
Stroustrup
Contact Me | Kuroyume's DevelopmentZone
kawecki posted Tue, 13 March 2007 at 3:00 AM
Quote - Seems almost conspiratorial to stretch to 8GB here or 32GB there when the 64-bit address space is MILLIONS of times larger than this.
Currents 64 bit CPUs have 48 bit address space and not 64 bits, but even this 48 bits gives 256 TB that is 32,000 times larger than the 32 GB limit!
Stupidity also evolves!
kawecki posted Tue, 13 March 2007 at 3:02 AM
Sorry, 8,000 times larger. (no way to edit)
Stupidity also evolves!
kawecki posted Tue, 13 March 2007 at 3:20 AM
One thing is not clear for me yet (have no 64 bit CPU for the tests) is what happens with the registers.
In a 64 bit CPU the old 32 bits registers are 64 bits, if you load a 32 bit register the upper 32 bits are filled with zero, until now is OK.
Now if I load this time the same register with the 64 bit prefix and load with some address above 4GB what happens if I access memory as [EAX] and not [RAX]? I shall address the full 64 bit space or the upper 32 bits are considered zero even have a value different from zero?
Stupidity also evolves!
kuroyume0161 posted Tue, 13 March 2007 at 3:30 AM
Funnily, I have to ask why? If there is to be 64-bit addressing, do 64-bit (unsigned) addressing for the sake of Saint Octaval! They seem to quibble alot about this segmented space for the OS and that for drivers and this for whatever. For Christie's SAKE, there are 17 ExaBytes of available space - you could put the OS in 900GB or so (as in ten million times that) and still have enough to simulate the entire universe for the first several million years - ya f&cks! Again, seems like hemmin' and haulin' to me. Either get down to it or say it as it is - 64-bit pseudo addressing as long as we can't figure out how to accomodate such large memory spaces.
Hands on forehead (as prophetic foreseer), I don't see any progress in this for the next five to ten years. In five years, we'll have reasonable motherboards that support 64-128GB of memory (paultry, crumbs). I mean, in the next six months I want to see the motherboard that addresses 1024GB (1TB) of memory for less than $10K. Prophet - mispelling of 'profit'. Deepak Chopra (whom I despise as a newagy technofile) said that - I am a profit.
Welcome to the machine - you have finally realized that money makes the world go round and that all power corrupts (the treasury). Whence one mentioned that monetary equality was laughable - I note that without such, humanity will perish. If only the ultra-rich survive the, say, impact of a multi-megaton asteroid, would you call that special survival? :)
C makes it easy to shoot yourself in the
foot. C++ makes it harder, but when you do, you blow your whole leg
off.
-- Bjarne
Stroustrup
Contact Me | Kuroyume's DevelopmentZone
kawecki posted Tue, 13 March 2007 at 3:38 AM
I was thinking about some interesting possibilities.
As with a 64 bit CPU you can do the same in 32 bit mode as in 64 bit mode, I can install XP32 (not XP64) in a 64 bit computer.
XP32 have no idea of what is a 64 bit CPU and any memory above 4GB would be ignored, so I can make my own memory manager for accesing the memory above 4GB leaving the lower 4GB for Windows and normal memory access. My program will be able to use and abuse of the upper memory and Windows XP will have no idea that I am using this memory, so it will not disturb my memory and my memory will be under my own control.
Stupidity also evolves!
pjz99 posted Tue, 13 March 2007 at 3:42 AM
This is a bit away from the original topic but:
If you're one of the ultra rich, sure - if you're one of the dirt poor, then no. Since rich vs. poor has been fundamental to the human experience since the dawn of history, I think this is less of an issue than it might appear in the immediate sense. It's not a very nice fact of life, but nonetheless it is a fact of life.
Note that the typical computer user isn't likely to have much use for more than a couple of GB for the foreseeable future anyway - how many gigabytes do you really need to download MP3s, track your recipe lists, and view porn?
pjz99 posted Tue, 13 March 2007 at 3:47 AM
Quote - XP32 have no idea of what is a 64 bit CPU and any memory above 4GB would be ignored, so I can make my own memory manager for accesing the memory above 4GB leaving the lower 4GB for Windows and normal memory access. My program will be able to use and abuse of the upper memory and Windows XP will have no idea that I am using this memory, so it will not disturb my memory and my memory will be under my own control.
I don't see how that is valuable over designing your app to be fully 64-bit aware and simply running under a 64-bit OS, as all memory management would be handled by the operating system and thus would save you TREMENDOUS amounts of work, but if you really want to know how, here's one interesting workaround for the 2gb process limit:
http://blogs.msdn.com/oldnewthing/archive/2004/08/10/211890.aspx
A funny top-down observation by Raymond Chen (same guy)
http://blogs.msdn.com/oldnewthing/archive/2007/03/01/1775759.aspx
Dizzi posted Tue, 13 March 2007 at 3:50 AM
Quote - The 3GB switch gives the OS 3GB not the application. If the application can address 3GB, it can use it.
That's wrong. The 3GB switch tells the OS to only use 1GB of each process instead of 2GB. The OS itself can use more than 2GB without the 3GB switch... > Quote - Well, I don't know what to tell you, because many times I've seen Poser 7 take >2GB of memory under XP 64 with no special configuration change... I guess it was compiled with that option ^_^
Yes, it was > Quote - Perhaps more strangely, I have frequently seen Poser 7 take >3GB of Virtual memory and still not crash (although not much beyond that, seems to die around 3.6-3.7GB).
No, that's not strange as large address aware apps get 4GB on Win64.
kawecki posted Tue, 13 March 2007 at 3:57 AM
Quote - as all memory management would be handled by the operating system
This is just what I don't want, I hate Windows memory management!!
Quote - and thus would save you TREMENDOUS amounts of work,
The only thing that I need to do is malloc() and free() functions and some few lines of code to reprogram the global descriptors.
Stupidity also evolves!
pjz99 posted Tue, 13 March 2007 at 4:12 AM
I think there's a little bit more to it than that, as you need some way to guarantee that application's memory is not grabbed by some other process (including OS) - and if you completely ignore the OS's memory management, as you seem to want to do, then you need some way to protect your app's memory from being ripped off by other running apps, which suggests to me you need to write a completely separate memory manager that the OS will use at a very low level. Good luck with that.
martial posted Tue, 13 March 2007 at 4:40 AM Online Now!
For not technical person,is this mean ,when i will buy another computer this year ,any 32 bits app on Windows Xp will not use more then 2 gig of memory and sometime 3 so i will not bother to buy 4 gig ddr2 like i would buy .But i am not sure about Windows Vista .Same conclusion??
jugoth posted Tue, 13 March 2007 at 4:43 AM
1 Important thing very important??????????.....
You partition hard drive and you instal 2 windows, 1 windows you install no microsoft updates as you need a bare windows.
Some stuff download if for video work but for render work, 3d work, video work, a basic windows will allow you to have more system free.
As on my other new base unit im installing a second windows which i will use for internet gaming and ther stuff, but main partition will be basic so i can do lot's work with no memory hangups.
And if you use other partition for poser ect remember to take ya cable lead out from modem or set top box.
A bare windows makes all the difference, for using applications that use lot's of system propertie's.
HeRe posted Tue, 13 March 2007 at 4:59 AM
You can give your OS (Windows XP and Server) 3Gbyte Memory with the 3-Gbyte-boot-switch.
Your application can only address as 32-bit-application 2Gybe.
You must modify your application for a 64-bit-address-room.
Poser 6 is a 32-bit-application. You must modify poser 6: ***editbin.exe /LARGEADDRESSAWARE "C:Pogram Filese frontierPoser 6Poser.exe.
Poser 7 is a 64-bit-application and can access more than 2GByte memory.
You can read this in: ***http://www.stewreo.de/poser/index.html
***Sorry for my bad english
pjz99 posted Tue, 13 March 2007 at 5:58 AM
Well, Poser 7 is still a 32-bit application, but it is built with the /LARGEADDRESSAWARE switch. It still chokes at >3GB, while a proper 64-bit application won't (e.g. Vue 6 Infinite installed as 64-bit).
HeRe posted Tue, 13 March 2007 at 9:26 AM
Poser 7 identify in the program-header as 64-bit-application
FutureFantasyDesign posted Tue, 13 March 2007 at 11:37 AM
OK well I went thru the system and I am running 32 bit system. I am checking with my ATI tech support on help.
Will write back soon.
Ariana
Is there water in your future or is
it being shipped away to be resold to you?
Water, the ultimate
weapon...
www.futurefantasydesign.com
kawecki posted Tue, 13 March 2007 at 1:50 PM
Quote - I think there's a little bit more to it than that, as you need some way to guarantee that application's memory is not grabbed by some other process (including OS) - and if you completely ignore the OS's memory management, as you seem to want to do, then you need some way to protect your app's memory from being ripped off by other running apps, which suggests to me you need to write a completely separate memory manager that the OS will use at a very low level.
That is why the idea is to use XP32 or Windows 98, neither of them have an idea what is 64 bits and the existence of the extra memory, so nobody will touch this memory (they don't know that it exist!). As for other applications, as they are 32 bit too, in the same way they don't know about the extra memory. The only problem can happen is if I try to run at the same time two applications made by me that use this special memory management, but for what I shall want to do two renderings at the same time!??
For the rendering process once all elements are loaded I need Windows for nothing. I only will need Windows memory management, once the rendering is complete, to copy the image from the render buffer to the video buffer.
A memory management to be used only by one active application is very simple and only has few lines of code (of course no virtual memory, for what I need the disk if I have a lot of RAM?).
Stupidity also evolves!
PXP posted Tue, 13 March 2007 at 2:07 PM
He/She knows his/her stuff does pjz99, you can always learn something from this member :)