Fri, Nov 8, 7:28 PM CST

Renderosity Forums / Poser - OFFICIAL



Welcome to the Poser - OFFICIAL Forum

Forum Coordinators: RedPhantom

Poser - OFFICIAL F.A.Q (Last Updated: 2024 Nov 08 8:41 am)



Subject: PPro Queue Manager...Waiting and Waiting and Waiting


TheJerry ( ) posted Sun, 18 May 2008 at 8:53 PM · edited Fri, 08 November 2024 at 7:20 PM

Hi,

So has anyone had much experience with the Poser Pro Queue Manager, yet?

First animation I sent to the super queue went in, got initial frames back and then nothing...

Setup:  2400 frames sent to queue on W2K8x64, 4 external que managers W2k8x64, W2K3x64,  Vista x64, W2k3x32.

The local queue mgr, and all queues on x64 machines (QM is a 32 bit process though) processed at least one image.  The w2k3x32 doesn't seem to have gotten any info from
the master queue, nothing in logs. 

While rendering every machine that got a job sent their cpu meters up and ground
down like expected.  I got 6 images back total, so that means some node(s) did more than one.

but after that all went back to normal, nothing much going on.

Anyway, after that the local QM ran through all 2400 images, I saw it count them up.

Now it says "Waiting" and none of the nodes seems to be doing much and images
aren't coming back, nothing in log files. 

Any thoughts?  Anyone had successes with lots of images and multiple queues?

Jerry


operaguy ( ) posted Mon, 19 May 2008 at 12:19 AM

Jerry, while I have PP, I have not tried the QM yet. Let me just ask, can you start with something very simple, send it out to one machine, see if you can get it to run, then slowly kick up the burden?

You are rendering out to image sequence, right?

::::: Opera :::::


TheJerry ( ) posted Mon, 19 May 2008 at 5:57 PM

Hi,

Well, it turns out to be very funky.   I now have it "working" mostly.

Yes, I am doing image sequences because I've decided its the best way to create flv files at full color.  (image sequence to quicktime to flv converter or image sequence to flv depending
on your flvr.  The poser flash stuff ain't so hot)

The master queue manager gets confused easily.  Experimenting around

1)  I changed the "file location back to" a local drive rather than UNC path

  1. turned off the QM that was never getting any communications with the QM

At that point stuff started working like it should.  I do note however, when I one time
started a new QM while a job was going, this seemed to kill the job across the board.

So I don't do that anymore.

I do have one machine (w2k3x64) of the QM that keeps dying after about 2/3 of the images and
then the rest continue and the one image that was last sent to it doesn't ever get done
and has to be resubmitted.

Restarting that QM between jobs doesn't seem to hurt anything.

Anyway, I can also note other things.

  1. Vista x64 and Windows 2008 x64 seem to be the best/most reliable platforms for PPro.
    W2K3x64 and x32 both have various issues

the local QM where you submit the job doesn't seem to run renderings as far as I can
tell.  It only farms out jobs, preserving your local machine to use PPro the GUI on and
make changes.--like to change that option--apparently can't.

The QM's all "talk" about having 0 or 1 procs free, there is no way to tell the QM's to use more than 1 Proc (e.g. on a 4 proc machine use 3 procs)

3) 
QM likes to send each individual image as a new job, resending textures etc.  excess overhead.

4) 
all broadcast; no way to tell it to use a qm on a remote LAN or internet.

All that being said...

A image set render that took me 3+ days to do on my Vista x64 box took the QM running a node
on the same vista box, and a w2k8x64 box and the w2k3x64 took under 18 hours, and that's
with the w2k3 box konking out 2/3 of the way through leaving me with only 2 nodes.

On stand alone image renders.  Poser Pro was able to do one killer image of mine that
the first time I did it on P6 I had to break up into 6 layers and composite.  Took me about
8 hours including compositing first time w/P6.  Poser Pro did the whole thing in one pass
in less than 5 minutes.


operaguy ( ) posted Mon, 19 May 2008 at 7:26 PM

I am building my Core2Quad Vista64 rig now, didn't even bother to install PP on my pathetic old rigs. Looking forward to gaining the performance kickups you are citing and also seeing if what can be gotten out of QM. Thanks for pioneering and please send an invoice to SMicro for your beta shakedown of their software.

Can I ask...under QM aren't you supposed to be able to send completely different jobs out to a remote computer assigned to different processes on that computer? different cores? But are you saying in point 2 above that it does not work?

::::: Opera :::::


TheJerry ( ) posted Mon, 19 May 2008 at 9:50 PM · edited Mon, 19 May 2008 at 9:52 PM

file_406463.jpg

Sort of kind of how beta testing works there...actually PP is pretty unfunky compared to fun in Poser 7, and before that P6, and before that P5, and P4 and...

So I guess if this is the PPro funkiness it's not as bad as some I had in P7 where I'd
suddenly lose a couple hours of work when it blew a gasket.  Or where I'd have to basically
reboot to get the library manager to come back from off the deep end.

The thing with the QM is I see lots and lots of great potential, but it's like most of the buttons
are missing.  You look at the thing and you think wow I should be able to do this or that, but
literally what you see is what you get and in fact it's not quite what you get.  Those menu
items don't have anything useful.  So what if I can show views of suspended jobs
or what not.  I can't do much to them, at least not remotely.

yes, if I go onto a machine where QM is running and 4 different PPro GUI's/PPro master queues have submitted stuff to my machine, I can sort the order around on that machine.

OR, if I have like 5 nodes, and I push out 5 different jobs, then I could go to each node and
it's QM and change the priority of particular jobs...

but that's about it.

I mean you can clearly see where "someday" there will be lots of cool functionality, I can
see where this could grow and be super for a team environment and "group render farms"
but before that

1)  It needs to be able to run as a Service (right now it's a user application that you "Start")
2)  you need to be able to have more than one processor per queue manager.
     --I want an option screen where I can say "use n of M processors",
    --I want to adjust system priority of jobs not just order...but like in Task Manager
    --Memory restrictions....use max of X memory
   --staging directories/drives---I want to define the scratch drives for each processor queue
3)  I want to do all this remotely from a single Master Queue Manager
    ---right now to see what node is working on what image, I have to go to EACH node and look.
   ---the QM should have a sub-table display that shows me what image is being run on which node and what the current status is.
4)  I want common logging to see info in one place for all nodes.

If you look at the image I attach that's pretty much all you see and you can only do on any
node what the button lets you.

The white QM is the master one where PPro is running (W2K8x64). 
You see one done job, you aren't seeing Canceled jobs, those just sort of vanish.

You see the job that's running, and how many images it has sent out.

You see the next job waiting to go.

The second one is a node running on my Vista x64 box.  You just see the single frame/image
it is working on right now.  No history no future no progress report.

The host field shows what host submitted it, so in theory, I could have another machine running
PPro submit to the queue and it would be next in the list and I could reprioritize...but that
would be it.  I want to assign proc's etc.

it's all sort of weird.  Maybe if I were doing something other than single images, like
an AVI or part of a big image or I don't know....It's like it's setup to someday do more,
but doesn't yet today.


TheJerry ( ) posted Mon, 19 May 2008 at 10:00 PM

BTW Operaguy, I just built out a Core2Quad box specifically for this purpose, that's the Windows 2008 box...rocks very nicely with PoserPro.

however, consider the use of Wiindows 2008x64 vs Versus Vista x64.  Even with SP1,
vista is much slower than W2K8.    (I've been doing Vista x64 for over a year now,
before that XPx64...just started moving to W2K8 x64 servers cause it's insanely fast).

W2K8 is actually impressive (and I'm a Systems guy)  the thing knocks the socks off of
W2K3, on identical hardware, W2K8 is visibly faster.  And W2K3 feels faster than Vista.

You can optionally add the Desktop Experience component, and get a lot of the Aero interface,
the only thing I seem to be missing is the "Explorer Preview" feature, which sort of sucks,
but one can always use a third party file explorer if preview panes are important.


operaguy ( ) posted Mon, 19 May 2008 at 10:40 PM

From your analysis I can tell you really want to love QM and that's the way I feel so far. You are probably quite right it has great upside potential. I like the conception of QM over the typical renderfarm stupidity such as you find in Vue.

i have no idea what windows 2008 is. I am not a systems person but respect your opinion. Is it officially called Windows 2008? I want to look into this, because all I care about is speed and stability, not Aero etc.

Is it server software?

I did manage to purchase cool server-intended enterprise level hard drives for my rig, three of them, Baracuda's with 32MB cache. They were hard to find; I was sent on the quest by a friend who talked me down from Raptors.
http://www.newegg.com/Product/Product.aspx?Item=N82E16822148309

This is the rig:
Antec P182 case
Zalman 850W Power Supply
Intel motherboard D975XBX2
Intel Core2 Quad Q6600
8 Gig Patriot DDR2 Memory 4-4-4-12 800 Mhz
3 Seagate Baracuda HD 250G|ST SATA2 7K ST3250310NS
Zalman CPU cooler UltraQuiet
XFX Nvidia GeForce 8800GT
Logitech Z5500 Speakers
Logitech wireless keyboard/mouse
Vista 64

::::: Opera :::::


operaguy ( ) posted Mon, 19 May 2008 at 10:50 PM

Ok, Googled. Intimidation city! To you this is baby talk to me it is nearly from Mars. Can you just install it, ignor all the server stuff and run from the desktop?

Anyway....would this do it?
http://www.ewiz.com/detail.php?p=MSR1802907&c

::::: Opera :::::


TheJerry ( ) posted Tue, 20 May 2008 at 11:54 AM

Hi,

Interesting, very similar setup to my server.  I used the EVGA GeForce 88000GTX and an
ASUS P5W DH mobo and slower SATA drives (2x 500 mirrored boot, 2x1 TB striped)
 
But same processor and same RAM.

Your link is for the CAL's for it, you dont' care about those.

Windows 2008 x64 Server.  Actually the interesting part, the basic install of W2K8 Server is
"less" crap than you get with Vista.  You have to go "enable" stuff to install it.  So for example
media player and aero etc, are check boxes that you install afterwards using (Programs And Software...in Vista/W2K8).

Windows 2008 is the "server" version of Vista.  This means it is the more stable/scaleable
version.  Traditionally this didn't do much for speed. However, in this case I have found
W2K8 to be a lot faster.  It came out a good year after Vista so they had time to get
things "right"

From a day to day point, if you don't install the Aero stuff, it looks like a cross between
XP and Vista.  if you install the Aero stuff, you won't be able to tell you are not on Vista
other than speed and a few Aero features like the Preview.

However, if you've already bought Vista x64...not sure that it is worth it.

Basic cost for Standard is like 2x Vista Ultimate.  There are other reasons/features that
cause this cost to be justified, but if you aren't going to use those features....it's an expensive
boost.  (although some of them you could find useful...like legally having 3 people being able to  use it at once, independently, via console/monitor and 2x remote desktop sessions and
some of it's very useful point in time file restoration features; and the ability to legally run a second copy of W2K8 standard inside as a virtual server; and if you have other computers in your house that have XP pro or Vista Business/Enterprise/Ultimate, you can create a domain and then life is so much nicer...but that's another story for another day)

--actually running the second copy of W2k8 as a virtual server might be interesting as
a work around for QM's problems.  if QM refuses to use more than 1 processor on a
machine, then I put QM on a virtual server running on my 4 processor box and suddenly
I have 2 QM's running instead of 1.---not sure how performance would be, but my gut
from playing with QM is that the multiple QM's slightly slower QM's is better than 1 faster
QM---

But anyway...as a dedicated graphics monster only....

I would say it's worth it, if you hadn't bought the Vista, or had another box that could use the Vista or could still return.  But to have bought both...

http://www.newegg.com/Product/Product.aspx?Item=N82E16832116452

Here is my gut feelings on performance...

Windows 2008 Server seems about 25-35% faster than Windows 2003 Server on the
same processor you have.  That is my comparison, that I've been talking about.
(ignoring graphics cards which for my test were lower end PCIExpress, my newest
box has the 88000 GTX...and things went into overdrive on the GUI front)

Windows 2003, generally seemed about 10% faster than XP (it's desktop counterpart).

Most people seem to think that Vista is about 15-20% slower than XP.  But this is highly
subjective based on graphics card, memory, hard drives.  if you have sufficient resources
vista can really kick into gear, it just has to get past the minimum requirements.

One can mangle these numbers together however one wants, not sure if it comes out
to adding up.  I don't have a direct Vista to W2K8 comparison, but I'd say you are talking
a solid 25% faster, if not more.  But again, newer graphics card and crap loads of memory
also add up.


One big thing you want to do, in all cases, is make sure your bios has memory mapping
turned on for x64 bit OS's.  otherwise you won't see all 8 GB of RAM.  Highly dependent
on mobo how to do this (some will automatically others won't and you'll either see like
4 Gb or 7 GB)


operaguy ( ) posted Tue, 20 May 2008 at 6:26 PM

Thanks for posting all that. I already have purchased Vista home 64 for about $90

http://www.tigerdirect.com/applications/SearchTools/item-details.asp?EdpNo=2864277&CatId=306,

and while it is unopened and I could probably send it back, going up to $700 won't work at this point...not that it is not worth it. Right now I have 2 licenses for XPHome and one for XP-Pro, both 32 bit. At some point consideration of going to server 2008 will crop up again.

I'll watch for that last tip you posted. Thanks

::::: Opera :::::


TheJerry ( ) posted Tue, 20 May 2008 at 11:45 PM

It is a big jump up, and one can easily argue said money is better spent on hardware pieces.

The biggest thing I can see, having thought about it some more, would be if one was insistant
on getting the most 'queue" out of QM and SM doesn't plan to ramp the thing up, then
it might be worth while to be able to do a virtual machine and use it as a second render
queue machine. (since QM won't use more than 1 proc per machine) 

Of course for that kind of money one could buy a cheap pc somewhere too given that you
already have extra licenses....

If you still have your old box...you should certainly think of using that for a QM machine.

 


Privacy Notice

This site uses cookies to deliver the best experience. Our own cookies make user accounts and other features possible. Third-party cookies are used to display relevant ads and to analyze how Renderosity is used. By using our site, you acknowledge that you have read and understood our Terms of Service, including our Cookie Policy and our Privacy Policy.