I wanted to get my network operational with the Lux render engine, so I loaded the LuxTime sample scene, found that I couldn't change the 800x600 resolution, hacked that in a text editor to 2560x1600, then loaded the new scene.  Got all the blades (rack-mounted servers) added in on the network, and when I went back to check it at 35m47s elapsed, it had already crystallised. :blink:

     Next project will be to export a Poser scene using Reality, learn to re-do materials (mine won't translate automatically), and how to assign meshes as lights, etc., and then network render that. :woot:

     Lux network UI display, and sample render after 1hr26min.

..

Beautiful... Though would never have that kind of patience....

which Samples/Pixel (S/p) value did you reach by then? It''s in the statistics line.

say that's X, then you made X * 2560 * 1600 samples in 2147 seconds => Y kS/s which is used as a Lux benchmark for your setup, it''s sort of constant over time, but scene specific.

ref: my i7 990X @ 4.0GHz (12 threads) runs at about 81 kS/s for this luxtime scene.

crosspost - your setup seems to be say 15+ times faster. Not too bad :-)

     Oooops.  I didn't save an FLM.  Lesson learned there.

     I threw a lot of cores at it.  My older workstation Galadriel has 24GB RAM and the core i7-990X H/T hex-core at factory stock 3.46GHz, but she is currently out of service and back under repair.  The primary workstation Cameron has dual H/T hex-core Xeons at 3.2GHz with 96GB RAM.  The rack-mount blades each have dual H/T hex-core Xeons, with at least 36GB RAM.  That's 288 cores (300 cores when Galadriel is online). 

..

Er, wow.  I need to win a lottery so I can build a server farm of my own!

     The trick is to buy refurbished used blades.  Large enterprises need to keep up with the most recent processors to remain energy-cost efficient.  They sell off the three-year olds.  But there isn't a big market of buyers for used blades, so the cost is a mere fraction of what it would cost to buy new.  Also, the cost is much less than buying a desktop/workstation with equivalent dual-Xeon processor pack and RAM.  Third, the blades use much less electricity because they don't have sound cards and multiple video cards or speakers - just a minimal onboard video chip to drive a (shared) monitor.  Whereas my workstation Cameron draws 752W when running hard, each blade draws 275W but has the same core count.  Finally, the blades are space efficient;  look at that rack with eleven blades in it, and imagine how much desk space would be needed for eleven desktop/workstations.

haha, don't fool me. You told the wife you just needed a new electric heater (I admit, this one looks good and has a nice story attached), and somehow you managed to get rid of the rest of that spacestation you bought at the NASA auction. And all this runs on solar cells only yeah? Okay, and a windturbine.

Me jealous :-)

Again, Wow!  I know I'm going a bit OT here, but how hard is it for the moderately non-tech oriented to put one of those together?  Parts I can find if I look around, I believe.  It's the "getting them all to play nice together" that always messes me up.  I know it's got to be more complicated than "plug-n-pray."

Awesome setup!!! I just ahve one question though. How do you keep the room cool?

maybe it's nice and cool in Ohio this time of year!  just leave the windows open.

wow nice setup ;-)

be overkill for me as a " hobbyist " but if I ever win the lottery be chasing you for advice

     First would be getting blades - I accumulated mine a few at a time.  $360 for a blade with dual L5520 quad-core Xeon processors and 24GB RAM, $668 for dual hex-core X5650 Xeons with 36GB (DeepDiscountServers on eBay).  Don't get a four-processor or eight-processor blade;  Windows will not acknowledge anything beyond two.

     F2 for BIOS;  set hard drive to AHCI (not IDE), and set the boot device so that force PiXiE boot is disabled (it will still check both aethernet ports at boot, but then it will check for other bootable devices).  Hard drive and USB boot need to be enabled.

Buy a Win7Pro 64bit OEM license.  $65 at SoftwarePlusTech dot com.  Buy an 8GB USB flash drive.  Buy a bare (desktop) hard drive.

     Reformat an 8GB (or bigger) flash drive to NTFS.  Copy the contents of a WIN7 install disk to the reformatted flash.  Now the NTFS flash is a bootable Win7 installer.  This step is because blades don't have an optical disc drive.    

     Park a blank hard drive in the left most tray (port 0).  Power the blade up.  It will check for PiXiE boot instructions from each aethernet port, then see an unformatted hard drive in port0, then see the bootable USB flash drive and begin booting from the flash.  You'll see "Windows is loading..."  When Windows asks where to install, choose the hard drive.  It will be formatted and have Win7 installed.  Remove the flash drive when Windows shuts down for a restart.

     Once you've gotten the blade set up, connect it via Cat6 cable to your router, and your workstation will see it as another computer on your network.  Really, Windows takes care of getting the networked computers to acknowledge each other for you.  Install Queue Manager (for Poser Pro), RenderCow (for Vue), and Lux (for Reality).

     For cooling, I have the workstations in one room with a sizeable air conditioner, and the rack is in another room with a sizeable air conditioner.  No, it is definitely *not* cool here in Ohio lately. :tongue1:  But I do use screened windows/door when the weather is mild.  You'll want the blade(s) in a remote room because their small diameter high velocity fans sound like a wee B-52 jet bomber.

     If all this seems expensive, compare what it would cost to buy a dual-processor workstation, or one with a single HyperThreaded twelve-core processor.

Nice farm ! Personally I would just throw 1 GPU at that render but even I gotta admit that this is a beastly setup :)

     I'm not geared for GPU rendering on either Octane or Lux. :blushing:  All of my desktops have nVidia cards, which leaves Lux out.  Lux wants OpenCL (think AMD/ATI/Radeon), and nVidia throttles their cards' OpenCL .  Octane wants nVidia's CUDA, but my cards are too old to have much CUDA capacity or they are nVidia Quadros, which aren't designed for parallel-processing speed.

     GPU rendering in Lux is a nice option to have, but not a panacea.  Citing the Reality User Guide:

"LuxRender can use the GPU to accelerate some rendering operations. While this is a great feature, it’s important to understand that there are limitations to what it can do and that the quality of the render might not be the same as what can be obtained with the standard, CPU-based, renderer.

Pure GPU ... mode is not as physically accurate as the standard LuxRender rendering but it provides the fastest response. With pure GPU acceleration the GPU is used for storing the geometric data and the textures."

     So a GPU render does not equal a CPU render, and using GPU limits the textures and geometry to the memory limits of your weakest video card.  By the way, Reality recognizes at most three GPU.  Your computer's power supply would need to be big to feed 3 GPU plus the rest of the computer.

     I don't mean to dissuade anyone from using GPU rendering.  GPU is much faster, and the results are generally going to be plenty good enough.  But some render engines only run CPU (Firefly, Vue, et al).

     I myself may some day build a GPU rig;  if so, I'll get a huge power supply (or dual power supplies), get three GPU with as much VRAM onboard as possible, and use water jacketing with an external radiator/fan array to cool them.  If I ever do, I'll probably go with nVidia cards and the Octane renderer.

ha,

1. Octane supports network rendering now (limit: 18) so your GPU's don't need to be all in one box and

2. you can add PCI extenders (=special board+power+cards) to your existing rig. Some of those can hold up to 4 cards each. And

3. Octane can use normal ram instead of videoram as well to support the big-texture scenes, and

4. I might be wrong but Octane might be able to run in CPU as well, nowadays.

so, keep an eye on them.

     I'll try to keep up on it, aRtBee.  The 3 GPU and VRAM limitations are for Reality/Lux combo.  Good to know about the eighteen GPU and network capability for Octane;  that would be an insane amount of rendering power!

okay.

As far as Reality/Lux is concerned, I've found (and reported) problems with the Pure = GPU-only mode, in my dual-nVidea-GPU setup at least. Paolo is looking into it.

a) the GPU enabling/disabling does not work properly / as expected. Seems a Reality-issue to me. 

b) a not-too-basic scene will experience the ram requirements going through the roof. In Normal and Hybrid the scene runs fine. A Lux issue.

Also I've found that the Hybrid = GPU + CPU mode runs slower and provides less quality, compared to the Normal = CPU-only mode.

So as far as I'm concerned, I'm not using my GPU's for Lux at the moment. Might be nVidia only, don't know.

any chance you could get water cooling on blades??

I've just added a corsair hydro h55 to my AMD octcore and the temps are awesome (10 d C when browsing and even running as much as I can get going it only hits 40 ) and as for sound I can bearly hear it running

was a pita to fit but it was well worth it ;-)

Wow.  Again.  Is it ok if I copy and paste these instructions into a document for use later?  I don't understand half of what is being said but I know people who do.

(heddheld) 

Quote - any chance you could get water cooling on blades?

You could.  You'd need two processor waterblocks for each blade, and the radiator/fan array would need to mounted externally/remotely.  I think you'd end up with more money in the liquid cooling apparatus than the blades themselves.  I'm content with locating the rack away from my workspace and sleeping area.  Blades are designed to run at full capacity nonstop, so they run cool enough as-is, they're just kind of... whistle-y. :laugh:  I have two workstations that are liquid cooled.

(cedarwolf) 

Quote - Is it ok if I copy and paste these instructions into a document for use later?  I don't understand half of what is being said but I know people who do.

Oh, sure - and if you decide to do something like this, feel free to ask for more specifics.  The whole reason I described this was because I'm basically saying "Hey, guys!  I found a very cost-effective way to add serious muscle to rendering!  Here's how you can do it, too!" :thumbupboth:

Yay! And again, my thanks!

To be fair a GPU based system + Octane/Redshift would be considerably more cost effective then any CPU solution you could use. 1 single 780TI 6GB ($800) could easily replace that entire farm and use much less power, cooling and money.

"radiator/fan array would need to mounted externally/remotely"

lol my fan/rad is hanging off the side of my case with mecanno

NEVER buying a small case again ;-)

I know there is a lot of talk on different forums about how poor Nvidia cards perform with OpenCL compared to ATI (with some people implying that they are almost non functional), but the speed of SLG with nvidia cards is still quite impressive. Take a look here to see some benchmark scores, there are a lot of Nvidia cards in the top GPU scores :blink:

The attached image is a screen grab of the LuxRender GUI of a Poser scene sent to Lux SLG via Reality3. Note that Lux/SLG (SLG3) is running at 4.26 MS/s (Million Samples per second), not too bad for a card that "won't work" with SLG. I'm sure that OpenCL probably runs better on ATI cards, as they are optimizing for OpenCL. Nvidia cards still do very well, however they do much better running Cuda based code.

Like Paolo, I definitely wouldn't recommend using SLG for rending now, it just isn't quite ready for prime time, unless you really want to dig under the hood to get it working well. But it can be fun to tinker with from time to time (just be aware that you can easily cause system instability). Note also that the attached image shows 3 devices. One is my GTX 670M, one is the onboard Intel chip, and the third must be the CPU because running SLG pegged the CPU (and this was SLG, not the hybrid rendered, hybrid never gets close to 1MS/s on my machine). This just shows how SLG is a bit of a black box.

PS: Your set up is killer - bet you don't need to heat your "server room" in the winter 

I forgot to add to my post that if I had a setup like yours, I probably wouldn't be too interested in GPU rendering. That's some serious processing power!!!

     Thank you for supplying empirical evidence, DustRider.  I'll tuck that away.  In my case, my cards wouldn't help much, and I wouldn't see that as being worth the potential instability of Lux GPU-only or GPU hybrid mode, but I will reconsider the nVidia cards' useability with Lux.

    Heddheld, no shame in external mounting, my friend - creative adaptation that works is a good thing!

     Ghonma, I absolutely concede that rendering in Octane on a Titan (or two, or three) would be faster, and even adding the cost of the Octane license, would be cheaper.  But that only applies if you are rendering in Octane.  I primarily render in Poser's Firefly, or in Vue, and have only begun to use Reality/Lux.  There are also other processing uses for which I can use the blades, but not GPU.  Titans will not avail me in Firefly or Vue, and Lux is shaky at GPU rendering, and loses accuracy and geometry/texture capacity.  I tend toward big complex scenes, so that is a considerable drawback to Lux GPU-only mode.  Octane wouldn't suffer that geometry/texture limitation;  that's why I said that if I ever build a GPU render rig, I would get nVidia cards and an Octane license.