Forum: Vue


Subject: OpenVue 0.1 - an attempt to a free public renderfarm

haegerst opened this issue on Jun 17, 2008 · 19 posts


haegerst posted Tue, 17 June 2008 at 2:37 PM

OK, many people told me this will not work or be crappy, but being stubborn as usual i do not give up that easy.

OpenVue 0.1 - a free public renderfarm

**About:
**
The idea is simple: You want to render an animation and your computer is so slow it takes days to finish. While rendering you can barely do anything else and your computer is blocked. With OpenVue we want to give unused processor time a new meaning. While you surf the web, write emails, listen to music or watch movies on your computer the processor is idle around 95% of the time and valuable processing power is converted to heat instead of being used. With a free renderfarm you could render your vue animations on other people's computers while they browse these forums, write their CV or play games.

**The Solution:

**OpenVue, a free public renderfarm. Besides Easel and Esprit all vue versions come with Hypervue, which enables you to render on up to 5 nodes at once, which should speed up rendering animations significantly.

**Current version:

**The current version is 0.1, which is the initial one. Simple Tests have been done, but before i write the software for handling the clients and keeping the farm together i need your help. I need you to participate, all you have to do i being a part of this project and hopefully render an animation here or there, hey the good news: You get processing time from all other participants. I can not pay money for your help, but i am thinking of ways to compensate for your time. Maybe i can arrange elevated privileges in the project team, priority rendering slots and so on.

**How to use the renderfarm:

**As an active user you just need to have a computer with rendercow installed in the version 6.60, built number 292711 (most recent).

Your rendercow must be reachable from the internet, so in case you connect this computer to the internet via a router, you need to set up a Network Address Translation Rule to forward that port from your router.
I know that not everyone knows how to do this as its a bit "techie stuff", but i hope at least a few people here are able to do this. I will try to get some tutorials for doing this, but this of course varies from one router model to the other, so only basic help can be given. At a later state - if this project develops as i hope - i will try to implement a function in the control suite that aids you in doing this step.

As a third step you will need to let me know the internet address (IP Address), so your rendercow can be reached by other computers. Most Internet connections have variable IP addresses, so this will change usually every 24 hours. To keep track of your computer easier i recommend to install a service like no-ip.com or similar free dyndns providers. This assigns an easy to remember name to your IP address and some small tools automatically update this address once your IP is changed.

I am aware this is all very technical and many people reading this will be unable to understand all of it. Still i hope some nice souls here support my cause and are able to set up a rendercow. Once you are done just send me your DNS name and port (Standard rendercow port is 5002 unless you changed that) so we can start the first serious tests.

**Rules, limitations, drawbacks:

**There are loads of drawbacks. This will never be a solution that can be compared to professional renderfarms like ranch computing - they have a whole room of servers that are high redundant, have super performance and run all day long, they are being monitored and are dedicated to rendering for customers only.

OpenVue will have the following drawbacks:

Other possible problems are stability (people turning their computer off while it renders a frame of your scene), performance and maybe even more. However - for many problems i will try to find a technical solution. Untill i start programming i do need excessive testing, feeback and people that are interested in this project.

**Demo

**If you want to help RIGHT NOW, you can render test scenes on my computer. Just pick a test scene you would like to try, open the render options in vue and select external renderer, then click on "edit" and choose hypervue network manager. Click on "Add" and select "name" and in the corresponding field enter "darkfire.zapto.org". Make sure "port" is set to 5002.

basic rule: Please do not pick scenes that "take ages" to render. I'd say around 30-60 minutes on a "normal single core" should be the maximum. I reserve the right to delete scenes that take too long for rendering as i only have this single computer for testing right now (Currently setting up a second machine).

If you like the idea contact me via site mail here to discuss details. Right now i need more people offering a rendercow reachable from the internet.

Please leave your feedback and ideas here. You can most probably not talk me into giving up right now, i will listen to all constructive criticism, but please do not post just "This is stupid" or "you will fail", this will not help anyone. Thanks in advance for your feedback.

Vue content creator
www.renderarmy.com


ArtPearl posted Tue, 17 June 2008 at 9:07 PM

Actually I think similar projects worked for scientists doing lengthy computer simulations. As far as I remember they recruited the general public not their peers, because their 'peers'  also used their computers to full capacity. At the moment I am using my computer full time so I wont be able to contribute much. (I also dont have any renders that take less than a couple of hours).
I wonder if you thought about the legality of  sending content over with the scene files. I think the comercial render farms aquired all the c3d content so they have a lot of the content that people use.  But if you do it with 'regular' users they may not have the right items.
I hope it works  - seems a worthwhile idea.

"I paint that which comes from the imagination or from dreams, or from an unconscious drive. I photograph the things that I do not wish to paint, the things which already have an existence."
Man Ray, modernist painter
http://artpearl.redbubble.com/


Rutra posted Wed, 18 June 2008 at 1:43 AM

I fully agree with ArtPearl on both points.
My computers are almost never idle, I'm almost always rendering something. If it's not a new image, it's a larger size of an old one, or a new version, or whatever.  If it's not Vue render, I'm doing a Poser cloth simulation (also CPU intensive) or working on other stuff for which I wouldn't like the computer to act sluggish.

Regarding the 2nd point, just some weeks ago, I had a file sent from another member of Renderosity for testing something and I couldn't render it because I lacked a couple of specific trees. I had to manually replace the trees by some others, which gave a completely different look to the scene.

There's also the security issue. I'm not very fond of the idea of opening my computer to the internet, even if only one more port. Generally speaking, the less doors are open, the better it is. One of my computers where I work in Vue is not even connected to the internet at all. It's physically isolated, so that I don't need to worry with firewalls, antivirus, spyware, and the like, and so I have maximum performance with it.

Anyway, you do have some strong obstacles in your way but I still wish you the best of luck. It's a good thing what you are trying to do for the community.


Rutra posted Wed, 18 June 2008 at 3:01 AM

I just remembered of what could be another big difficulty, but I hope I'm wrong. I'm talking about the RenderCow version. When I made some tests with this some months ago, I found that apparently Vue requires that the rendercow version is the same as the main Vue engine. Vue even has an embedded feature where it tries to update the rendercow version if it's different.
There are many people around with earlier builds of Vue. Many people don't upgrade to the latest build or go back to the previous one when they have problems with the latest. For example, I only go to the latest build if I have problems with the one I'm using, otherwise I don't change.
Of course, this difficulty could be solved by establishing some rules. For example, by establishing what's the build that the community should use in order to use OpenVue. It's a bit restrictive but it would work, I think.


Jonj1611 posted Wed, 18 June 2008 at 5:09 AM

Sounds interesting, however as artpearl and rutra said my computer is always doing something and is off at night(its in my bedroom).

However I believe the idea is solid if not littered with obstacles, if you can get it up and running successfully I believe it could be a great resource.

Good luck!
Jon

DA Portfolio - http://jonj1611.daportfolio.com/


haegerst posted Wed, 18 June 2008 at 7:11 AM

Thanks you all for your comments!

I will try answering what i can:

@Artpearl: Yeah now in the earliest beta phase i still dont ahve a software that controls the resource usage on your computer, but i plan at a later phase to have the rendering threads adjusted so they will NEVER steal CPU power from your other applications by making the rendercow thread lowest priority of all processes, so it will only get CPU power if the CPU has true idle time. You can however be a passive tester if you want, right now i am at work, so in these times it is also okay to have a "larger" test render processing if you want to.
There might be users that drive their processor at 100% load for around 95% of all the time, but from my research (I interviewed some "hobbiist" artists) this is a rare occassion and while many people have days where the render 24/7 there are "usual" also times when the CPU is idle (i.e. the phase when you create scenes, do postwork or just need a creative break).

@Rutra: Generally you can save a scene with all textures included, there ARE Cornucopia3D objects that you can buy "protected" or "unprotected". If you buy the protected version it unfortunately cant be helped. All professional renderfarms will also have this problem, only exception is Ranch computing, it is rumoured they got a special version by e-onsoftware that ignores the protected flags and can "access everything" regardless of the serial. I dont expect to get that version for obvious reasons, but this is a quite small limit that should also occur in your network at home. If you have a rendercow installed on some random laptop in your home network or some user of OpenVue in lets say asia it should not be a difference. A cow is a cow is a cow. I think with byuing protected content you are willingly buying such problems - but i have not yet had the chance to test this to confirm it is really impossible to render this. I challenge you to just try it out, my node is waiting for you.  smile  =)

About security: rendercow has NO known exploits, meaning it is safer then about every other internet related software. As the rendercow and what it does is quite simple compared to programs like winamp, emule, other p2p software etc i think it is very unlikely that exploits appear as the accepted data is very very limited, i yet have not found any handle for exploits and think it is unlikely one will appear.

For the rendercow version we try to have the same version (latest 6.60) for all nodes in the beginning to make testing easier. This version is reported to be stable for most users and seems to be widely accepted, so for pure testing this should be okay. Later on i might implement a version control in the management module, so Vue 6.1 users will only render to 6.1 cows. Maybe there is also another solution, but we will care about that at a later point, we have to care for the basics first.

@jonj1611: Well maybe you have some times here or there where you dont drive your computer to 100% cpu load, i initially thought my own computer has the same workload, being ALWAYS in use, but de facto it is not utilizing the CPU over a constant 100% when i really measure it. I might also add a feature that only "offers" your cpu if your screen saver is running and the CPU load is below a set threshold (like under 50%). unless your computer is a pure renderfarm node it is very unlikely that your cpu is really at 100% "all the time" (For a desktop you'd probably have problems if this is the case).

Right now i need testers! Torture my node with your test renders! Try to not overdo it too excessive, but right now you can also send files that take long to render. better a very busy node then having noone testing at all! And it would be awesome cool if we could get another rendercow node for testing! crossing fingers

Vue content creator
www.renderarmy.com


Jonj1611 posted Wed, 18 June 2008 at 7:36 AM

Hi,

Just wanted to add, that copy protected items should not be a problem unless :-

If I have got this correctly, do the users send the scene to you and then you just distribute it and have your computer as the master program? Ie, you render everything, work is completed and you send scene back to user.

Or

Do you more or less just provide a mechanism of known IP addresses that are open rendercows?

Because if its the second one then its no problem with protected content as the content is already on the host computer, difference between this method and The Ranch for instance is you are sending you whole scene to ranch. If you are rendering on your home computer whether you have set it to render using cows or not the protected content has already been authorised and unlocked for your machine.

Jon

DA Portfolio - http://jonj1611.daportfolio.com/


haegerst posted Wed, 18 June 2008 at 9:58 AM

Hi Johnj1611, the current approach utilizes the second mechanism - the ip addresses (hostnames) of participating clients will just be added to your hypervue list, so there is no difference between a network rendercow you have at home (maybe your old laptop or secondary computer?) and an OpenVue rendercow, the only difference is that your home rendercow has a local IP Adrress and OpenVue rendercows have an internet IP Address.

I have not yet tested anything with protected content yet, but from my understanding - if you can render protected content on your home rendercows it should also be possible with the OpenVue rendercows.

And again: Everyone is welcome to render your test scenes via my official OpenVue rendercow. Just add darkfire.zapto.org with Port 5002 to your hypervue and give it a try.

:-)

Vue content creator
www.renderarmy.com


Jonj1611 posted Wed, 18 June 2008 at 10:17 AM

Thats exactly right Haegerst, it doesn't matter if its protected content or not, it WILL render without problems.

Cheers
Jon

DA Portfolio - http://jonj1611.daportfolio.com/


Warangel posted Wed, 18 June 2008 at 12:26 PM

I would be happy to participate if you can address two immediate concerns that come to mind.

  1. I have no idea how to setup my router like you mentioned, and that's intimidating sounding.
  2. How tight is security? I mean, will people be able to start blasting my IP to hack my system? I wouldn't want that.

And no offence, I appreciate the initiative and actually think it will work, but Ranch Computing is awesome in price and performance. <--- free plug.


haegerst posted Wed, 18 June 2008 at 1:37 PM

@warangel:

  1. I recommend looking at this site:

http://www.portforward.com/routers.htm

It gives good instructions for about every existing router model. On the site you can pick a game to set up, just take any random one and in your mind replace the "port number" mentioned there with 5002. This is the standard port rendercow uses on startup.

  1. As there are no known exploits for rendercow you can not get hacked through it. Due to the simple nature of the rendercow communication chances that flaws appear in future versions tend to zero. E-on would really need to be terribly drunk to make a future rendercow insecure.

Yeah Ranch computing has awesome performance, and while being "affordable" in terms of professional services it is for sure no solution a hobbyist user would ever use. Really! I had once played with the idea of using the ranch for a 1 minute animation of a very complicated scene with a huge ecosystem and on my system here at home it would have taken ~ 120 days to render the animation. After tweaking lighting/atmosphere/resolution i got it down to about 20 days which was still too much for an impatient guy like me. I used the ranch price calculator and while the animation would have rendered in less then 6 hours at the ranch it would have costed like 1200 USD. And 1200 US Dollar for 60 seconds of animation is not what the usual home user can afford. Just for the records: in the original settings that would have taken my computer 120 days it would have costed a whooping 7900 USD.

Yes - maybe some home users can afford to render a single frame still image for a few bucks there, but OpenVue is for hobbyists like me that want to render 1 minute animations a bit faster then 20 days.  :-)

Don't get this as a try to defame ranch Computing, their prices and performance are much better then anything else and for a professional service they are probably super cheap and stuff, but for private users like me and many others here that had to save up the bucks for Vue itself, it is out of question to spend 100 Dollars here and there for faster rendering.

I don't think OpenVue will be appealing for professionals or that it could be acompetition for ranch Computing. This is neither the focus of the project nor very likely.

Vue content creator
www.renderarmy.com


nruddock posted Wed, 18 June 2008 at 4:08 PM

I was under the impression that RenderCows were tied to a particular Vue serial / INST code.
Also that the numbers of cows you can use is also tied to the number licenses purchased, so it would be odd if you could add an arbitrary number of machines that aren't setup with your license info.


haegerst posted Wed, 18 June 2008 at 4:37 PM

No, your hypervue is by default limited to 5 nodes, so you can not render your animation with more then 5 nodes unless you buy additional rendercow licenses. You can install rendercow on any machine, the cow itself will not ask for any serial or be bound to one.

Sidenote: Cow was down for server re-configuration, but now its up again, so everyone can stress it again.  =)

Vue content creator
www.renderarmy.com


ArtPearl posted Wed, 18 June 2008 at 6:49 PM

There are two independent issues:

  1. Can it be done technically
  2. is it legal to do so

I'm no expert on either, but I could for example, send someone a vue file with all objects and textures included, and they could open it fine, but if I sent them any poser content or  vue content, I'm pretty sure this is illegal.

Being that it is a public venture, it is quite possible the merchant that I bought the content from will find out about it. I do not want the legal sharks knocking at my door.
In fact you maybe liable too as an organizer, and it  would be a shame if a public spirited person like you gets in trouble.

So, I understand you are keen to try it out to see if the technical part would work, but I think you should FIRST officially contact VUE & cornucopia3d and also poser content sellers like daz3d, content paradise and of course the market place here.
If you dont have their consent, I for one wouldnt want to be involved, and I dont think others should either, not worth it in my opinion.

"I paint that which comes from the imagination or from dreams, or from an unconscious drive. I photograph the things that I do not wish to paint, the things which already have an existence."
Man Ray, modernist painter
http://artpearl.redbubble.com/


haegerst posted Thu, 19 June 2008 at 1:40 AM

Dear Artpearl,

  1. We are currently doing it technically, so YES.

  2. What is illegal with using someone else's computer to render???? If this is illegal then all render farms including Ranch Computing must be illegal following your logic. I am no attorney, but I don't think so.

Vue content creator
www.renderarmy.com


Jonj1611 posted Thu, 19 June 2008 at 2:35 AM

It is not illegal to render the way Haegerst has said.

Jon

DA Portfolio - http://jonj1611.daportfolio.com/


chippwalters posted Fri, 20 June 2008 at 12:31 AM

haegerst,

I agree with Jonj611, I believe using rendercows as you've suggested is just 'extending' the network outside one's own LAN and should be legal. But, you of course know you can't alter in anyway the existing RenderCow software including modifying it's 'affinity value' for processor usage.

But here's where I think you're going to find trouble (and you've already identified it previously): The amount of files and data sent over the internet is typically huge for scenes which need to take advantage of multiple computer power. In many cases, this will take longer than the actual render time, and noting one is limited to a max of five render nodes, I doubt there will be much gained overall.

You mention you want to 'limit' the size of files, but how can you do that w/out interfering with existing proprietary RenderCow code? That's a problem.

This is the reason Ranch and others have you send the whole scene. It only needs to be uploaded one time, and then is dispersed through their own LAN to render.

Another issue is the overall robustness of RenderCows currently. At best, network rendering in Vue is-- shall we say-- an 'aquired taste.' More than once it hangs up, doesn't complete, doesn't connect, doesn't transfer, etc-- and that's for those of us who use it all the time. I can't imagine how frustrating it could be to have it not work after a 15 minute upload of a scene to an internet enabled RenderCow. And interrupting network renders sometimes results in having to restart the RenderCows on all associated computers.

I use a 4 node hypervue system here and find many times the LAN upload traffic takes as long as the renderings-- and that's at 1 Mb/sec. Imagine how slow it would be at only a few K per second? Not to mention confi

A nice idea, but I have my suspicions there's a lot to overcome. Good luck though!

 


haegerst posted Fri, 20 June 2008 at 5:07 AM

Hi Chipp,

Yes you are right - "altering rendercow" is of course out of question. So the final software will just act more like a "macro", entering info like it was the user. Also in the discussion is currently the possibility of directly writing into .ini files (where settings are stored) or the registry. The application itself will definitely NOT be changed as this would not be possible due to licensing reasons.

In Windows setting the affinity is possible with the WIndows task manager, this is part of the operating system not the application and should be easy to control. Here the application also will NOT be changed. I do not (yet) have a mac to try, so unfortunately software development will need to focus on the windows version first unless some friendly mac programmer wants to help out.

The amount of data needs to be limited, talking about the file size of the scene. I am not 100% sure how to do this with the software, either the software has to check the filesize by the user loading the scene into the OpenVue software, or with some kind of demon process that continously checks the files submitted for their size. For the beginning we will need to just "agree" not submitting ridiculous large files, but we will try finding a software solution.

I can only speak for my windows related rendercow, but in the current version (6.60) it is remarkably stable, i have not yet had one single crash in the tests. I can not say the same for the windows rendercow running under WINE (in a linux environment), but this is rather a side project and I am working on that.

Oh and Chip - is that true - you have a LAN with 1 MB/Second??? O.o
I have gigabit Lan here (1000 MB/second) and i would REALLY want to talk you into upgrading, Gigabit Network cards became really cheap (Around the price of a good lunch here in germany), and having 1000 times faster network is really worth investing a few bucks.

However - Yes this IS a flaw, agreed, i already mentioned that in my original post. Sending terrabytes over the internet will not be an option, OpenVue is not meant to handle terrible huge files.

Vue content creator
www.renderarmy.com


ehliasys posted Mon, 23 June 2008 at 6:57 AM

 Well, I actually tried that once when I thought keeping my business PC busy on weekends might be a good idea - but the frustrating point was, that my home farm (3 PCs - not the fastest whatsoever) would have rendered the image twice or 3 times while the scene and all the textures were still uploading over the internet.

ADSL - speedy down, lame up.  Duh.

And having that in both directions - upload (scene) to the cow - upload (image) from the cow -  doesn't make things any better.

Think, it's a nice community-ish idea, but maybe some years too early....