Forum: Community Center


Subject: Change to the galleries rating system

Spike opened this issue on Aug 01, 2002 ยท 22 posts


Spike posted Thu, 01 August 2002 at 4:17 PM

We have changed the rating system for the galleries from a numbered system to a word system. Just a FYI.

You can't call it work if you love it... Zen Tambour

 


spook posted Thu, 01 August 2002 at 7:08 PM

so noted. thank you for the change. your efforts to keep improving the site are appreciated. (never an easy proposition, i'm sure, when dealing system-wide.) the scale has a lot of granularity! why?


tammymc posted Thu, 01 August 2002 at 8:20 PM Site Admin

could you explain more about this sentence.. i do not understand. "the scale has a lot of granularity! why?" thanks tammy


firefly posted Thu, 01 August 2002 at 8:49 PM

Hi Spook! I believe that with a numbering system of 1 to 10 you get 10 choices in your rating. The granularity now available is a touch smaller than the numbering system but looks to me to be granular enough to suit most folks feelings about the image in question :) Does this make sense or help?


DragonWalk posted Thu, 01 August 2002 at 8:58 PM

Hmmm...I believe that what spook is referring to with "granularity" is that perhaps some choices are too similar...or...that perhaps there is too wide a range of choices...however...I am only assuming this is what spook means... As for myself...I do welcome this change, it should be interesting to see how it goes...although I do wonder how many artists will feel rather trolled or at least somewhat uninspired "if" they happened to get "needs overhaul" under they're work...I wonder if "this terminology" was at all necessary...it seems a little harsh in it's context...especially when it comes to artwork?? Cheers, Peter


tammymc posted Thu, 01 August 2002 at 9:09 PM Site Admin

ah, ok.. we have received so much feedback on a number scale. what one person thinks a 7 is -which is good .. someone else considers it not so good so in order to possibly make a better change we would just implement the word - meaning instead. dragonwalk.. what suggestion would you have for "needs overhaul".. this is not in stone so we can change it. : ) suggestions


drinksnapple posted Thu, 01 August 2002 at 9:27 PM

I for one like the new system much better. The numbered system had different meanings for everyone but the new system narrows it down so you know (along with their comment) what they meant to rate your work.


Crescent posted Thu, 01 August 2002 at 9:33 PM

I might suggest: "Needs Touch-ups" instead of "Needs Work" and "Needs Work" instead of "Needs Much Work." I'd drop the "Needs Overhaul" personally. If the picture needs more than touch-ups, then it needs a good amount of re-working. There's no need to rub the artist's face in it. I think that even if meant well, "Needs Overhaul" could start a small flame war without very careful comments in the critique area. The one issue I see with these rankings is that it is more difficult for people who do not speak English to participate. Maybe a pop-up with translations for non-English speakers would help?


DragonWalk posted Thu, 01 August 2002 at 9:47 PM

Hi Tammy....let me think about that one okay...I am just in between a ton of work here...;-) Get back to you... Cheers, Peter


pendarian posted Thu, 01 August 2002 at 9:48 PM

Need Overhaul??? Oh yeah, I can see the flames starting on that one too Crescent...ouch! Pendy


tammymc posted Thu, 01 August 2002 at 10:03 PM Site Admin

Great suggestons - Cresent... will get this changed. thanks tammy


DragonWalk posted Thu, 01 August 2002 at 10:14 PM

Hi again Tammy: Listen...let me think on this a bit...I will e-mail you tomorrow morning...I want to play around with these words a bit... Just to re-itterate...I can remember when I first started out on these forums...really green...scared...shy...not all that good with the material but hey...trying...I think if someone had thrown one or two "needs overhaul" under my pics I would maybe have just thrown in the towel...LOL... Hey...I mean...someone buys poser...buys some stuff in the marketplace..just dying to use this neat stuff, creates something...craves to post it...they may not be all the great yet because they are learning...they get these negative responses; they may well wind up just feeling they may as well not bother or go elsewhere. We all know criticism happens in the real world...it is an essential part of growing...but we also know that criticism can be "constructive" or "destructive"...we are not rating "cars"...we are rating ART...with art it is difficult to assess values on terminology...we have to be very careful for even the BEST of the BEST may do very obscure artwork to portray messages...or just to portray a particular or peculiar style or experimentation quality...would that then be a "needs overhaul"?? Artists are a sensitive lot at the best of times...I think "respect" is the key...the wording...no matter what...should be something that may indeed "rate" somewhat, but still ratain that essential "value" to go on... Talk soon, Peter


DragonWalk posted Thu, 01 August 2002 at 10:15 PM

Crescent...good job...;-)


Spike posted Thu, 01 August 2002 at 10:15 PM

Very cool feedback!

You can't call it work if you love it... Zen Tambour

 


lemur01 posted Fri, 02 August 2002 at 3:47 AM

Ehm... at this point dare i mention the persistant murmurs for some kind of dedicated critique forum where artists who are seeking help with their work can air their images for a more in depth crit. Jack


tammymc posted Fri, 02 August 2002 at 8:41 AM Site Admin

lemur01 we are working on creating something that might work well for critiques. thanks tammy


archetype posted Fri, 02 August 2002 at 2:01 PM

I appreciate the efforts of the Renderosity team to help make the Ranking system better. With that said, Im not sure if changing the ranking system from numbers to words really fixes the core flaw in the system.

Currently the system ranks perfect "Excellent" scores above all others without weighing the number of rankings each image has in the Best Rankings list. This means an image with 1 "Excellent" rating is placed higher than an image with, say 100 "Excellent" and 1 "Great" rating. I personally feel this gives too much power to a single vote.

If an image happens to have 50 "Excellent" rankings, they would currently be listed as the #1 image in the All Galleries, Best Rankings list. If I were to decide to give that image a single "Good" rating, it would be immediately bumped down to #19,369 on the list. That kind of a drop is going to discourage me from giving it any kind of rating other than Excellent, as I am not an evil bastard bent on destroying an artists visibility. If I knew my rating wouldnt drop an artists image down too far on the list, I might be inclined to rank it honestly. As it currently stands though, I usually just skip ranking any images unless I feel it really deserves an "Excellent".

Now one could argue that where an image is on that Best Rankings list isnt really all that important. But if an artist feels this way, why would they enable rankings in the first place? Im not sure what other purpose a ranking system would have beyond giving an artist an impression of how their work stands up against other artists.

Sorry for the long rant. I know its impossible to expect a system that can accurately and fairly rank something as subjective as art. I just wanted to pose one possible reason for why everyone seems to be so reluctant to place anything less than a perfect rating on any artwork.


Entropic posted Fri, 02 August 2002 at 8:07 PM

After considering my own images, and the rankings for them ( when enabled ) I'm pretty sure,archetype, that the system is completely unchanged, save for the replacement of words for numbers. I like it. I might be cautious about giving someone a rating of 5, but a rating of "Needs Work" is a lot easier to swallow, and a lot less insulting to the sensitive amongst us. Besides, if you look at the "Highest Rated" segment of the galleries, those images with more 10's ( Or Excellents ) gain priority over those with only a few 10's. Besides, ratings are pretty meaningless, anyway.. it's the critique that matters, which we stil need help with. ;) Paul


spook posted Fri, 02 August 2002 at 8:39 PM

my! i didn't realize there'd be a question to me on this. i think the rating system may be too "detailed" (based on some experience in this field). to be effective as a "quick critique" feature, the ratings should probably be held to 5 options, overall. this is digestable and succinct enough without allowing for "hedging" when used for overall assessment. however, if the goal of the rating system is to provide a comparator - to index artists against each other - then the granularity is appropriate. granularity provides for a level of discrete differentiation among many with similar comparators. that all being said, some folks prefer "grades;" other do not. but i thank the admins for doing what they do and trying to improve what they can. it's most appreciated.


archetype posted Fri, 02 August 2002 at 9:53 PM

Attached Link: http://www.renderosity.com/gallery.ez?Start=19657&Sectionid=0&BestRanking=Yes

To Entropic: " I'm pretty sure,archetype, that the system is completely unchanged, save for the replacement of words for numbers." Based on what Ive seen, I agree with you there. "Besides, if you look at the "Highest Rated" segment of the galleries, those images with more 10's ( Or Excellents ) gain priority over those with only a few 10's." Thats not quite accurate. An image that has 48 Excellents (10) and 1 Good (9) will be placed below an image with only 1 Excellent. If you doubt me, go and check out page 1093 of the Best Ranked list (at 18 thumbnails per page. Ive attached a link to that page.) There you will find all the works that were once at the top of the list, but were bumped into obscurity by a single persons ranking. "Besides, ratings are pretty meaningless, anyway.. it's the critique that matters, which we still need help with. ;)" I couldnt agree with you more. And thats the reason why Ive had only comments enabled on my work for a very long time.

roobol posted Sat, 03 August 2002 at 1:13 AM

I also noticed that a rounding factor has been introduced. Now, lots of my pictures have an excellent, although in fact they all vary from great.greatnice to great.goodhmm (very large grin). I agree with archetype. Although the present system looks more sympathetic, it's only a cosmetic change. It would make more sense to change the current "best ranking by average" into a "best ranking by total". Then, even a hmm would make people go up, so less complaining, I guess.

http://www.roobol.be


tuttle posted Sun, 04 August 2002 at 2:22 PM

I think the ratings system should be simplified to include just three images, pickable from a list:- (1) a steaming dog turd (2) a shrugging monkey (3) a bar of gold That way even if your pics are rubbish, you can still have a laff ;) But seriously, whether words or numbers or 1-5 or 1-100, a free ratings system can only be taken as a vague indication of worth purely because of the varied talents of those voting. If you accept this then fine. If you don't like it, then as archetype says, don't use it. IMHO the only method of introducing real robustness to the ratings system would be to introduce respect voting (where the actual ratings are weighted dependant on the "respect" value of the voter) but as this is a pain in the ass to implement and isn't suited to sites with high member fluctuations, it's probably best to leave it as it is. Probably...