It's all about your light, and definitely changing the default light colors to.. what you'd find in the real world. you can sample colors from

http://www.itchy-animation.co.uk/light.htm

and you will notice a near instant improvement in the quality of your renders. Once your lights are good,  then you can look into tweaking Firefly settings. Finally, your can auto color / auto contrast in Photoshop, but that will only work when your lights are good.