I did a similair thing 5 years ago. At that stage I was still churning out two or three renders a day non of which were very good, but I was addicted to Bryce. I left my job in a hotel which I had been at for 6 years and became a self employed window cleaner. This gave me twice the income for around 25-30 hours a week as opposed to 40 hrs at the hotel. Obviously this gave me more time to do my hobby of Bryce. 
Whilst i don't actively look for client based work, I've had a couple of real nice comissions in that time from people coming across my various galleries on the web. Yes window cleaning is boring and repetitive, but at least I'm my own boss and the stress of my old job doesn't exist.
So I say why not leave your job but try to get another that will allow you to do a similair thing as me. Having a job that will allow you the flexibility to concentrate on establishing your graphics business will help to alleviate any worrys about money. When I do get a commision then it's a lovely bonus which has allowed me to take holidays and buy things I wouldn't normally. If you put yourself about and self promote I'm sure you will be able to concentrate solely on your graphics in the near future. I just don't like the uncertainty of not having a fixed income to fall back on. The main thing is this, it's better to regret something you have done, than something you haven't. Oh and no, I don't have any qualifications regarding art, I'm totally self taught.

http://joevinton.blogspot.com/