Try one of the presets that you see when you click the "Load Preset" button. Start with one of the Medium settings (there are some for CPU and GPU). If you need higher quality move up to High. And Ultra gives the best quality.  Naturally, the higher the quality, the longer the render will take. The Options menu should also allow you to choose OptiX since you have an RTX card.

The settings I show here are for OptiX High