Generally you want to start with a relatively low number of pixel samples so you can get some quick renders to access textures, lighting, etc. But for a final render you have to crank those pixel samples up to get rid of the noise. Depending on the image, I've used 30-60.

When I do test renders in Firefly (rather than Superfly with a low number of pixel samples) and then re-render in Superfly, I sometimes find that an image that looks like it is posed well is not quite right. The greater accuracy on shadows and lighting really reveals where e.g. a hand is floating above a surface and not actually touching it. Firefly is more forgiving about depth. Once you've done the work to fix those things, I think the results are usually worth it.