Best way to delete them all is to go to Edit > Preset Manager and select Gradients. Click top left gradient, hold shift and click last gradient and hit delete. 

Yes, you do have do your gradient and click New to add your new gradient to top section. Clicking Save will save a gradient file consisting of everything in top section.