Sure enough, there is a delete group button.  I don't know why I thought there wasn't a way to do this, try it.

In the lower part of the group editor window, there are buttons for "add group" and "add material".  If you wanted to rename a group, you could:
Go to the Setup room
Select the bodypart you want to rename
Rename the appropriate bone
Open Group Editor
"Create group" button
"Add Group..." button
Select the original bodypart from the pulldown box that pops up

Then to remove the old group:
Select the old group in the Group Editor's pulldown box
Press Delete Group (if that works)

Odd little blind spot I had there, I never realized the Delete Group button was there.
 

My Freebies