I think  I have discovered the problem...

I didn't delete the inner layer when I shelled, so there's this hollow effect with two layers of cloth for no reason.  Second, when I extruded, I forgot to check "By local Normals" (or whatever) and some of the extrusion was inverted instead of raised.

I have fixed all this and regrouped and  am about to try it out...