It's my belief that the cloth room doesn't work very well with small, intricate geometries. The fingers of gloves - whether on a hand or not - would fall into that category, if it's true. I tried clothifying transmapped hair once, and the simulation went screwy around the figure's ears.

You might be able to get better results if you scale everything up by, say, 10 before simulating?