Try getting the best fit at frame one then export out the clothing. Then delete the clothing and import the saved version. This will effectively have the fit baked in and possibly be a better starting point for the cloth simulation.