I've found that white clothes with nearly ANY reflection setting on it, turn out looking like plastic or way brighter than anything else in the sceen.  I always turn the highlight color to a dark dark grey and turn reflection off.   Unless you are looking to create the look of latex or satin/shiny, it tends to make white look "off"...... and glaringly bright.