My approach is to modify the Gloss/Specular Converter node, delete the Constant value node, and substitute the Color Math Node for the BrightContrast node.  Note the image below.

                     

Modifying the compound node, I can control the chip color at one source: the PhysicalSurface node.  Now, you have a slider to control the Specular map brightness.