Plugging a map directly into alternate diffuse will result in glow.

A simple test is to render the scene without any lights and IDL turned off. It should render black. If it doesn't, the shaders are wrong. IDL is simulating realistic light behavior, and that can only give realistic results when the scene you put into it is realistic too.