IIRC, normal mapping has a better impact when viewed straight on like a bump map as opposed to viewing at an angle.  The only diversity to a normal map is the how directional light interacts with the material shader.  But, I think, the sticking point here is when generating a normal map, you need to take account which direction the light is coming from (X,Y,or Z).  If, for example, you create a normal map with light coming from left to right, and you use the mapping with light coming from above in the scene, then the effects way not appear correct.  Normal maps can impart more realistic views on low poly structures, as in gaming, but they are reserved for distant background scenes.

Backgrounds, normal maps are applicable; closeups, bumps and displacement.  And, also they are set, not variable as Richardson mentioned.