With yellow in the map it suggests you have a map in 'World space' instead of 'tangent'.

I don't know if that's the cause of your problem but there are quite a few programs that can convert Normal maps to 'Tangent space' format. Its more common.

xNormal for example.

regards
prixat