In general, in 3d graphics, the easiest way to make things look like they do in real life, is to recreate the real life conditions as accurately as possible.
Adopting this principle in your workflow early on will maybe be difficult at first, but it will give you a good insight in what you're doing. Eventually it'll help you create whatever visual look you're after more easily and efficiently.
It will also mean your textures, models, light setups etc. u make can be more easily re-used in future projects if so required. 

Basically, for ur current problem, this means distant mountains, should in deed be located in the distance. That way the hazy effect builds as you would expect in real life.
In some real life scenario's you will also find some low altitude clouds between the viewer and the mountains in the distance. This is why the earlier suggestion of cloud planes can also work great in some situations.
Actually making the mountain objects very large (like real life mountains) works great as well, but i guess thats required anyways if u put them far, far in the distance.

(_/)
(='.'=)
(")(")This is Bunny. Copy and paste bunny into your signature to help him gain world domination.