Hi, Neyjour.

I hope you don't mind me taking liberties with your pic...

I'd like to try to help, but 1st we need to narrow down exactly what the problem is, then it becomes easier to suggest solutions.
In my doctored pic, the area outlined in red is the area I think you're having problems with.
The area outlined in yellow is another possible area, but I don't think you're referring to that (that looks more like a lighting artefact).

What I think is happening is that the cloth near the neckline bends towards the figure, to give the illusion of thickness (the area highlighted in green is what makes me think this).

If what I have stated above is correct, then the effect you are seing is exactly what one would expect.
Displacement works right-angles from each face in the mesh (along the "normal"). You can see this happening where the cloth bends over the shoulder - there are low and high parts showing up, giving the "fuzzy" texture you want to get.
Unfortunately, the same effect is happening along the neckline (and for the same reason).
If all the above is true, then there is no quick'n'simple answer that I know of.

The most obvious answer would be to create a separate material zone along the neckline (as a hem) and apply bump to it, instead of displacement..

Are you working on your own mesh or on something created by someone else?
Is it dynamic or conforming cloth? (I suspect conforming).
Since you want to create a texture pack, I suspect that it's not your own mesh, so using the grouping tool to create a hem isn't a solution.
Do you have existing texture maps (IE .jpg images that can be applied to the clothing)?
If so, you could create a hem on the texture map (as a greyscale image) and use it to create a cross-over from the displacement to the bump.

The technique is not difficult to do, but difficult to describe in words. I'm not going to go into details here yet, since it presumes that all be assumptions above are correct...

Cheers,
Diolma