Yes, it is tedious, but there is no way around it.

You have to work in small steps to get closer and closer to the shape you want.

Always smooth, switch between main and bulge map, and fix up the parts that you smoothed too much.

Baby steps, but eventually you'll get this:

The general shape of the shoulder is now much better and the crease is smoother and starts lower.