Sam,

Here's what you said, using math:

Letting M represent the mask, and D represent the original unmasked displacement:

Blend(M, D, M) <---- this is what you described

Definition of Blend is

Blend(a, b, f) = (1-Clamp(f))*a + Clamp(f) * b

Since M is being used as f, and M is, by definition, already in the unit range [0, 1], the Clamp aspect can be ignored.

So, you have

Blend(M, D, M)

= (1-M)*M + M * D

However, the goal as I said earlier is to multiply M with D, so that you have M * D.

The Blend includes M * D, but also adds the term (1-M)*M which is interesting nonsense.

When studying any formula, it is instructive to examine the low, medium, and high input values.

For a map, the low is 0, medium is .5, and high is 1. Let's use this notation: [l, m, h].

So what does (1-M)*M produce with that vector?

1-M = 1-[0, .5, 1] = [1, .5, 0]

So (1-M) * M is:

[1, .5, 0] * [0, .5, 1] = [0, .25, 0]

What does this mean?

It means that where the mask is 0 or the mask is 1, this term is 0 and will have no influence on displacement.

However, where the mask is in transition between black and white, there will be shades of gray that will produce non-zero values. These will create a raised outline of the mask shape. This is not at all desirable.

---

Renderosity forum reply notifications are wonky. If I read a follow-up in a thread, but I don't myself reply, then notifications no longer happen AT ALL on that thread. So if I seem to be ignoring a question, that's why. (Updated September 23, 2019)