Silverblade - with respect to regular waves following the coastline - I got the attached image by adding a quick filter (just added 3 dips to the default straight line filter) between the coastline mask and the map filter. Would need a bit of refining, but seems to work.

Is this what you wanted?

The thing is  the waves generated by the Vue's water editor dont look consistent, being all parallel rather than follow the coastline.  (That's true in general, not only for my examples).
I imagine I should be able to modify the waves too using a coast line mask techniques...

"I paint that which comes from the imagination or from dreams, or from an unconscious drive. I photograph the things that I do not wish to paint, the things which already have an existence."
Man Ray, modernist painter
http://artpearl.redbubble.com/