Well the plot thickens, and I may have been wrong about some of my determinations. See blow:

So I also use a GTX 1050 Ti, but I don't usually use it to render. It just drives my monitors. These tests were performed using the same test scene above (Femme default scene at 1440x1080), with the same render settings (GPU Medium Quality).

1. As you can see from the log, if I use the 1050 Ti and the 1080 Ti together then I get a much more reasonable result: 37.56 seconds.
2. The GTX 1050 Ti by itself actually renders quite quickly as well: 111.17 seconds.
3. The GTx 1080 Ti by itself though, then takes 691.01 seconds. So the 1080 Ti is operating much more closely to its proper abilities when working with the 1050 Ti.
4. Out of curiosity, I did a CPU render with the same "GPU Medium Quality" settings which gave 238.69 seconds. Much faster than the GPU render, though not up to the 1050 Ti speeds. But then I was using GPU settings that were not optimal to a CPU.
5. Using "Medium Quality" (just regular Medium Quality, not the GPU one) I get a much better result on the CPU of 43.98 seconds.

So I may be wrong that my CPU renders are out of whack as well as my GPU renders. I said that based on some test renders I did using CPU Ultimate Quality settings. Those tests weren't performed with the settings I would normally use for a CPU render. So that is my mistake I think. CPU renders seem like they might be fine on my machine given results above.

So this begs the question of whether it's an issue with 1080 series cards, or is it related to having multiple cards in your machine?

Can anyone confirm that they run multiple GPUs, not SLI, in a single machine and do not have any render time slow downs?