This can be a problem when you actually want a picture full of sky, with atmosphere but no ground. Deleting the ground plane doesn't give you a 360 degree sphere full of sky.

One trick that I have used successfully is to give the ground plane a mirror texture.