You could try different codecs and see if still happens. As a general rule, you should prefer rendering to a series of still images over rendering directly to a movie file. For example, you can try different movie codecs and compression ratios without having to rerender the whole thing over and over again.