Probably there is no setting to avoid the Originals.
But if it'll help ease your mind, imagine animating 3,121 frames, with the background being separate from two foreground layers. After a while you're like: meh, originals. Lett-em stay. They actually come in handy to blend between the original, and the denoised version which tends to be too blurry.

Don't look at it as a hindrance, but instead look at it as an unexplored opportunity.