Raytraced DOF needs a lot, a lot of anti alias to avoid the noise. If you use Infinite, use the hybrid dof instaed, with 3 or 4 passes. It is less accurate, but renders a bit faster, and it's noise free.