Share 'Fret work scratches....'

Hey all. I have been seeing crowning scratches towards the tops of my frets after dressing. I usually go 320, 400, 600, 0000 Steel Wool. Is there something I might be doing wrong when using the Stew Mac #300 offset diamond file? I have been going back to chase them but it just seems like a terrible use of time if I can just change something while I am working I would rather do that.