Fork/frame issue

I'm trying to swap forks on 2 of my bikes. The issue i'm running into is that the fork won't sit flush against the head tube. If I remove the bearings, it will. Would swapping these parts pictured below solve my issue? If so, how can I remove them?

Thanks!Image:...