I'd consider some small shims on the axle to keep it from going that far to one side. I trim bread closures round and use those as shims, slip them over the axle on each side and you should be good. They come in various thicknesses.
Cut on the red line and slip it over the axle on each side.