After some measurements with O72 curves and the limits of the wheel slop, I've decided to remove 1.25mm from the inside of the wheels an put a very slight chamfer on the inner edge of the wheel. That still allows margin to prevent the wheels from falling off the rail in a turn in either direction and looks like it allows sufficient clearance for the switch rail.
I'm still going to fix my switches to eliminate this issue, but I have to fix this locomotive anyway.