I have Gargraves/Ross track throughout the layout. I also have a 'coal yard' area with several coal-handling accessories grouped and interactive. But, sadly, I never made room for the 497 loader. So I can't give you first-hand experience on your question.
However, I did find a YouTube video on the 497 integrated into a layout with FasTrack. The vid clearly shows that the owner/operator simply adapted a length of Lionel tubular track within the loader zone, FasTrack on either side. Of course that's pretty much the simple way of resolving the problem...even though it disrupts the appearance/continuity. I know Gargraves sells the adapter pins...std O and O27 versions...for making such a transition.
Lionel also re-made/improved this accessory, 6-14107, around 2003. The link photo shows this version simply used a Lionel UCS Remote track. I can only imagine that the accessory base was modified from the original version to accept the UCS. BUT...if that's the case, Gargraves makes their version of the UCS, s/n 108. Ergo, you could modify your accessory's base to accept the 108 Operating Track, thereby giving you a more consistent appearance.
Finally, I wonder that you might be able to remove wood ties from a section of Gargraves track to mimic the rail/tie arrangement of a couple O27 pieces of track, and simply work from there. After all, they're both 'tubular'...Lionel and Gargraves. You might have to do a little fitting/filing here and there, but it should be doable. Again, no experience here with an original version of the 497 myself.
Just some thought-starters. Can't believe someone hasn't dealt with this problem before. I've posted before with no, or long tardy, responses. Hang in there!
BTW...OTOH, you may be the first to achieve this adaptation!! Please share with us how you resolved the situation...text, photos, video, etc.. Or, write an article, submit to OGR, get some moolah for your effort! And then enjoy your celebratory vacation to Tahiti!!!