I was shocked to hear the difference between my hand-laid track on wood ties and the dirtbag Atlas plastic ties. I hate plastic, but it is way quieter!
Milt Sorenson used outdoor carpeting under his TruScale roadbed, and it was dead-quiet! I don't think the product is still available - it was like a 1/4" sheet of porous rubber, with simulated grass on one side. No nails - just glue.
Ballast is definitely a noise transmitter - but maybe if you isolate it the noise stops? Some folks tried rubber ballast, but it never caught on.