My layout's ballast is Woodland Scenics Gray Blend Coarse, and it is rock. Probably limestone. Most of the ballast I see around here on the CSX is modeled after the Woodland Scenics product (bad joke alert). The CN (ex-GM&O) track work shows a warmer, tan-ish shade of stone. Of course, ballast sources vary. I do not know how much size variation RR's typically allow in their ballast specs. Must be a bit.
Recent track work on my layout using WS Coarse Gray Blend; GG switch: