The ballast is roofing granules. I bought it in pails at a Louisville roofing supply house. You can't get it at the home centers. They apparently make it in many colors, but I was only able to get "white". It's actually a light gray. It cost about $20 for 50 lbs so it's quite inexpensive. It's a manufactured product, not small stones. It also contains an iron compound as a mold and mildew inhibitor and therefore is SLIGHTLY MAGNETIC! This means that it can (and will) be attracted to the wheels (and gears) of locos with Magna-traction. It did this with my Lionel late-model F-3s and stopped the engines from running. If you run non-magna-traction engines it's great stuff.
For the yard ballast, I used alcohol plus India ink for the wetting agent before adding the scenic liquid cement. This dulls down the brightness a bit.
