Here is my formula....all the track and ties gets spray painted with Rust-oleum Camo Brown. It is a very flat color. Than I spritz some Camo Black over everything in a hit or miss random pattern. Clean off the rails with lacquer thinner. Almost all of this was done outside in the fresh air or if I had to do it in the train room, I made sure I had adequate ventilation and a supply of fresh air!
For ballast, I used Brennan's Better Ballast. I got the gray color. It is real granite and scale sized. I used about 50 pounds for the layout. Check out my article last month in Run 313. The paint and the ballast obtained the look I wanted.
My opinion, I do not care for the "red" color or rust primer. Just does not look right to me unless used on a siding or something and then covered a little with the camo brown. Remember, just my opinion.