I've used used coffee grounds for 30 years in many different climates without issue. I agree that unused coffee is probably a bad idea, since it contains many volatiles.
I also agree that real dirt is great. But I don't see it as either/or. You want many different textures. Earth and coffee grounds complement each other nicely. Peat moss is good, too. All of these old school materials are documented in old Lionel literature--e.g. the Model Railroading book. Tried and true.