If you don’t have a lot of space then you want O-27. I would check craigslist, I bet you could get a good used set for around $70. If it has to be new-in-the-box and you want fastrack, you can find a 4-4-2 steam set with an 80 watt transformer for around $150-160, the Pennsylvania Flyer sets are actually very nice. You can swap out the curves for 0-31s.
From there I would buy a Lionel Pullman passenger expansion pack, and a pair of alco diesels from Williams or maybe a used GP off of ebay. That gives you one steam engine, one diesel pair, and both a freight and passenger consist. Then I would STOP buying rolling stock and put everything else into the scenic items and track.
If space is really, really tight, you can do an all-traction layout using Superstreets/EZ Streets D-21 curves.
