I tried that approach, just using the shelves for display and other locations for storage but I just have acquired to many engines and too much rolling stock. I have some Glenn Snyder shelving on the wall and a bunch of wooden shelves I have built around the bottom of my layout.
Now as I switch stuff out, I just do my best to get everything in a proper storage place but that can be hard. I always seem to end up with some rolling stock tucked in a corner of the basement because there are no places left.