Would it be beneficial to have a lift out/lift up section lined up with the closet? If you put the lift out lined up with the closet, having the lift out removed might give you more work space in your work closet... or did I misunderstand? If you intend to work in there (the closet) having extra chair space might be needed.
I might consider putting your yard ops along the top wall, as it is the widest (left to right), and could be as much as much as 24 or 30" deep. I might make the lift out section the narrowest and the top section the deepest. I would make the center area, in other words, as small as comfort allows, while still allowing you to reach the furthest track.
I have it drawn as 3' x 6' -- if there are going to be two people inside, that would be the minimum for two fat old guys. (Me x 2) If someone else wants to watch from outside, then you might be able to make it smaller in the middle.
How important is scenery to you? In tight spaces you can have lots of track everywhere and no room for industry or structure or even an empty field.
Good luck. It's your railroad and your home.
