I also agree with the suggestion to create a tunnel through the upper section of your closet. On my shelf railroad (RIP '08) I tunneled through the Domestic CEO's Kitchen Pantry to avoid an awkward curve arc situation--see rear track with passenger cars on lower shelf.
She was not happy with the tunnel, but I reminded her that the deal was to "deed" me the 14x32x20'h Kitchen/Breakfast Room wall space for removing layout from her Den Carpet. But I eventually had to agree not to run trains while she was watching Oprah..