I figure when I make my next move the trains and any layout will come with me. I've pretty much heard the same thing from real estate agents that no one wants your "stuff" in their new home. As stated above a train layout is not a selling point. It appeals to a very, very, small demographic.
I would also never buy a home with a layout as I wouldn't want to inherit anyone's track plan or thought process.