I left N scale because I was sick of the rivet counters and it got worse every year. I run what I want and people enjoy it. I have fun with friends' kids when they come over and get mesmerized by the operating accessories under the Christmas tree.
I appreciate the beauty and detail of the scale locomotives, and cars. I admire the detail in many of the scale model structures, and the skills in weathering rolling stock.
Except for a couple of exceptions, the scale people smile from the memories of what they had as a kid and the accessories.
So do what you want. Heck I'm keeping one N scale train so I can have a 1/4 scale live steam park on an o scale layout.
You can always make a 3 rail layout for under the tree and let the grandkids play with it.
Introduce a younger generation with what you had as a kid. The options are endless. Just have fun.