Have had very little problem with juvenile visitors to the layout. But I usually change out the trains to some marx or williams, cheap stuff with not value. But still, have had little trouble. I also understand that they'll push things a little when running trains.
Now, if you want to talk a bout a fifty year old picking up a 1955 GG1 by the stripes at the local TCA show....I've had that problem a few times.
Tim