I would think the big issue with property ownners is LIABILITY! I would not allow the train to run on my property unless I was absolutely clear of anything done by the railroad.
And also there would be those who would trespass on property beyond the ROW.
Was there any consideration to buy the ROW, and seal it off?