We had to do a realignment and had to use denatured alcohol to dissolve the ballast holding the track in place (it also attacked the cork underneath.)
Matt,
What bonding agent did you use to secure the ballast? Regular white glue will easily loosen up with a water.