So you have not yet tried the entire mechanism without the flywheel? Will you put a different motor in there and drive this flywheel? That would be an equally valid test of the flywheel. If it is smooth, then I get to eat that crow.
Don't forget - that anemic little high speed motor probably went to triple its normal rpm when you held it in your hand. It was screaming like a JT8D probably.