Wow, lots of good suggestions. A couple of details to add. The tall ship in the Inner Harbor is the former USS Constellation, a mid-19th century "Sloop of war" that looks like a traditional man-of-war frigate.
Often over looked is the US Coast Guard Cutter Taney. Last remaining ship that was present at Pearl Harbor on Dec 7th.
The Ellicott station is said to be the national's oldest. Examine it closely, and you'll notice the back section was in the past an engine house, with the remains of a small one/half turn table pit in the front. While I haven't been there in a number of years, if you look at the roof line, you'll see the smoke stack from the stalls are near towards the center of the building, not the ends; perhaps to support the B&O famous vertical boilered "Grasshopper" locomotives.
The Thomas Viaduct at Relay, MD, if you enjoy the 1:1 models, is an excellent location to railfan.
At the B&O museum if you take the 1st mile ride, you can't help to notice the hulk of PRR GG1 4876 in the deadline. This GG1 is famous for crashing into Washington Union Station in 1952, ending up in the basement of the station and being removed later in pieces where it was put back together and back into operation.
In the roundhouse, you might notice the lower parts of the walls are painted black to about the height of a person. This was done to hide the grime stains from when workers would lean against the walls during breaks working on the cars that were built and maintained in the round house.
Finally, a bit south of Ellicott city is the Bollman Truss Railroad bridge in Savage Mill, MD. Built in the 1850's the bridge is perhaps the oldest iron bridge in the US and represents the earliest design of an all metal bridge. Today, it carries a trail, but is a neat comparison to the massive stone Thomas.
Bob
