The biggest issue was their reliability. Typical government specification had the requirement of 160mph when modified Silverliners were doing 156 already in the mid 60's. Budd and the PRR rushed these into production without a whole lot of testing and it took most of 67 and 68 to get them into regular service.
They ran fairly well until just before the mid 70's and then went into the rebuild program. GG1's pulling Amfleet held the Metroliner schedules until they returned to service in 77 or so. By the early 80's the Metroliner cars were working the Philly the Harrisburg service and AEM7's with Amfleet cars were pulling Metroliner trains.
I saw the Metroliner cars twice as a child. The first time was at Metropark, NJ on a trip to DC circa 1977. We were riding an E60 pulled "local". The second time was outside of Paradise, PA in the early 80's after the rebuild.
These cars were certainly fast and ahead of their time, but ultimately the reliability lead to their demise. Too bad, these still look modern and sleek today.