You are comparing completely different services, though, right? Private freight lines versus passenger? Amtrak runs on lines owned by the freight companies. I've been on an Amtrak train that had to sit on a siding while a freight train passed. The passenger service has to have a published, predictable schedule. The freight moves when it is needed and breaks the Amtrak schedule, leaving a crap passenger service.
I think the extent to which freight shoves aside passengers varies from region to region. My worst experience was going to West Virginia from DC. I find the Vermonter keeps its schedule pretty well.