Amtrak back on track after server breakdown forces dozens of cancellations
Sounds like some IT teams are due for some training
Amtrak trains across the US are back on schedule after a server breakdown forced the rail service to cancel dozens of routes across the American Midwest and parts of California.
In a statement released over the weekend, Amtrak attributed the sudden spike in cancellations to "intermittent communications issues" with a rail-industry control system.
"Amtrak is among most railroads, including five largest freight carriers, terminal railroads, commuters/regionals and some short-lines who are clients of a system operated by Wabtec Corporation," the transport biz told The Register. System design and architecture questions are best answere by them." We'll update if we hear back from Wabtec.
As we understand it, this system is what allows Amtrak's passenger lines to safely use railroads built for and operated by freight rail lines. Amtrak began experiencing "intermittent" issues with the server responsible for this process early Friday, with trains traveling in and out of Chicago — the third largest city in the US and home to 2.7 million people — bearing the brunt of the disruption. However, we're told that services in Los Angeles and the San Francisco Bay Area were also impacted.
Initially, Amtrak reported that the server issues were causing delays of 30-60 minutes and some trains were running at "reduced speeds." However, when the issues persisted, Amtrak began issuing a slew of cancellations Friday into Saturday.
A rough count of cancellations posted to Amtrak's alert account on Twitter indicates that more than 70 routes were either canceled or delayed as a result of the system's breakdown.
"We worked around-the-clock with our partners to continue troubleshooting and addressing these issues, which were resolved early Sunday," an Amtrak spokesperson told us. Customers whose routes were canceled have had their fees waived.
- FAA sets 2024 deadline for preventing 5G crash landings
- Vessels claiming to be Chinese warships are messing with passenger planes
- FAA grounds all US departures after NOTAM goes down
- Southwest promotes internal IT executive to CIO in wake of that Christmas meltdown
While we have some sense of the outage's scope, it's not clear what actually triggered the breakdown in communications with the rail-industry control system. Amtrak didn't directly address our questions when asked about the nature of the outage; we've followed up and will let you know if we hear anything back.
US transportation system buckles again
Amtrak's service disruption marks the latest in a string of IT related headaches which have plagued the US transportation system in recent months.
In January, the Federal Aviation Administration ordered airlines to pause all domestic departures after the Notice to Air Mission Systems (NOTAM) system was taken offline by a configuration error. The system relays essential information to flight operations personnel.
And just this weekend the FAA had another embarrassing outage when it appeared to lose two nameservers within its systems, causing widespread disruption.
Meanwhile, in December, thousands of Southwest Airlines passengers were left stranded after an IT breakdown compounded by winter storms and staffing shortages crippled operations.
The issue, it was later revealed, was outdated scheduling software for mating crew members to planes. For a brief period, the airline was forced to schedule flights manually.
Aging equipment and systems in US transportation is nothing new, though several larger airlines — including Delta and America — have recently transitioned to cloud-based offerings as a means to modernize their systems. ®