Massive outage grounded US flights because someone accidentally deleted a file
Our lives are in your hands, and you have butterfingers?
The US Federal Aviation Administration says its preliminary investigation of last week's system outage that caused the first nationwide grounding of flights since September 11, 2001, has uncovered the cause: contractors accidentally deleted some essential files.
In its first word on the outage since January 11, the day the FAA's Notice to Air Mission Systems (NOTAM) went offline, the agency said contract personnel were working to correct a synchronization issue between the live primary database and a backup copy. In the process, some incorrect keys were apparently pressed and more than 11,000 flights were grounded.
NOTAMs are notices of changes that may affect flight plans, like construction, weather or other emergencies. Pilots on long-haul flights can be stuck reading through hundreds of pages of NOTAMs before taking off; in short, they're pretty essential.
The outage last week was relatively brief, and only saw flights due to take off in a roughly three-hour window delayed or canceled before the FAA said everything was restored at 0900 Eastern Time.
Flights that were in the air or due to land during the window were unaffected by the outage, the FAA said.
The regulators maintain the outage showed no signs of a cyber attack – just plain old butterfingers.
- FAA grounds all US departures after NOTAM goes down
- FAA sets 2024 deadline for preventing 5G crash landings
- FAA wants pilots to be less dependent on computer autopilots
- FAA asks for vendor feedback on $10b tech contract
"The FAA made the necessary repairs to the system and has taken steps to make the NOTAM system more resilient. The agency is acting quickly to adopt any other lessons learned in our efforts to ensure the continuing robustness of the nation's air traffic control system," the FAA said in a statement, adding that its investigation is ongoing.
While it's reassuring that there wasn't a malicious actor involved, the FAA still gave little information as to what actually happened. There's no indication in its update of what sort of file was deleted, whether any data had been irrevocably lost, exactly how a file essential to the operation of an important production system was able to be deleted or whether the contractor(s) involved had been fired.
We posed these questions to the FAA, which responded with the official statement that answered none of these points.
Lawmakers want answers, too
FAA acting administrator Billy Nolen reportedly plans to hold a briefing for lawmakers and staff later today.
US lawmakers had stern words for the FAA in the days following the outage, when a group of 120 Congressional representatives wrote to the agency demanding answers.
The representatives told Transportation Secretary Pete Buttigieg that flight delays and mass cancellations had been on the rise over the past year, and argued that some of the issues have been within the FAA's control. Included in the letter is a list of questions the signatories say they want investigated; Buttigieg and his people have until January 25 to provide answers.
"The failure to improve legacy systems is unacceptable, and the American people expect and deserve better," the representatives said. ®