NASA overspent $15m on Oracle software because it was afraid an audit could cost more
Houston, we have a problem: Millions wasted on license penalties
NASA is rubbish at software asset management, has not implemented federal government guidance on how to address it, and as a result is spending too much on code it doesn't use – including $15 million on unused Oracle software alone, under a twelve-year-old license the space agency was afraid to examine.
So says the aerospace agency's Office of Inspector General, which on Thursday published a report [PDF] that opens with the unflattering observation that NASA's software asset management (SAM) practices "currently expose the Agency to operational, financial, and cyber security risks with management of the software life cycle largely decentralized and ad hoc." The report rates NASA’s SAM capabilities as "Basic" – the lowest ranking on the four-tier scale the US government employs.
It gets worse. The report finds that NASA hasn't embraced best practice, or implemented the SAM systems that discover, inventory, and track license data as required by federal policy. The agency's Software Asset Management Office and Software Manager positions "are misaligned and do not report to the Chief Information Officer as required by federal policy."
NASA is examining 'how and why' Oracle licensing became so cumbersome and complex to manage
Nor does NASA have a consistent process for negotiating with software vendors, or handling license audits. The report suggests NASA is therefore exposed to higher costs and penalties for violations of software license agreements.
The report uses the example of NASA's Oracle deal to demonstrate the issues, detailing how the agency was "unwilling to risk a license audit by Oracle because of the lack of solid, centralized visibility into deployment and use of the software."
Officials in NASA's office of the CIO told the Office of Inspector General they "knew better than to try our luck with an audit."
"Simply put, merely the potential threat of being audited by the vendor encouraged overbuying when the accuracy of Agency Software Asset Management was suspect," the report states.
NASA therefore spent $15 million on Oracle software it didn't use. And it's probably been spending too much with Oracle since 2011 – the year when it signed up with Big Red to manage the end of the Space Shuttle program.
"The Oracle license overspend has been in effect for more than a decade," the report states. "The Agency has not sufficiently tracked the full cost of license expenditures for the life of the existing contract which includes multiple option years in a manner which would allow the full costs to be known."
The auditor estimates NASA "could have saved approximately $35 million over the past five years in fines and overpayments ($20 million in penalties plus $15 million in Oracle overspend)" and is therefore questioning the costs.
The report acknowledges that "funding and staffing shortfalls" have contributed to NASA's poor SAM capabilities.
But lock-in hasn't helped, either.
"NASA purchased large amounts of Oracle products to support Space Shuttle processing and other mission operations during that timeframe containing licensing terms that made transitioning to a competitor difficult due to proprietary technologies," the report explains.
- NASA boss says US may lose latest space race with China
- NASA retires Mars InSight mission after it enters ‘dead bus’ condition
- NASA awards $60m to Texas biz for 3D printing future Moon base
- Oracle's compliance cops now include Java in license audits
NASA's Oracle licenses are due for renewal in April 2023 and the report states that the agency's officials "are gathering requirements and examining 'how and why' Oracle licensing became so cumbersome and complex to manage."
"In parallel, the Agency is also reviewing the current and desired licensing environment to quantify the true cost of doing business with Oracle."
Another revelation in the document is that NASA paid $4.36 million in software license violation penalties during FY 2021 alone.
NASA was able to negotiate some fees down to zero but sent $3.85 million to SUSE and $415,000 to SAP. The auditor suspects other payments may have been made over the last five years – probably to the tune of $20 million.
On top of the Oracle mess, that's $35 million of bad software spend, which the auditor thinks could have been avoided had SAM been in place – at a likely cost of $3 million to implement and $2.5 million a year to operate.
The report also offers the following unpleasant observations:
The report calls for NASA to get SAM right immediately. Until it does, "the Agency risks procuring software in a costly and ineffective manner, as well as incurring tens of millions of dollars in penalties for license non-compliance."
NASA management largely accepted the report's findings and stated that a SAM pilot will commence in October 2023, but that agency-wide implementation will not be complete until 2027.
The auditor has previously found that NASA has sub-par cyber security and grossly underestimated the cost of a cloud storage migration.
Seriously NASA, it's not rocket science. ®