Red Hat has warned that hackers were able to commandeer its systems and tamper with code - but said that since its content distribution was not hit, it is confident that polluted code has not served up to users.
The first hint that something was wrong came last week when Fedora rebuilt its systems, a reconstruction that was accompanied by extended outages. Red Hat sponsors the Linux distribution. Fortunately Fedora packages weren't interfered with following the attack, but Red Hat Enterprise Linux packages were touched up by as yet unidentified miscreants.
"Last week Red Hat detected an intrusion on certain of its computer systems and took immediate action," Red Hat said in a critical security advisory issued on Friday. "While the investigation into the intrusion is ongoing, our initial focus was to review and test the distribution channel we use with our customers."
While checks on its content distribution networks came back clean, it did turn up some problems.
"An intruder was able to sign a small number of OpenSSH packages relating only to Red Hat Enterprise Linux 4 (i386 and x86_64 architectures only) and Red Hat Enterprise Linux 5 (x86_64 architecture only).
"As a precautionary measure, we are releasing an updated version of these packages, and have published a list of the tampered packages and how to detect them here."
In a parallel posting to the Fedora announce mailing list early on Friday morning Paul Frields, Fedora project leader, confirmed that an intrusion by computer hackers had prompted the unprecedented rebuild by the Linux distribution, which is sponsored by Red Hat.
"Last week we discovered that some Fedora servers were illegally accessed. The intrusion into the servers was quickly discovered, and the servers were taken offline.
"Security specialists and administrators have been working since then to analyze the intrusion and the extent of the compromise as well as reinstall Fedora systems."
Among the compromised Fedora servers was a machine used for signing Fedora packages. Following a forensic examination, the Linux distribution is convinced that hackers were not able to capture the passphrase used to secure the Fedora package signing key. "Based on our review to date, the passphrase was not used during the time of the intrusion on the system and the passphrase is not stored on any of the Fedora servers," Frields writes.
Nonetheless, as a precaution, Fedora has changed its signing key. Access to the key would have potentially allowed hackers to offer up code with built-in backdoors carrying the Fedora hallmark, the risk Red Hat is grappling with in the case of the doctored OpenSSH packages.
Fedora has carried out checks that suggest the integrity of its packages and source code have not been affected by the breach. It said it was simply playing it safe when it advised users to hold off from downloads last week, a piece of advice that stoked speculation that a security breach was behind the then unexplained outage.
"The effects of the intrusion on Fedora and Red Hat are not the same," Frields added. "Accordingly, the Fedora package signing key is not connected to, and is different from, the one used to sign Red Hat Enterprise Linux packages." ®