FYI: Ticking time-bomb fault will brick Cisco gear after 18 months

Replacement units will be on their way – just don't call it a recall


Updated Cisco has issued a warning that an electronic component used in versions of its routing, optical networking, security and switch products prior to November 16, 2016 is unreliable – and may fail in the next year and a half, rendering affected hardware permanently inoperable.

"Although the Cisco products with this component are currently performing normally, we expect product failures to increase over the years, beginning after the unit has been in operation for approximately 18 months," Cisco said in its advisory.

"Once the component has failed, the system will stop functioning, will not boot, and is not recoverable."

And without naming names, Cisco said that the clock-signal-generating component is also used by other companies. Expect further notices of this sort from other vendors shortly.

Cisco said it learned about the issue in late November and has worked with the component supplier to fix the faulty part. As a result, currently shipping products are not affected.

For customers with affected products under warranty or covered by service contracts through November 16, 2016, Cisco intends to provide replacement products. It is prioritizing replacements for those who have been operating the affected products the longest, because of the correlation between operation time and component failure.

Cisco insists this isn't a recall; rather it's a proactive replacement. The company also said that while the component maker indicated failures will become more likely after 18 months, it expects it will take three years of runtime before its products show a spike in failures.

In its advisory, Cisco specifically declines to identify the supplier that made the faulty part, or other affected vendors. And company declined to do so in response to a request from The Register for further information.

"Cisco strives to deliver technologies and services that exceed customers' expectations, and meet rigorous quality and customer experience standards," a company spokesperson told The Register in an email. "We became aware of an issue related to a clock signal component manufactured by one supplier. We have worked with the supplier to resolve the issue, and we're providing information and support for our customers."

Cisco's advisory affects the following products:

Networking FN-64230 NCS1K-CNTLR
Routing FN-64231 NCS 5500 Line Cards
  FN-64252 IR809/IR829 Industrial Integrated Services Routers
  FN-64253 ISR4331, ISR4321, ISR4351 and UCS-E120
Security FN-64228 ASA 5506, ASA 5506W, ASA 5506H, ASA 5508, and ASA 5516
  FN-64250 Cisco ISA 3000 Industrial Security Appliance
  Meraki Notification MX 84
Switches FN-64251 Nexus 9000 Series N9K-C9504-FM-E/N9K-C9508-FM-E/N9K-X9732C-EX
  Meraki Notification MS350 Series

Network engineer Tony Mattke in a blog post said rumors about ISR 4331 router problems and a possible recall have been circulating for several months.

"Many of us have received phone calls at this time from our account managers ... while some of us, myself included, have been left in the dark," he wrote. "This is troubling considering how many products are out in the field." ®

Updated to add

It looks like the dodgy electronic component at the heart of the matter is an Intel Atom C2000 system-on-chip Cisco uses in its gear. The processor can fail to produce a clock signal required to drive the whole device – see this rather worrying errata, specifically AVR54, on this Intel data sheet, dated January 2017:

Similar topics


Other stories you might like

  • Cisco warns of security holes in its security appliances
    Bugs potentially useful for rogue insiders, admin account hijackers

    Cisco has alerted customers to another four vulnerabilities in its products, including a high-severity flaw in its email and web security appliances. 

    The networking giant has issued a patch for that bug, tracked as CVE-2022-20664. The flaw is present in the web management interface of Cisco's Secure Email and Web Manager and Email Security Appliance in both the virtual and hardware appliances. Some earlier versions of both products, we note, have reached end of life, and so the manufacturer won't release fixes; it instead told customers to migrate to a newer version and dump the old.

    This bug received a 7.7 out of 10 CVSS severity score, and Cisco noted that its security team is not aware of any in-the-wild exploitation, so far. That said, given the speed of reverse engineering, that day is likely to come. 

    Continue reading
  • Cisco execs pledge simpler, more integrated networks
    Is this the end of Switchzilla's dashboard creep?

    Cisco Live In his first in-person Cisco Live keynote in two years, CEO Chuck Robbins didn't make any lofty claims about how AI is taking over the network or how the company's latest products would turn networking on its head. Instead, the presentation was all about working with customers to make their lives easier.

    "We need to simplify the things that we do with you. If I think back to eight or ten years ago, I think we've made progress, but we still have more to do," he said, promising to address customers' biggest complaints with the networking giant's various platforms.

    "Everything we find that is inhibiting your experience from being the best that it can be, we're going to tackle," he declared, appealing to customers to share their pain points at the show.

    Continue reading
  • Intel offers 'server on a card' reference design for network security
    OEMs thrown a NetSec Accelerator that plugs into server PCIe slots

    RSA Conference Intel has released a reference design for a plug-in security card aimed at delivering improved network and security processing without requiring the additional rackspace a discrete appliance would need.

    The NetSec Accelerator Reference Design [PDF] is effectively a fully functional x86 compute node delivered as a PCIe card that can be fitted into an existing server. It combines an Intel Atom processor, Intel Ethernet E810 network interface, and up to 32GB of memory to offload network security functions.

    According to Intel, the new reference design is intended to enable a secure access service edge (SASE) model, a combination of software-defined security and wide-area network (WAN) functions implemented as a cloud-native service.

    Continue reading
  • Microsoft fixes under-attack Windows zero-day Follina
    Plus: Intel, AMD react to Hertzbleed data-leaking holes in CPUs

    Patch Tuesday Microsoft claims to have finally fixed the Follina zero-day flaw in Windows as part of its June Patch Tuesday batch, which included security updates to address 55 vulnerabilities.

    Follina, eventually acknowledged by Redmond in a security advisory last month, is the most significant of the bunch as it has already been exploited in the wild.

    Criminals and snoops can abuse the remote code execution (RCE) bug, tracked as CVE-2022-30190, by crafting a file, such as a Word document, so that when opened it calls out to the Microsoft Windows Support Diagnostic Tool, which is then exploited to run malicious code, such spyware and ransomware. Disabling macros in, say, Word won't stop this from happening.

    Continue reading
  • Azure issues not adequately fixed for months, complain bug hunters
    Redmond kicks off Patch Tuesday with a months-old flaw fix

    Updated Two security vendors – Orca Security and Tenable – have accused Microsoft of unnecessarily putting customers' data and cloud environments at risk by taking far too long to fix critical vulnerabilities in Azure.

    In a blog published today, Orca Security researcher Tzah Pahima claimed it took Microsoft several months to fully resolve a security flaw in Azure's Synapse Analytics that he discovered in January. 

    And in a separate blog published on Monday, Tenable CEO Amit Yoran called out Redmond for its lack of response to – and transparency around – two other vulnerabilities that could be exploited by anyone using Azure Synapse. 

    Continue reading
  • CISA and friends raise alarm on critical flaws in industrial equipment, infrastructure
    Nearly 60 holes found affecting 'more than 30,000' machines worldwide

    Updated Fifty-six vulnerabilities – some deemed critical – have been found in industrial operational technology (OT) systems from ten global manufacturers including Honeywell, Ericsson, Motorola, and Siemens, putting more than 30,000 devices worldwide at risk, according to private security researchers. 

    Some of these vulnerabilities received CVSS severity scores as high as 9.8 out of 10. That is particularly bad, considering these devices are used in critical infrastructure across the oil and gas, chemical, nuclear, power generation and distribution, manufacturing, water treatment and distribution, mining and building and automation industries. 

    The most serious security flaws include remote code execution (RCE) and firmware vulnerabilities. If exploited, these holes could potentially allow miscreants to shut down electrical and water systems, disrupt the food supply, change the ratio of ingredients to result in toxic mixtures, and … OK, you get the idea.

    Continue reading
  • This startup says it can glue all your networks together in the cloud
    Or some approximation of that

    Multi-cloud networking startup Alkira has decided it wants to be a network-as-a-service (NaaS) provider with the launch of its cloud area networking platform this week.

    The upstart, founded in 2018, claims this platform lets customers automatically stitch together multiple on-prem datacenters, branches, and cloud workloads at the press of a button.

    The subscription is the latest evolution of Alkira’s multi-cloud platform introduced back in 2020. The service integrates with all major public cloud providers – Amazon Web Services, Google Cloud, Microsoft Azure, and Oracle Cloud – and automates the provisioning and management of their network services.

    Continue reading
  • Inside the RSAC expo: Buzzword bingo and the bear in the room
    We mingle with the vendors so you don't have to

    RSA Conference Your humble vulture never liked conference expos – even before finding myself on the show floor during a global pandemic. Expo halls are a necessary evil that are predominatly visited to find gifts to bring home to the kids. 

    Do organizations really choose security vendors based on a booth? The whole expo hall idea seems like an outdated business model – for the vendors, anyway. Although the same argument could be made for conferences in general.

    For the most part, all of the executives and security researchers set up shop offsite – either in swanky hotels and shared office space (for the big-wigs) or at charming outdoor chess tables in Yerba Buena Gardens. Many of them said they avoided the expo altogether.

    Continue reading
  • Info on 1.5m people stolen from US bank in cyberattack
    Time to rethink that cybersecurity strategy?

    A US bank has said at least the names and social security numbers of more than 1.5 million of its customers were stolen from its computers in December.

    In a statement to the office of Maine's Attorney General this month, Flagstar Bank said it was compromised between December and April 2021. The organization's sysadmins, however, said they hadn't fully figured out whose data had been stolen, and what had been taken, until now. On June 2, they concluded criminals "accessed and/or acquired" files containing personal information on 1,547,169 people.

    "Flagstar experienced a cyber incident that involved unauthorized access to our network," the bank said in a statement emailed to The Register.

    Continue reading
  • 1Password's Insights tool to help admins monitor users' security practices
    Find the clown who chose 'password' as a password and make things right

    1Password, the Toronto-based maker of the identically named password manager, is adding a security analysis and advice tool called Insights from 1Password to its business-oriented product.

    Available to 1Password Business customers, Insights takes the form of a menu addition to the right-hand column of the application window. Clicking on the "Insights" option presents a dashboard for checking on data breaches, password health, and team usage of 1Password throughout an organization.

    "We designed Insights from 1Password to give IT and security admins broader visibility into potential security risks so businesses improve their understanding of the threats posed by employee behavior, and have clear steps to mitigate those issues," said Jeff Shiner, CEO of 1Password, in a statement.

    Continue reading

Biting the hand that feeds IT © 1998–2022