The factory floor of a modern paper manufacturing plant is a ballet of heavy machinery and razor-sharp blades, pressing, dying, rolling, unrolling and cutting dead tree pulp by the ton. To James Cupps, it's something else, too: a target rich environment for cyber attacks.
Cupps came to this perspective about three years ago, when, as newly-appointed information security officer for a large U.S. paper manufacturer, he got a phone call from an engineer posing a theoretical, but troubling, question. "He was worried about whether somebody from another site could control his equipment remotely," says Cupps. "And I looked into it, and, sure enough, they could."
At issue were the Programmable Logic Controllers that served as the electronic brains of each major piece of plant equipment. PLCs are microprocessor-based systems programmed to make the timing and control decisions in machine automation that once required arrays of electromechanical relays. They're essentially discrete computers wired into the machinery, monitoring and controlling functions like the speed of a motor or the movement of a conveyer belt.
Those PLCs are in turn manipulated remotely from a plant's control room. On older systems, PLCs communicated over RS-232 serial lines -- slow going, but relatively secure. But modern PLCs can plug right into a plant's Ethernet, exposing them to whatever threats lurk therein.
Coming from an IT environment, Cupps hoped to find that the control systems at his company's plants were protected by at least as much security as a Windows desktop. But when he set up a sniffer and monitored the traffic between a remote control program and one of the PLCs, he was dismayed to witness the program handshaking with the device by sending it a single UDP packet, with six plaintext ASCII characters as the data field. That's how Cupps learned that the secret password to take control over much of the hardware on the factory's assembly line was a hardcoded "hihihi."
"Script Kiddy Material"
"We talked to the vendor after this, and they talked to us a bit and they gave us recommendations," says Cupps. "But what it comes down to is they don't have any authentication mechanisms built into their tool, and until they do it's not going to be fixed."
The controls systems at Cupps' company are made by Rockwell Automation, but Cupps hastens to point out that the absence of authentication on PLCs is an industrywide problem, and not at all limited to one particular vendor. Other experts agree, and say the root cause is historical: the control systems rely on protocols and industry standards that were built for dedicated serial lines - not shared TCP/IP networks. "It's script kiddy material to control PLCs," says Eric Byres, a researcher and critical infrastructure security specialist at the British Columbia Institute of Technology (BCIT). "When the protocols were designed it wasn't Ethernet, it was a closed system. Then when the Ethernet was added the protocols remained the same."
The implications are disturbing to Byres and Cupps; in factories across the globe PLCs control pumps, conveyer belts, paint sprayer booths, welding machines, motors and other equipment. Neither expert envisions hacked robotic welding arms turning on their human masters, but the costs of an attack that shuts down an assembly line can be significant. "For most companies, if you interrupt production for even ten minutes, you're talking about tens of thousands or even hundreds of thousands of dollars," says Cupps.
"We found numerous ways to perform single-packet denial of service attacks against PLCs," says Byres. "You send one packet and this box isn't going to be working for a while."
On Wednesday, BCIT put some numbers to the problem. A report released in conjunction with the UK-based PA Consulting Group counts a tenfold increase in the number of successful cyber attacks on control systems since 2000. The study is based on an analysis of entries in BCIT's Industrial Security Incident Database, a decades-old voluntary industry information-sharing program.
That attack spike isn't as ominous as it sounds; since its launch in 1981, the BCIT database has logged a total of only 34 confirmed incidents. But Byres believes that's the tip of the iceberg -- that for every attack reported another 10 to 100 are kept secret by the victim.
Moreover, Byres says the most significant finding in the report is that the source of attacks has shifted. The 13 cyber security incidents logged between the years 1982 and 2000 were almost all attributable to accidents, inappropriate employee behaviour, or sabotage by disgruntled employees. In contrast, 14 of the 20 incidents reported from 2001 through 2003 were from external sources, like the Internet. "There was always an assumption that your biggest threat was coming from the inside," says Byres. "That's now incorrect. Your bigger threat is coming from the outside, and that surprised me."
Processer Power Issues
In a lot of those external attacks, control systems were merely collateral damage from IT issues like worms, "because we have Windows running all over the plant floor," says Byres. So far, directed attacks against PLCs are virtually unheard of. "I don't think the hacker community has totally woken up to the opportunity, fortunately," Byres says. "I think we've got a bit of a jump on them."
There's no telling how long that will hold, though, and a number of industry, governmental and public initiatives are trying to close the vulnerabilities before serious attacks take place. Efforts range from a US Department of Commerce plan to develop security standards for control systems, to an open-source firewall project designed to protect PLCs that speak Modbus/TCP, the networked update to the industry standard MODBUS protocol, which lacks authentication.
Michael Bush, security program manager at Rockwell Automation, acknowledges that Ethernet-enabled control systems "change the rules significantly" from the days of dedicated serial lines. But he says that PLCs simply haven't had the processing power to handle encryption and authentication protocols. "A typical plant floor device has significantly less processor bandwidth, horse power, speed and memory than a PC," Bush says. "A lot of things like the authentication protocols and the encryption protocols that are in PCs use enormous amounts of power."
Bush says that's just now changing with the industry's latest generation of controllers, and that authentication is on its way. "As devices on the plant floor start to have the processor capability to support these advanced protocols, we'll begin incorporating them," says Bush. "We're right on the cusp of that." But he cautions that PLCs can have a lifecycle as long as 20 or 30 years before plants replace them.
In the meantime, Rockwell advises customers on how to secure networks that run control systems, and publishes a detailed whitepaper on the topic. For his part, Cupps says he took emergency measure to shore up the control systems at his company, then committed to a massive reorganization of its networks, putting the factory floors on their own subnets, adding firewalls between them, and installing intrusion prevention systems, among other things. He estimates the effort took over two years and $1 million dollars to complete at the company's 15 factories around the world. And while he's confident that the measures are adequate, he'd still like the devices to speak a more secure language.
"The problem is the hard-and-crunchy on the outside and soft-and-chewy on the inside syndrome," Cupps says. "The reason you need an authentication mechanism is there are vulnerabilities that are unique to IP sessions, like source address spoofing... That's why it's important for these companies to take a look at this stuff and use some sort of asymmetric key to make sure the right machines are talking to the right machines."