You've patched your cloud software, and feeling smug when... bang! A legacy edge device bricks itself

Don't want to be an enterprise id-IoT? Time to look at process

Backgrounder Sitting on many enterprise networks is a constellation of equipment. A lot will be legacy network-connected industrial Internet of Things (IIoT) gear, together with some relatively new IIoT gear installed within the last few years as a move to what has been sweepingly called the "digitalisation" of business.

As the IoT network trundles away, your organisation will likely have moved into other areas: adopting new, flexible approaches to delivering technology and infrastructure such as cloud, agile development and DevOps, for example.

How and why should you reconcile these two movements? How do you bring those new tools and techniques for the way software is built and delivered to a world of sensors, scanners, devices, cameras and other devices? And, moreover, how can you do this when you're the proud owner of legacy?

Cloud, cloud – don't go away

Cloud is increasingly providing core enterprise technology infrastructure in compute and storage.

And cloud is proving particularly useful as a backend for IIoT, with devices already some way towards being cloud-ready. The 2018 SANS Industrial IoT Security survey (PDF) found almost a third (32 per cent) of IIoT devices connect directly to the internet while AWS IoT and Azure IoT have been developed to offer access and management for these devices.

"The cloud guys are driving a faster pace of innovation," explains Matthew Short, senior IoT research director at IHS Markit. "People want to be able to use that over time, and so they're shifting to that type of more nimble deployment."

There are big benefits to be gained in connecting IIoT devices to cloud infrastructures and enabling them to be updated quickly. Stephen Mellor, Industrial Internet Consortium CTO, reckons that as devices get smarter, the need to update them with new capabilities in software will increase.

He points to Tesla's over-the-air (OTA) 2017 upgrade to its vehicles in September 2017. The IoT tinkering enabled the company to extend the vehicles' battery life. Porsche and Audi, meanwhile, both have connected car partnerships with US carrier AT&T.

"A particular area of great interest is integrated transportation systems, and particularly automobiles," he says. "And anything that provides provisioning, such as electricity meters. DevOps technology is pretty clearly required in this world."

Enter DevOps

DevOps has for some time been gaining currency as a way to supercharge software build. The DevOps process is structured in terms of integrated and continuous lifecycle as opposed to the old method of big releases, with dev, test and run-time teams working behind separate walls. It uses software toolsets to create an automated pipeline from design through to deployment and beyond to operation.

Some might stop DevOps at continuous integration, which uses a series of software "gates" to automate the quality control and testing of software and its merging into a master branch, but others automate deployment too – with software that passes all the tests automatically pushed for release using tools to provision and configure infrastructure. This is the "Ops" part, and can include monitoring and feedback of operational software to inform more development choices.

DevOps, it's fair to say, is revolutionising enterprise apps – a field historically notorious for software projects going late and not delivering as promised. But enterprise apps have historically operated in stable, predictable and somewhat well provisioned environments. IIoT can differ on all of these, meaning that when it comes to IIoT, DevOps can be more problematic.

One problem is the device footprint. Industrial IoT devices generally have small amounts of memory and computing power. This places extra constraints on development teams, who have to be extra vigilant with their code. "Even an expectation of an HTTPS connection, or one secured with TLS, becomes a challenge when the code to run such a protocol barely fits within the available RAM of the processor," says Tim Mackey, Synopsys senior technical evangelist.

Another issue is safety and reliability, especially with industrial IoT devices in environments. With these devices often controlling production lines that can cost thousands of pounds each hour if downed, companies can't afford to make any mistakes when updating devices, explains Srinivas Rao Bhagavatula, Mindtree IoT practice lead.

For this reason, he sees the need for three layers of testing in a DevOps environment before new code hits IIoT devices. First comes unit testing, where the code is tested for errors in the IDE. Then you have simulator testing, where the code is tested in a simulated environment. These two stages can be automated – assuming you have DevOps tooling in place that can be controlled by a command line interface and therefore triggered as part of an automated process.

The final stage is deployment to a production line, where the code is used in anger. To be safe, he advises "single-line deployment" in a factory environment where the code is implemented on one of several production lines to check everything works properly in a real-world environment.

Network worries

It isn't just coding errors that can cause problems in IIoT, though. Developers must anticipate unreliable links when deploying. If the wireless networks for OTA updates - especially those in large-scale deployments like smart grids, say - are themselves unreliable, then two things can happen.

One: the device could simply stop working. "It's easy to patch software in the cloud because it's always connected, but you need to have special consideration for devices that sit on the edge," says Amido principal consultant and DevOps/SRE leader Richard Slater. "The update cannot go wrong or we risk the device becoming a glorified brick."

Secondly, not all devices may be able to update at once, meaning the DevOps team will have different versions of code running in its IoT infrastructure – at least in the short term.

One way to mitigate all these potential problems with devices in a DevOps environment is to use an intermediary device.

"If a lot of intelligence is placed on nodes as code, then the likelihood of them needing updates is a lot higher," explains Bhagavatula. "But if that intelligence can be externalized onto commands and parameters, or moved to gateways, then it is simply a matter of pushing the right commands to the right gateways."

You need to update fewer devices, which can have more computing and storage power and can coordinate IIoT devices themselves.

Security

The other challenge for IIoT deployment teams is security. Many deployments mean dealing with far larger networks of devices than ever before, constituting hundreds of thousands or even millions of devices, and they are very hackable.

In many cases, hacked workplace IoT systems are used as jumping off points to compromise other systems. For example, hackers apparently infiltrated a casino through the IP-connected thermometer in its aquarium and jumped from there to its high-roller database.

No wonder, then, that well over half (59 per cent) of respondents to an IoT survey from wireless IoT consortium Wi-SUN cited security concerns as a challenge. In the SANS survey, 14.1 per cent (the largest number) considered the integrity of embedded software or firmware to be the biggest danger.

DevOps must deal with this on a process basis. The problem is that the power of the process can exacerbate security issues, warns WhiteHat chief scientist Eric Sheridan. "Security teams, understandably so, are generally behind software development trends," he says. "This means that legacy security testing methodologies and processes are ill equipped to support the secure development of software in DevOps."

This doesn't mean DevOps can or will weaken security. By building automated security tools and testing directly into the DevOps process (a process that advocates are calling DevSecOps), teams can bake security protections directly into their development.

Baby steps to IoT success

Perhaps the biggest benefit of DevOps in an IIoT environment is the ability it provides to develop in incremental phases and support business ideas as they develop, with new services delivered via software over the air or down the network as and when – so there's no big-bang release.

That's a blessing in IIoT, which, according to Mendix vice president of platform strategy Gordon Van Huizen, has one of the fastest rates of change in the industry. You must, therefore, find business-use cases that deliver real value, knowing these may adapt over time as business needs change.

As a cautionary note, a survey of more than 1,800 IT pros and business decision makers by Cisco found that only 26 per cent of IoT projects were deemed a success. Among other factors, it blamed integration, budget overruns and time to completion. "All this points to a heightened need to build in small increments, test, learn and adapt, which is the fundamental reasons to adopt agile," he says. "And DevOps provides the mindset and practices necessary to effectively support agile delivery."

With this in mind, DevOps is perhaps the most prudent way to digitalise with IIoT.

The key comes in starting small – that may be identifying industrial scenarios that benefit from automation of physical processes or finding ways to monitor or deliver new services through the combination of software and device. If your IoT device can stop an expensive piece of equipment from failing, if you can update device firmware on the shop floor, or enhance the battery efficiency of your fleet of cars using new code over the air then these are clear wins.

DevOps means you can bring not just an incremental approach to the process of starting to digitalise legacy, but also you can support the software, and thus your devices, once things grow. ®