Today’s hardware and software choices will define your AI project’s success

Machine-learning deployment without the fuss

Sponsored Everyone seems alive to the potential of Artificial Intelligence in business and the public sector. According to research from PwC, worldwide we can expect to see a boost to economies of $15.7tn by 2030 as more organisations unlock new opportunities in big data and advanced analytics in such fields as financial services, retail, transport and government.

The sweet spot of AI is the potential scale of data processing at a volume beyond human capabilities – combined with the capacity for systems to learn and develop unprompted responses.

But while PwC’s numbers are impressive – and explain why so many are keen to start big-data and AI initiatives – it’s important to remember we’re still in a very early stage of deployment. “We have to think that AI is still a teenager,” says Walter Riviera, EMEA AI Technical Engineer at Intel®.

As it is a fast growing field, organisations have time to properly plan their AI deployment – but how? “The first thing that needs to be defined when talking about deployment is what you’re hoping to achieve: what are the success metrics?” Riviera says.

Every AI project brings challenges at different levels: planning, development and deployment. At each level, move the focus on different aspects to take into account, Riveria said. “For example, while developing the solution, priorities could include the level of accuracy we might want to achieve for solving a given task, while in the deployment phase, we might want to understand the latency requirements (i.e. time to response) of the final product. For example, in an airport when checking in baggage, you’re operating on low latency and want an answer as soon as possible. But when it comes to some banking applications, processing all the transactions of a day can happen overnight.”

Or is there is a need for a low-powered response that could sit at the edge? For example, Riviera cites the case of a large city car park that uses AI to spot empty spaces before allowing cars in. In that case the decision can be taken at the edge with a smart camera. This is a very different set-up from a banking application, for instance, that needs to leverage the data centre infrastructure. These examples showcase just a couple of the many questions that could rise when deploying AI solutions.

This is why it is important that organisations understand their requirements and challenges because these will define their AI strategies and are directly related to the outcomes. We have already seen how to prepare your data ready for AI. The extract, transform, load element is an important part of the ramp up to AI, as it sees you prepare the data that your machine models will feed on to learn – but what’s next? Where should companies concentrate their attention to ensure a smooth rollout and to hit their desired outcomes?

It is here that the question of platform and tools becomes important – the hardware and software choices you make now will define your project’s success. These choices will impact ease of development, runtime performance, return on investment and cost effectiveness, scale-out and support. Moreover, depending on the application, the choice you make now will be generational – you will be committed for the long term, so it is important to make the best choice.

Infrastructure support

“This is where the wide Intel® portfolio of hardware and software solutions currently available, can help customers. From Datacentre, where the Intel® Xeon® Scalable powered infrastructures can be easily turned into AI capable facilities, to the edge where the deployment can be simplified by the adoption of opensource software tools such as OpenVINO™, Intel® has something to offer to support the customers in any step of an AI journey” says Riviera.

To this end, the company has introduced several enhancements to its Intel® Xeon® Scalable Processors, thereby allowing businesses to take advantage of an architecture they will already either have or be familiar with. “Intel® have made concrete efforts to improve and make the Intel® Xeon® Scalable processors more agile to ensure that they can host more AI tasks (such as the Intel® Deep Learning Boost), while preserving the strong ability to manage traditional enterprise workloads, like High Performance Computing for example.”

Riviera says the same efforts are also visible if we look at the software stack: “Companies already using Apache Spark to store and access their historical data collection on Hadoop, can equip their systems with BigDL and turn their infrastructure into an AI capable facility. If customers are already working with Intel® Xeon® Scalable family, they can adopt Intel®’s optimised versions of Tensorflow, Pytorch and many other frameworks to speed-up their computations. The same considerations apply to the Intel® Distribution for Python. It’s important that Customers and Users around the globe can work with what they’re used to.”

“They should be looking to use BigDL, which is free, and integrate that,” he adds.

Given the proposed changes, I think this would flow smoothly if we start from Riviera is, of course, referring to one of the newest initiatives from the company: BigDL is a distributed deep learning library for Apache Spark that was developed by Intel®. BigDL supports several deep learning frameworks, including numeric computing and pre-trained neural networks (available on the Analytic Zoo). BigDL can be used in a variety of deployments. For example, organisations that want to perform detailed analytics on the same cluster where the data is stored will be looking to write deep learning programs using BigDL.

Intel®, of course, has a broader focus than BigDL. One of its most useful developments has been in VNNI (vector neural network instruction). VNNI uses a single instruction for accelerating Convolutional operations within Deep Learning computations. The combination of VNNI with Intel® DL Boost, integrated on the Xeon® Scalable chip, reduces compute and memory bandwidth requirements, delivering a massive improvement in performance.

Underlying all of these initiatives are the Intel® oneAPI tools and libraries. Intel® oneAPI is a unified programming model designed to let developers easily build and deploy data-centric workloads and applications on a variety of different architectures. It allows programmers to tackle software that spans all aspects of Intel’s technology range, whether those are CPU, GPU or FPGA. Intel® oneAPI eliminates the need to run separate code bases and handle multiple languages, thereby streamlining the development and management lifecycle process.

It is instructive to look at the Intel® Distribution of OpenVINO™ as an example of the way that some of Intel®’s more specialist technology has been deployed, OpenVINO™ simplifies the deployment process by speeding-up the time required to test different architectures. It accepts a trained model as input and through a ‘model optimizer’ that can help speeding-up the computation and ‘inference engine’, it allows the users to generate a binary file that can be hosted by a targeted inference device. OpenVINO is ideal, but not limited to, deployment of solutions at the edge. The tool is a complete suite, fully integrated with computer vision functions and pre-optimised kernels that can accelerate the development and deployment of application within the Vision environment.

Familiar path

AI is destined to change the business landscape but to reach the pay-off, you must make an informed platform choice. The hardware and software you pick will help determine the success of your AI. Intel® not only delivers a complete stack but, importantly, it’s an architecture that brings power with a low barrier to entry. All the software tools mentioned, are publicly available and free to download,” says Riviera.

Want to find out more about Intel®’s AI solutions? Please make your way Intel®’s artificial intelligence overview.

Sponsored by Intel®