Nvidia pulls sheets off EGX, an edgy machine learning platform based on Kubernetes

Built with help from Red Hat


All kinds of software is moving into lightweight application containers, and the fact hasn't been lost on Nvidia – the GPU supremo has launched a platform for machine learning at the edge of the network, based on Kubernetes.

EGX is compatible with all mainstream devices made by Nvidia, from the tiny Jetson Nano boards used for prototyping (shipping in June) to full racks of T4 GPUs typically found in cloud data centres.

It is meant to simplify the task of collecting information from the sensors and running it through machine learning algorithms to get a response in real time, without having to haul the data back to a corporate bit barn.

The platform was created with Red Hat and uses its OpenShift container orchestration software. It was unveiled at Computex in Taipei, the world's largest computer show.

EGX marries OpenShift with Nvidia's Edge Stack – which includes drivers, a CUDA container runtime, a CUDA Kubernetes plugin, CUDA-X libraries and a selection of containerized AI frameworks and applications, like TensorRT for TensorFlow inference, and DeepStream for computer vision.

Nvidia EGX architecture

Nvidia EGX architecture

All "cloud-native" applications – those packaged in app containers – can run on EGX, and any applications developed for EGX can run in any major public cloud.

The platform can be managed remotely using AWS IoT Greengrass and Microsoft Azure IoT Edge services.

It has also tested for compatibility with networking kit from Mellanox and Cisco, for customers who want to run EGX on-premises.

The platform is positioned for applications in industries including telecommunications, healthcare, manufacturing, retail and transportation. Early adopters of EGX include Foxconn, which uses machine learning to scan for factory defects on PC production lines; GE Healthcare, which uses algorithms to speed up the MRI process; and Seagate, which has deployed ML to inspect the quality of hard disk read-and-write heads.

"The Nvidia EGX platform dramatically accelerates inference at the edge, allowing us to see subtle defects that human operators haven't been able to see in the past," claimed Bruce King, senior principal data scientist at Seagate. "We expect to realise up to a 10 percent improvement in manufacturing throughput and up to 300 percent ROI from improved efficiency and better quality." ®

Broader topics


Other stories you might like

  • Nvidia wants to lure you to the Arm side with fresh server bait
    GPU giant promises big advancements with Arm-based Grace CPU, says the software is ready

    Interview 2023 is shaping up to become a big year for Arm-based server chips, and a significant part of this drive will come from Nvidia, which appears steadfast in its belief in the future of Arm, even if it can't own the company.

    Several system vendors are expected to push out servers next year that will use Nvidia's new Arm-based chips. These consist of the Grace Superchip, which combines two of Nvidia's Grace CPUs, and the Grace-Hopper Superchip, which brings together one Grace CPU with one Hopper GPU.

    The vendors lining up servers include American companies like Dell Technologies, HPE and Supermicro, as well Lenovo in Hong Kong, Inspur in China, plus ASUS, Foxconn, Gigabyte, and Wiwynn in Taiwan are also on board. The servers will target application areas where high performance is key: AI training and inference, high-performance computing, digital twins, and cloud gaming and graphics.

    Continue reading
  • GPUs aren’t always your best bet, Twitter ML tests suggest
    Graphcore processor outperforms Nvidia rival in team's experiments

    GPUs are a powerful tool for machine-learning workloads, though they’re not necessarily the right tool for every AI job, according to Michael Bronstein, Twitter’s head of graph learning research.

    His team recently showed Graphcore’s AI hardware offered an “order of magnitude speedup when comparing a single IPU processor to an Nvidia A100 GPU,” in temporal graph network (TGN) models.

    “The choice of hardware for implementing Graph ML models is a crucial, yet often overlooked problem,” reads a joint article penned by Bronstein with Emanuele Rossi, an ML researcher at Twitter, and Daniel Justus, a researcher at Graphcore.

    Continue reading
  • Intel is running rings around AMD and Arm at the edge
    What will it take to loosen the x86 giant's edge stranglehold?

    Analysis Supermicro launched a wave of edge appliances using Intel's newly refreshed Xeon-D processors last week. The launch itself was nothing to write home about, but a thought occurred: with all the hype surrounding the outer reaches of computing that we call the edge, you'd think there would be more competition from chipmakers in this arena.

    So where are all the AMD and Arm-based edge appliances?

    A glance through the catalogs of the major OEMs – Dell, HPE, Lenovo, Inspur, Supermicro – returned plenty of results for AMD servers, but few, if any, validated for edge deployments. In fact, Supermicro was the only one of the five vendors that even offered an AMD-based edge appliance – which used an ageing Epyc processor. Hardly a great showing from AMD. Meanwhile, just one appliance from Inspur used an Arm-based chip from Nvidia.

    Continue reading
  • Cerebras sets record for 'largest AI model' on a single chip
    Plus: Yandex releases 100-billion-parameter language model for free, and more

    In brief US hardware startup Cerebras claims to have trained the largest AI model on a single device powered by the world's largest Wafer Scale Engine 2 chip the size of a plate.

    "Using the Cerebras Software Platform (CSoft), our customers can easily train state-of-the-art GPT language models (such as GPT-3 and GPT-J) with up to 20 billion parameters on a single CS-2 system," the company claimed this week. "Running on a single CS-2, these models take minutes to set up and users can quickly move between models with just a few keystrokes."

    The CS-2 packs a whopping 850,000 cores, and has 40GB of on-chip memory capable of reaching 20 PB/sec memory bandwidth. The specs on other types of AI accelerators and GPUs pale in comparison, meaning machine learning engineers have to train huge AI models with billions of parameters across more servers.

    Continue reading
  • Nvidia taps Intel’s Sapphire Rapids CPU for Hopper-powered DGX H100
    A win against AMD as a much bigger war over AI compute plays out

    Nvidia has chosen Intel's next-generation Xeon Scalable processor, known as Sapphire Rapids, to go inside its upcoming DGX H100 AI system to showcase its flagship H100 GPU.

    Jensen Huang, co-founder and CEO of Nvidia, confirmed the CPU choice during a fireside chat Tuesday at the BofA Securities 2022 Global Technology Conference. Nvidia positions the DGX family as the premier vehicle for its datacenter GPUs, pre-loading the machines with its software and optimizing them to provide the fastest AI performance as individual systems or in large supercomputer clusters.

    Huang's confirmation answers a question we and other observers have had about which next-generation x86 server CPU the new DGX system would use since it was announced in March.

    Continue reading
  • Lenovo reveals small but mighty desktop workstation
    ThinkStation P360 Ultra packs latest Intel Core processor, Nvidia RTX A5000 GPU, support for eight monitors

    Lenovo has unveiled a small desktop workstation in a new physical format that's smaller than previous compact designs, but which it claims still has the type of performance professional users require.

    Available from the end of this month, the ThinkStation P360 Ultra comes in a chassis that is less than 4 liters in total volume, but packs in 12th Gen Intel Core processors – that's the latest Alder Lake generation with up to 16 cores, but not the Xeon chips that we would expect to see in a workstation – and an Nvidia RTX A5000 GPU.

    Other specifications include up to 128GB of DDR5 memory, two PCIe 4.0 slots, up to 8TB of storage using plug-in M.2 cards, plus dual Ethernet and Thunderbolt 4 ports, and support for up to eight displays, the latter of which will please many professional users. Pricing is expected to start at $1,299 in the US.

    Continue reading
  • Will optics ever replace copper interconnects? We asked this silicon photonics startup
    Star Trek's glowing circuit boards may not be so crazy

    Science fiction is littered with fantastic visions of computing. One of the more pervasive is the idea that one day computers will run on light. After all, what’s faster than the speed of light?

    But it turns out Star Trek’s glowing circuit boards might be closer to reality than you think, Ayar Labs CTO Mark Wade tells The Register. While fiber optic communications have been around for half a century, we’ve only recently started applying the technology at the board level. Despite this, Wade expects, within the next decade, optical waveguides will begin supplanting the copper traces on PCBs as shipments of optical I/O products take off.

    Driving this transition are a number of factors and emerging technologies that demand ever-higher bandwidths across longer distances without sacrificing on latency or power.

    Continue reading
  • Cloudflare's outage was human error. There's a way to make tech divinely forgive
    Don't push me 'cos I'm close to the edge. And the edge is safer if you can take a step back

    Opinion Edge is terribly trendy. Move cloudy workloads as close to the user as possible, the thinking goes, and latency goes down, as do core network and data center pressures. It's true  – until the routing sleight-of-hand breaks that diverts user requests from the site they think they're getting to the copies in the edge server. 

    If that happens, everything goes dark – as it did last week at Cloudflare, edge lords of large chunks of web content. It deployed a Border Gateway Protocol policy update, which promptly took against a new fancy-pants matrix routing system designed to improve reliability. Yeah. They know. 

    It took some time to fix, too, because in the words of those in the know, engineers "walked over each other's changes" as fresh frantic patches overwrote slightly staler frantic patches, taking out the good they'd done. You'd have thought Cloudflare of all people would be able to handle concepts of dirty data and cache consistency, but hey. They know that too. 

    Continue reading
  • AWS sent edgy appliance to the ISS and it worked – just like all the other computers up there
    Congrats, AWS, you’ve boldly gone where the Raspberry Pi has already been

    Amazon Web Services has proudly revealed that the first completely private expedition to the International Space Station carried one of its Snowcone storage appliances, and that the device worked as advertised.

    The Snowcone is a rugged shoebox-sized unit packed full of disk drives – specifically 14 terabytes of solid-state disk – a pair of VCPUs and 4GB of RAM. The latter two components mean the Snowcone can run either EC2 instances or apps written with AWS’s Greengrass IoT product. In either case, the idea is that you take a Snowcone into out-of-the-way places where connectivity is limited, collect data in situ and do some pre-processing on location. Once you return to a location where bandwidth is plentiful, it's assumed you'll upload the contents of a Snowcone into AWS and do real work on it there.

    Continue reading
  • Train once, run anywhere, almost: Qualcomm's drive to bring AI to its phone, PC chips
    Software toolkit offered to save developers time, effort, battery power

    Qualcomm knows that if it wants developers to build and optimize AI applications across its portfolio of silicon, the Snapdragon giant needs to make the experience simpler and, ideally, better than what its rivals have been cooking up in the software stack department.

    That's why on Wednesday the fabless chip designer introduced what it's calling the Qualcomm AI Stack, which aims to, among other things, let developers take AI models they've developed for one device type, let's say smartphones, and easily adapt them for another, like PCs. This stack is only for devices powered by Qualcomm's system-on-chips, be they in laptops, cellphones, car entertainment, or something else.

    While Qualcomm is best known for its mobile Arm-based Snapdragon chips that power many Android phones, the chip house is hoping to grow into other markets, such as personal computers, the Internet of Things, and automotive. This expansion means Qualcomm is competing with the likes of Apple, Intel, Nvidia, AMD, and others, on a much larger battlefield.

    Continue reading

Biting the hand that feeds IT © 1998–2022