Samsung, others test drive Esperanto's 1,000-core RISC-V AI chip

Now you're talking our language

Samsung's IT services arm and other companies are said to be testing out a processor that sports more than 1,000 general-purpose RISC-V cores to deliver what the chip's designer claims is faster and more energy-efficient AI inference performance than power-hungry specialty silicon.

The chip designer, Esperanto Technologies, said Thursday Samsung SDS and other unnamed companies, which it only identified as "lead customers," are doing initial evaluations of the startup's ET-SoC-1 AI inference accelerator.

The Mountain View, California-based startup was founded in 2014 by Dave Ditzel, a semiconductor industry veteran who worked on parallel computing architectures at Intel and, earlier in his career, led the development of the SPARC CPU instruction set architecture at Sun Microsystems. Esperanto is now led by Art Swift, who has held several semiconductor leadership roles, including CEO of Wave Computing.

The ET-SoC-1 packs 1,088 energy-efficient, 64-bit processor cores that use the RISC-V instruction set architecture, an emerging alternative to the x86 and Arm ISAs. These cores are accompanied by their own vector/tensor math units for acceleration machine-learning operations, and the chip also comes with four high-performance RISC-V cores, plus more than 160 million bytes of on-chip SRAM (more than 152MB, then) and interfaces for flash memory and external DRAM.

With this architecture, Esperanto claims it has the fastest AI RISC-V chip. Combined with its low-power requirements, the startup said the ET-SoC-1 was "designed to meet the performance, power and total cost of ownership requirements of large-scale data center customers."

While Esperanto claims the ET-SoC-1 can "run any machine learning workload well," the company said the chip excels at ML recommendation, one of the most common server applications run by so-called hyperscalers like Facebook parent company Meta and Amazon.

Plaudits from Samsung SDS

Patrick Bangert, vice president of AI at Samsung SDS, said his data science team "was very impressed" with the company's first test of the ET-SoC-1. He added that processors from Esperanto's competitors do not offer the same level of performance scaling as the ET-SoC-1, which means that the chip does a good job at providing a higher level of performance in proportion to the amount of chips used.

"It was fast, performant and overall easy to use. In addition, the SoC demonstrated near-linear performance scaling across different configurations of AI compute clusters. This is a capability that is quite unique, and one we have yet to see consistently delivered by established companies offering alternative solutions to Esperanto," Bangert said in a quote provided by Esperanto.

This is quite the testimony from a major IT provider, which sells a variety of AI and data analytics services through its Brightics AI software platform.

But it's one thing to sample a new chip. It's another thing entirely for a large company to make a large-volume purchase order and rely on such chips to power revenue-generating products and services. Thus, Esperanto's sampling news is but one of many milestones it will need to achieve before causing concern for the dominant provider of AI chips, Nvidia, and other chip designers.

As such, Esperanto said it's looking for other companies that want to join its evaluation program, which allows users to rest the ET-SoC-1 on a variety of off-the-shelf AI models, including recommendation, transformer and visual network models. The program lets users test various models, data types, batch sizes and compute configurations of up to 32 clusters.

Good software support, but is it too late?

Karl Freund, principal analyst at Cambrian-AI Research, said Esperanto demonstrated "rock solid" performance for ET-SoC-1 with the ResNet 50, DLRM and Transformer models, though he can't share the results yet. He added that he expects the chip to only require 20 watts to run at full power.

Freund said he was initially skeptical that the ET-SoC-1 could provide a high level of inference performance using general-purpose RISC-V cores, but the results proved him wrong. "What really makes this approach unique, is that the RISC-V cores are actually doing the heavy lifting, not offloading the matrix multiplies to a MAC core or a GPU," he said.

Just as important, Freund said, Esperanto "has the programming tools and software stack to more easily adapt to new AI workloads, alongside non-AI workloads, all running on the same silicon."

However, Freund admitted that Esperanto's chip has arrived a "bit late," so there is a question of whether the startup can keep up with other companies working on low-power chips for inference. ®

Other stories you might like

  • Samsung invests in ML chip startup NeuReality
    Coining the term hardware-based 'AI hypervisor' has to be worth several million, dontcha think?

    The venture capital arm of Samsung has cut a check to help Israeli inference chip designer NeuReality bring its silicon dreams a step closer to reality.

    NeuReality announced Monday it has raised an undisclosed amount of funding from Samsung Ventures, adding to the $8 million in seed funding it secured last year to help it get started.

    As The Next Platform wrote in 2021, NeuReality is hoping to stand out with an ambitious system-on-chip design that uses what the upstart refers to as a hardware-based "AI hypervisor."

    Continue reading
  • AMD touts big datacenter, AI ambitions in CPU-GPU roadmap
    Epyc future ahead, along with Instinct, Ryzen, Radeon and custom chip push

    After taking serious CPU market share from Intel over the last few years, AMD has revealed larger ambitions in AI, datacenters and other areas with an expanded roadmap of CPUs, GPUs and other kinds of chips for the near future.

    These ambitions were laid out at AMD's Financial Analyst Day 2022 event on Thursday, where it signaled intentions to become a tougher competitor for Intel, Nvidia and other chip companies with a renewed focus on building better and faster chips for servers and other devices, becoming a bigger player in AI, enabling applications with improved software, and making more custom silicon.  

    "These are where we think we can win in terms of differentiation," AMD CEO Lisa Su said in opening remarks at the event. "It's about compute technology leadership. It's about expanding datacenter leadership. It's about expanding our AI footprint. It's expanding our software capability. And then it's really bringing together a broader custom solutions effort because we think this is a growth area going forward."

    Continue reading
  • RISC-V International emits more open CPU specs
    First edicts of 2022 include firmware, hypervisor-level specifications

    Embedded World RISC-V International has grown its pile of royalty-free, open specifications, with additional documents covering firmware, hypervisors, and more.

    RISC-V – pronounced "risk five", and not to be confused with the other architecture of that name, RISC-5 – essentially sets out how a CPU core should work from a software point of view. Chip designers can implement these instruction set specifications in silicon, and there are a good number of big industry players backing it.

    The latest specs lay out four features that compatible processors should adhere to. Two of them, E-Trace and Zmmul, will be useful for organizations building RISC-V hardware and software, and the other two could prove important in future, aiding the development of OSes to run on RISC-V computers.

    Continue reading
  • Amazon can't channel the dead, but its deepfake voices take a close second
    Megacorp shows Alexa speaking like kid's deceased grandma

    In the latest episode of Black Mirror, a vast megacorp sells AI software that learns to mimic the voice of a deceased woman whose husband sits weeping over a smart speaker, listening to her dulcet tones.

    Only joking – it's Amazon, and this is real life. The experimental feature of the company's virtual assistant, Alexa, was announced at an Amazon conference in Las Vegas on Wednesday.

    Rohit Prasad, head scientist for Alexa AI, described the tech as a means to build trust between human and machine, enabling Alexa to "make the memories last" when "so many of us have lost someone we love" during the pandemic.

    Continue reading

Biting the hand that feeds IT © 1998–2022