DDN's cache wolves come howling out of the creek

IME14K has a client driver, and an out-of-band and open API. No word on price

WolfCreek, Data Direct Network’s burst buffer caching system, has arrived in the shape of the IME14K product – apparently ready to leap burning buildings, solve climate change, and make a $5m HPC cluster perform like a $100m one.

The idea here is that high-performance computing (HPC) is dealing with ever-bigger data sets, and the parallel file system repositories used can’t keep up with the compute clusters and their multiplying and hungry cores.

To get out of the IO-wait trap, solid state storage is needed plus software to put right parallel file system features that don’t suit HPC compute IO demands. Step forward DDN’s WolfCreek technology.

The Infinite Memory Engine (IME14K) has a client driver, a solid state cache, and an out-of-band and open API. It's "14K" suffix doesn't mean it's necessarily tied to DDN's SFA14K arrays.

"An IME14K creates a DDN pre-configured appliance leveraging the DDN 14K hardware and the IME software," said DDN chief marketing officer Molly Rector. "As IME pushes data from its solid state cache and into the parallel file system storage, that file system can be provided by DDN or an alternative provider."

DDN said the IME is, in principle, compute-, storage- and vendor-agnostic.

The background here is this:

"The DDN 14K platform is the name of the DDN custom-designed hardware (previously known as WolfCreek) upon which DDN is building several different appliances."

"For example, embedding the SFA operating system to create high-performance block storage which is the SFA14K, embedding IME software and loading with just SSD to make the IME14K appliance, or loading SFA operating systems and Lustre to make a Lustre appliance."

The IME14K burst buffer sits between a parallel file system (PFS) store and a compute cluster, providing read and write IO acceleration. It’s scalable to multiple TBs in size and protects its data with hash-based erasure coding.

The system buffers incoming writes and manages them, aligning fragmented data to striped writes, to accelerate PFS writes, and, DDN said, eliminates POSIX file locking penalties, with instant open/close.

DDN suggests that a compute cluster can get 25GB/sec bandwidth from a PFS storage system. With the IME14K interposed between the compute resource and the PFS that jumps to 50GB/sec ... yes, GB/sec not MB/sec. It claims the compute cluster could be operating at 6,250 IOPS without the IME and 12.5 million IOPS with the all-flash NVMe IME, a 2,000X acceleration.

If your HPC-style compute clusters are IO-bound then such claims need looking at.


DDN IME schematic diagram

The IME14K engine features:

  • 4U dual controller enclosure with 14-core, dual socket, Haswell Xeon CPUs
  • 48 x NVMe PCIE gen 3 4-lane 800GB or 1.6TB SSDs with performance increasing as the SSD number increases
  • 2546GB DRAM per controller

"[The] IME14K can connect to a cluster via EDR (100Gbit/s) or FDR (56Gbit/s) Infiniband today," said Rector. "It will support 40 and 100GbitE in Q1 as well as Omni-Path as soon as Omni-Path begins shipping." Ditto for connection to back end parallel file system stores.

The out-of-band API is used to integrate with job schedulers and manage data flushing to the PFS and also pre-stage the IME cache for new jobs so that first reads are from the IME cache instead of the backing disk store if possible.

Where HPC-style jobs have spiking IO loads which causes PFS delays, the IME can soak them up without the users have to upgrade their PFS store to handle such peak loads.

The IME14K is designed to deliver up to 50GB/s of throughput. Additional appliances can be clustered up to well over 1TB/s throughput or to increase the capacity of the NVMe cache.

"Limited availability shipments begin in December as the NVMe devices become available. Full availability in Q1 2016," said Rector. "We are not quoting pricing at this time." ®

Other stories you might like

  • IT staffing, recruitment biz settles claims it discriminated against Americans
    Foreign workers favored over US residents because that's what clients wanted, allegedly

    Amtex Systems Incorporated, an IT staffing and recruiting firm based in New York City, has agreed to settle claims it discriminated against American workers because company clients wanted workers with temporary visas.

    The US Department of Justice on Wednesday announced the agreement, which followed from a US citizen filing a discrimination complaint with the DoJ's Civil Rights Division’s Immigrant and Employee Rights Section (IER).

    "IT staffing agencies cannot unlawfully exclude applicants or impose additional burdens because of someone’s citizenship or immigration status," said Assistant Attorney General Kristen Clarke of the Justice Department’s Civil Rights Division, in a statement. "The Civil Rights Division is committed to enforcing the law to ensure that job applicants, including US workers, are protected from unlawful discrimination."

    Continue reading
  • Will this be one of the world's first RISC-V laptops?
    A sneak peek at a notebook that could be revealed this year

    Pic As Apple and Qualcomm push for more Arm adoption in the notebook space, we have come across a photo of what could become one of the world's first laptops to use the open-source RISC-V instruction set architecture.

    In an interview with The Register, Calista Redmond, CEO of RISC-V International, signaled we will see a RISC-V laptop revealed sometime this year as the ISA's governing body works to garner more financial and development support from large companies.

    It turns out Philipp Tomsich, chair of RISC-V International's software committee, dangled a photo of what could likely be the laptop in question earlier this month in front of RISC-V Week attendees in Paris.

    Continue reading
  • Did ID.me hoodwink Americans with IRS facial-recognition tech, senators ask
    Biz tells us: Won't someone please think of the ... fraud we've stopped

    Democrat senators want the FTC to investigate "evidence of deceptive statements" made by ID.me regarding the facial-recognition technology it controversially built for Uncle Sam.

    ID.me made headlines this year when the IRS said US taxpayers would have to enroll in the startup's facial-recognition system to access their tax records in the future. After a public backlash, the IRS reconsidered its plans, and said taxpayers could choose non-biometric methods to verify their identity with the agency online.

    Just before the IRS controversy, ID.me said it uses one-to-one face comparisons. "Our one-to-one face match is comparable to taking a selfie to unlock a smartphone. ID.me does not use one-to-many facial recognition, which is more complex and problematic. Further, privacy is core to our mission and we do not sell the personal information of our users," it said in January.

    Continue reading
  • Meet Wizard Spider, the multimillion-dollar gang behind Conti, Ryuk malware
    Russia-linked crime-as-a-service crew is rich, professional – and investing in R&D

    Analysis Wizard Spider, the Russia-linked crew behind high-profile malware Conti, Ryuk and Trickbot, has grown over the past five years into a multimillion-dollar organization that has built a corporate-like operating model, a year-long study has found.

    In a technical report this week, the folks at Prodaft, which has been tracking the cybercrime gang since 2021, outlined its own findings on Wizard Spider, supplemented by info that leaked about the Conti operation in February after the crooks publicly sided with Russia during the illegal invasion of Ukraine.

    What Prodaft found was a gang sitting on assets worth hundreds of millions of dollars funneled from multiple sophisticated malware variants. Wizard Spider, we're told, runs as a business with a complex network of subgroups and teams that target specific types of software, and has associations with other well-known miscreants, including those behind REvil and Qbot (also known as Qakbot or Pinkslipbot).

    Continue reading
  • Supreme Court urged to halt 'unconstitutional' Texas content-no-moderation law
    Everyone's entitled to a viewpoint but what's your viewpoint on what exactly is and isn't a viewpoint?

    A coalition of advocacy groups on Tuesday asked the US Supreme Court to block Texas' social media law HB 20 after the US Fifth Circuit Court of Appeals last week lifted a preliminary injunction that had kept it from taking effect.

    The Lone Star State law, which forbids large social media platforms from moderating content that's "lawful-but-awful," as advocacy group the Center for Democracy and Technology puts it, was approved last September by Governor Greg Abbott (R). It was immediately challenged in court and the judge hearing the case imposed a preliminary injunction, preventing the legislation from being enforced, on the basis that the trade groups opposing it – NetChoice and CCIA – were likely to prevail.

    But that injunction was lifted on appeal. That case continues to be litigated, but thanks to the Fifth Circuit, HB 20 can be enforced even as its constitutionality remains in dispute, hence the coalition's application [PDF] this month to the Supreme Court.

    Continue reading

Biting the hand that feeds IT © 1998–2022