Linux literally loses its Lustre – HPC filesystem ditched in new kernel

Version 4.18 rc1 also swats Spectre, cuddles Chromebooks


Linux has literally lost its Lustre – the filesystem favoured by HPC types has vanished in the first release candidate of version 4.18 of the Linux kernel.

Linus Torvalds’ announcement of the new release lauds the fact it’s shrunk markedly, much of which can be attributed to the removal of Lustre.

“The removal of Lustre may not be all that notable, because it does look like a lot of the development has been happening out of tree, which may be why it never really ended up working as well as people hoped in the staging tree,” Torvalds wrote. “ Greg [ Kroah-Hartman] clearly got pretty frustrated about it, so now it's gone.”

How frustrated? Kroah-Hartman explained Lustre's omission by saying it has "been in the kernel tree for over 5 years now" but "has not really moved forward into the 'this is in shape to get out of staging' despite many half-completed attempts."

"And getting code out of staging is the main goal of that portion of the kernel tree," Kroah-Hartman wrote. "Code should not stagnate and it feels like having this code in staging is only causing the development cycle of the filesystem to take longer than it should."

"There is a whole separate out-of-tree copy of this codebase where the developers work on it, and then random changes are thrown over the wall at staging at some later point in time. This dual-tree development model has never worked, and the state of this codebase is proof of that."

"So, let's just delete the whole mess," he decided, but left the door open for Lustre to be re-admitted once Lustre's developers "take the time they have spend doing those types of housekeeping chores and get the codebase into a much better shape, and it can be submitted for inclusion into the real part of the kernel tree when ready."

Lustre developers appear to be unfussed by the news: our scan of the project’s mailing lists didn’t find any comment.

Linux is not alone in losing interest in Lustre: Intel hired more than a handful of its prominent developers and added support for Hadoop but in April 2017 decided to stop offering its own version of filesystem and instead offered its code to the open-source community.

CES 2015 crowds brace for bluster

Hasta la vista Lustre, so long Spectrum Scale: Everyday HPC is here

READ MORE

When Lustre emerged in the year 2003 it had little competition for creation of large-scale filesystems. Nearly 15 years on and Red Hat offers Gluster, IBM’s Spectrum Scale (aka the GPFS General Parallel File System) and scale-out NFS can all do plenty of what made Lustre useful. HDFS has emerged, too, for big data workloads.

Linux losing Lustre won’t therefore deprive penguinistas of access to scale-out storage clusters. Indeed, even HPC rigs won’t necessarily miss the filesystem, given such operators tend not to need the very latest kernel. And even if they do want the newest kernel, there’s no indication they won’t be able to mount Lustre.

Torvalds’ post on the new release lauds 4.18’s new features, among them support for AMD GPUs, fixes for Spectre V4 – aka Speculative Store Bypass - on Arm CPUs, and drivers to make two-in-one Chromebooks better at switching between desktop and tablet mode.

“We had 1500+ developers attributed as authors in this merge window,” Torvalds concluded, adding that the changes in this release involved “91 maintainers, so just think about that for a while.” ®

Similar topics


Other stories you might like

  • Protonmail celebrates Swiss court victory exempting it from telco data retention laws

    Doesn't stop local courts' surveillance orders, though

    Encrypted email provider Protonmail has hailed a recent Swiss legal ruling as a "victory for privacy," after winning a lawsuit that sees it exempted from data retention laws in the mountainous realm.

    Referring to a previous ruling that exempted instant messaging services from data capture and storage laws, the Protonmail team said this week: "Together, these two rulings are a victory for privacy in Switzerland as many Swiss companies are now exempted from handing over certain user information in response to Swiss legal orders."

    Switzerland's Federal Administrative Court ruled on October 22 that email providers in Switzerland are not considered telecommunications providers under Swiss law, thereby removing them from the scope of data retention requirements imposed on telcos.

    Continue reading
  • Japan picks AWS and Google for first gov cloud push

    Local players passed over for Digital Agency’s first project

    Japan's Digital Agency has picked Amazon Web Services and Google Cloud for its first big reform push.

    The Agency started operations in September 2021, years after efforts like the UK's Government Digital Service (GDS) or Australia's Digital Transformation Agency (DTA). The body was a signature reform initiated by Prime Minister Yoshihide Suga, who spent his year-long stint in the top job trying to curb Japan's reliance on paper documents, manual processes, and faxes. Japan's many government agencies also operated their websites independently of each other, most with their own design and interface.

    The new Agency therefore has a remit to "cut across all ministries" and "provide services that are driven not toward ministries, agency, laws, or systems, but toward users and to improve user-experience".

    Continue reading
  • Singaporean minister touts internet 'kill switch' that finds kids reading net nasties and cuts 'em off ASAP

    Fancies a real-time crowdsourced content rating scheme too

    A Minister in the Singapore government has suggested the creation of an internet kill switch that would prevent minors from reading questionable material online – perhaps using ratings of content created in real time by crowdsourced contributors.

    "The post-COVID world will bring new challenges globally, including to us in the security arena," said Minister for Defence Dr Ng Eng Hen at a Tuesday ceremony to award the city-state's 2021 Defense Technology Prize.

    "For operations, the SAF (Singapore Armed Force) has to expand its capabilities in the digital domain. Whether for administrative or operational purposes, I think that we will need to leverage technology to the maximum," he declared.

    Continue reading

Biting the hand that feeds IT © 1998–2021