Linux Foundation projects on AI and data merge – because one of these concepts simply can't exist without the other

Combined org hoping to attract more industry support


Open Source Summit Europe LF AI Foundation and Open Data Platform Initiative (ODPi), until now two orgs under the Linux Foundation umbrella, are merging to become the LF AI and Data Foundation, a home for projects devoted to AI, machine learning (ML), data, and analytics.

The new foundation was unveiled at the Open Source Summit Europe, which is of course virtual-only (though held in timezones friendly to European folks), and taking place this week.

"We've been having an increasing number of data projects join LF AI," said executive director Dr Ibrahim Haddad in a press briefing. "It's the nature of AI, one is dependent on the other."

Amazon hosts a project with us. Microsoft hosts a project with us. Uber hosts three projects with us, and none of them are a member

The LF AI Foundation was set up to address issues like fragmentation, lack of integration, and governance across open-source AI projects, Haddad told us. "We started two-and-a-half years ago with a single project," he said, "and today we are hosting 22 projects."

Several projects cover one of the hottest topics in AI: fairness and explainability.

The Linux AI Foundation Landscape puts its projects into context at https://landscape.lfai.foundation/

The Linux AI Foundation Landscape puts its projects into context at https://landscape.lfai.foundation/ (click to enlarge)

On the ODPi side, program director John Mertic said the organisation's activity has shifted since it was formed in 2015 with a focus on Hadoop. It has devoted a lot of attention to Egeria, a standard for exchanging metadata between products and databases.

The core issue is that while different products and projects often store similar entities – such as names, addresses, and social security numbers – they may store them in different ways so exchanging data or applying compliance and security rules across a whole organisation is problematic. There are many existing approaches, but Mertic said Egeria's code-first outlook is distinctive.

Is Egeria winning adoption? "It's being brought into IBM products, it's on the roadmap for SAS, some end users are looking at it," he told us. "It's up and coming. Combining forces with LFAI and creating a larger community will only increase that."

ODPi currently lists seven members including IBM, Cloudera, and SAS. LF AI shows 11 premier members, 14 general members, and 12 associate members. Premier members include AT&T, Baidu, Huawei, Ericsson, IBM, SAS, and Tencent. Some big names, but we noticed that they do not include well-known companies deeply involved in AI and ML, such as Google, AWS, Microsoft, and Nvidia, and that some of the biggest open-source projects in AI and ML, such as Facebook's PyTorch and Google's TensorFlow, are not LF AI projects. Does this make LF AI, and the new merged organisation, less representative than it should be?

There are some 10 companies to be added, said Haddad, but it also felt like we touched a nerve. "There are gaps, but from my perspective they are financial gaps," he said.

"Amazon hosts a project with us. Microsoft hosts a project with us. Uber hosts three projects with us, and none of them are a member. That goes to the specific open-source strategy at these companies and how they see their investment.

"However, from a technical perspective, I don't think there is a gap because all of these companies are using or integrating some of our projects in their cloud offerings, but they are not a member. It's a very interesting question and I would rather you ask that question to those companies – why are you not a member of LF AI although you host a project there?"

It does seem that merging these two entities is in part about trying to increase the scope and clout of the foundation in order to attract more support from some of these giant players with deep pockets. "It is a natural fit," said Mertic, "and the justification for companies like Google becomes a heck of a lot easier." ®


Keep Reading

China’s government-anointed Git operator says it will become a Linux Foundation mirror

Starting with just two projects - Baetyl and EdgeX - but eventually all code will be shared behind Great Firewall

Facebook to take board seat at Linux Foundation after signing as Platinum member

Already a big mover in Foundation projects like Presto, GraphQL, Osquery and ONNX, so why not go all-in?

Linux Foundation rolls bunch of overlapping groups into one to tackle growing number of open-source security vulns

OpenSSF to take projects from CII and OSSC under its umbrella

Linux Foundation, IBM, Cisco and others back ‘Inclusive Naming Initiative’ to change nasty tech terms

But Red Hat points out its code-base is so big that mastery of inclusivity will take time

Security-focused microkernel goes in for some seL4-care, becomes a foundation to keep that ecosystem growing

Being under the Linux Foundation umbrella is a good way to get that done

Linux Foundation starts new group to build pandemic-popping software

Decentralised contact-tracing apps from Ireland and Canada are first off the rank

Relying on plain-text email is a 'barrier to entry' for kernel development, says Linux Foundation board member

Interview Microsoft's 'open source wonk' Sarah Novotny wants to see easier ways for people to get involved

At Mozilla VPN stands for Vague Product News: Foundation reveals security product will launch eventually, with temporary pricing, in unspecified places

But it does have a name. 'Firefox Private Network' is out, ‘Mozilla VPN’ is in.

Biting the hand that feeds IT © 1998–2020