Mozilla's MDN web standards reference platform makes move to GitHub, now in beta

Jamstack, Kubernetes, community contributions, and automatic translation – what could go wrong?

Mozilla's MDN JavaScript and web technology reference documentation is migrating to a new GitHub-based platform, codenamed Yari, which is now in beta and taking contributions.

Unlike most development platforms, there is no single official source of documentation for the concoction of JavaScript, HTML, and CSS used to code web pages.

There is an official ECMAScript standard, various W3C standards at various stages of recommendation, further standards at the WHAT WG (Web Hypertext Application Working Group), and documentation for browser engines meaning mainly Blink (Chromium), WebKit (Safari), and Mozilla's Gecko or Quantum, plus the JavaScript runtimes V8, SpiderMonkey, and Apple's JavaScriptCore.

The result is that when a hapless developer types something like "how to search JavaScript array" into a search engine they are likely to find (alongside blogs and StackOverflow posts) reference information on two sites, w3schools and Mozilla's MDN. W3schools is not connected with the W3C, takes advertising, and is aimed at learners, though it is a cleanly presented site and often helpful. MDN is more detailed, more technical, and generally preferred by professional developers. Both sites include the crucial browser compatibility tables which tell developers whether they can realistically use some nice feature.

MDN is therefore important, so when troubled Mozilla laid off many of the MDN documentation team back in August it caused consternation among web developers. At the time Mozilla CEO Mitchell Baker wrote vaguely about a "new focus on product" while also informing employees of a reduction in the workforce of about 250 heads.

It was obvious that for MDN to continue it requires more voluntary contributions than before and senior tech writer Chris Mills made a "communication call for participation" last month. He also said that MDN is migrating to GitHub, with content currently in a MySQL database moving to a Git repository, the aforementioned Project Yari.

More details came last week and yesterday Yari went into beta and its repository is now public. Mills said "the exact form of the platform is yet to be finalized" but also that "the first release will happen on December 14." The team must have considerable confidence in the beta to deliver version 1.0 in just six weeks.

Mills said that the existing platform is "complex and hard to maintain" and that Yari will be simpler to manage. It also moves MDN from a Wiki model to one based on pull requests, which Mills said is "much better for contribution," one of the reasons being that contributors can edit using whatever code editor they prefer. It also means that content edits are reviewed before approval, unlike the Wiki approach where they appear instantly and are reverted if someone notices an error or spam.

Project Yari will deploy content from GitHub to AWS S3 for service as a Jamstack site

The new MDN will be deployed to Amazon's platform using a Jamstack model, where websites are composed of static files supplemented by services called from JavaScript for dynamic content. A daily process will deploy content from GitHub to AWS S3, where it will be served to users via a CDN (content delivery network). AWS Lambda will be used for dynamic content, along with a Kubernetes cluster with data in MySQL for account and search services. ElasticSearch is also used as part of the index building process. The existing MDN uses Kubernetes and MySQL for all the content, so the new Kubernetes cluster is "radically smaller" than the previous one. In July, Mills said that MDN attracts "roughly 15 million views per month".

Mozilla has also created a new Sass (CSS pre-processor) for MDN, called mdn-minimalist, which is used for styling. Mills said that many accessibility issues will be addressed by the new design.

Translation and localisation is another tricky area, again constrained by limited employee resources. While it appears that no final decision has been taken, the plan is to use "machine learning-based automated translations" for the most important MDN content, and to allow community members to improve the automated translation manually.

Mills said that "machine translations are an acceptable solution in most cases, if not a perfect one." In testing, Mills said that users preferred content that was badly translated but up to date to better translation of out-of-date content. In practice, users can also turn to services like Google Translate or Microsoft Translator to get content in their language – though automatic translation is hazardous when it comes to technical content because precision is required.

The unknown question is the extent to which MDN will attract the resources it needs to maintain its good track record. Mozilla is providing the platform, but it will still require extensive community input for the Yari project to deliver on its goals. ®