Apache packages: creating a support vacuum

OSS in the real world

Comment Right at the end of OSCon in September, I got the opportunity to harangue Ubuntu's Mark Shuttleworth about the support vacuum distros such as his create. He didn't have an answer. OK, I was putting him on the spot, but I don't think he'd have done much better if he'd had notice of the question. Frankly, I don't think there is an answer.

So what's the problem? We at Apache know our own software, and are prepared to support it in fora such as the Apache Users mailinglist, and the #apache IRC channel.

Among the diverse issues we have to deal with are the many users who have installed Apache from a third party package. Many of these packages are rearranged so as to be essentially unrecognisable; and in particular, some packagers have very bizarre ideas about how to organise Apache's configuration files. That makes it hard, sometimes impossible, to support the end-users: we may be talking completely at cross-purposes. And for a change, the worst problem is not with Windows packages, nor with commercial Apache packagers like IBM and Oracle (for which, at worst, we can disclaim all responsibility), but with Linux distros. By far the biggest offender, in terms of the support headache it throws on us, is the Debian family, notably Debian itself and its near-clone Ubuntu.

Now, if we at Apache stick to what we know about, we'd tell the questioners to go to a Debian support forum. But the Debian people will tell them to use an Apache forum. Where should the burden of support lie?

newbie: How do I stop it listing all my files when there's no index.html?
niq: Options -Indexes
fajita: Options -Indexes turns off directory indexing. Ask me about mod_autoindex and Options for more details.
newbie: So where do I set that?
niq: httpd.conf
...fajita explains how to find httpd.conf ...
newbie: httpd.conf is empty. Do you mean apache2.conf? Or something in conf.d/?
niq: Probably. See your distro's docs.
newbie: huh? it's just a standard apache installation.
niq: distros
fajita: distros sometimes mangle your apache installation, putting apache and its files in different places to the standard. It's hard for us on #apache to help if your installation is mutilated this way.

And no one is satisfied. And that's just a simple example: others involve a lot of confusion with nonsense like, for example, per-module conditional .conf files, and distro-specific ways of manipulating them.

They've configured it right, but Apache ignores it because it needs some extra distro-defined magic to activate the configuration file in question. Now they're banging their head in desperation, and we can't help because we don't know the magic. And we get the blame.

You might of course reply: therein lies a great opportunity; a niche market waiting to be exploited. And yes, there are people and companies (your humble scribe included) who earn our bread and water in something not far from this niche. Though far from being as lucrative as, say, supporting Oracle (with open source and decent documentation, the barrier to entry is far lower), it works. But that's at the top end: medium and large businesses, with custom needs and a budget to meet them. Not all Apache users fall into that category! So, that still leaves a support vacuum.

Back to Ubuntu and my encounter with Mark Shuttleworth. I put the question to him in general terms, citing Apache merely as the example I've had experience with at the sharp end of support. His answer, in similarly general terms, was that Ubuntu would expect to be the first point of support, but he would hope that upstream providers would also get involved. Sorry, but that just doesn't work.

First, Debian/Ubuntu would have to provide a complete set of documentation, at the level of Apache's own. Otherwise, people will naturally come to apache.org, and get confused when it appears to bear little relation to what's installed on their computer. And even if they did provide a full handbook, that still leaves a Tower of Babel amongst third-party articles.

Second, sorry, but speaking as an Apache developer, I make the time to chip in to the task of supporting our users. But I draw the line at taking the time to figure out the quirks of someone else's gratuitously much-mangled package. If there was just the one distro, things might be different, but it's not just Debian/Ubuntu - it's Gentoo, Redhat, SuSE, FreeBSD, Windows, Solaris, etc, etc. If you want me to learn the quirks of your distro so I can support it, you can bloomin' well pay me! And to be fair, the commercial distros do pay Apache people - which may be one reason why the biggest support headache thrown back on us comes from the pure freebies.

Well, OK, so don't support distros then. Is it that hard?

Actually, yes it is, unless we stop supporting anything at all. We can't stop people with distro-induced trouble asking in our fora, and they will blame us (sometimes loudly and obnoxiously) if they don't get what they want. This creates real tension.

Now, there's an experimental effort underway to try to close this gap with documentation. The core of it is a wiki page, now at http://wiki.apache.org/httpd/Recipes/DistrosDefaultLayout. It's driven by Apache support people, frustrated with a constant stream of "huh? what are you talking about?" posts. It's not yet clear how useful this will be in closing the support gap; at the moment, it's essentially a quick-reference card for mapping between distros.

Having sounded off at distro packagers, I should say that they do of course, play a vital role. Packaging Apache, including as they do a lot of third party stuff, lowers the bar to entry for the many users who would be uncomfortable with the idea of compiling a package from source. Some third party packages can be challenging for a non-expert admin to build, and distro packagers can be exactly what they need.

Given all that to contend with, do we really need confusing packages to thrust extra complexities onto both us and our users? A plea to Debian and Ubuntu: if you must repackage Apache in a radically different manner, please document it properly! That includes, along with the manual, a prominent notice to your users, that Apache and third-party documentation and support describe something different, and will not always be applicable!

There's feedback on this article on Nick's blog here

Other stories you might like

  • It's primed and full of fuel, the James Webb Space Telescope is ready to be packed up prior to launch

    Fingers crossed the telescope will finally take to space on 22 December

    Engineers have finished pumping the James Webb Space Telescope with fuel, and are now preparing to carefully place the folded instrument inside the top of a rocket, expected to blast off later this month.

    “Propellant tanks were filled separately with 79.5 [liters] of dinitrogen tetroxide oxidiser and 159 [liters of] hydrazine,” the European Space Agency confirmed on Monday. “Oxidiser improves the burn efficiency of the hydrazine fuel.” The fuelling process took ten days and finished on 3 December.

    All eyes are on the JWST as it enters the last leg of its journey to space; astronomers have been waiting for this moment since development for the world’s largest space telescope began in 1996.

    Continue reading
  • China to upgrade mainstream RISC-V chips every six months

    Home-baked silicon is the way forward

    China is gut punching Moore's Law and the roughly one-year cadence for major chip releases adopted by the Intel, AMD, Nvidia and others.

    The government-backed Chinese Academy of Sciences, which is developing open-source RISC-V performance processor, says it will release major design upgrades every six months. CAS is hoping that the accelerated release of chip designs will build up momentum and support for its open-source project.

    RISC-V is based on an open-source instruction architecture, and is royalty free, meaning companies can adopt designs without paying licensing fees.

    Continue reading
  • The SEC is investigating whistleblower claims that Tesla was reckless as its solar panels go up in smoke

    Tens of thousands of homeowners and hundreds of businesses were at risk, lawsuit claims

    The Securities and Exchange Commission has launched an investigation into whether Tesla failed to tell investors and customers about the fire risks of its faulty solar panels.

    Whistleblower and ex-employee, Steven Henkes, accused the company of flouting safety issues in a complaint with the SEC in 2019. He filed a freedom of information request to regulators and asked to see records relating to the case in September, earlier this year. An SEC official declined to hand over documents, and confirmed its probe into the company is still in progress.

    “We have confirmed with Division of Enforcement staff that the investigation from which you seek records is still active and ongoing," a letter from the SEC said in a reply to Henkes’ request, according to Reuters. Active SEC complaints and investigations are typically confidential. “The SEC does not comment on the existence or nonexistence of a possible investigation,” a spokesperson from the regulatory agency told The Register.

    Continue reading

Biting the hand that feeds IT © 1998–2021