Ghost of DEC Alpha is why Windows is rubbish at file compression

OSes need one algo for all machines, so Redmond coded for the worst CPU it knew


Microsoft's made an interesting confession: Windows file compression is rubbish because the operating system once supported Digital Equipment Corporation's (DEC's) Alpha CPU.

Alpha was a 64-bit RISC architecture that DEC developed as the successor to its VAX platform. DEC spent much of the late 1990s touting Alpha as a step beyond anything else on the market at the time. It was mostly right: x86-64 didn't arrive until the year 2000. But that didn't stop DEC faltering and being acquired by Compaq in 1998.

But because Alpha was pretty impressive for its time, Microsoft did versions of Windows NT 4.0 and Windows Server 2000 to run on the silicon.

And that effort, writes Microsoftie Raymond Chen, is why Windows file compression remains feeble.

Chen says Windows' compression algorithms are chosen to ensure they don't degrade overall system performance no matter what box Windows runs on.

In the best possible world, Chen says you'd design an OS to “use compression algorithm X if the system is slower than K megahertz, but algorithm Y if the system is faster". Which is a fine way to match compression to a machine's capabilities, but a lousy way to make data portable because if a system only has access to algorithm Y, data created on an algorithm-X-using machine won't be readable. Which could make it impossible to move drives between machines.

“In other words, you cannot change the compression algorithm depending on what machine you are running on … every system has to be able to decompress every compression algorithm.”

Chen says one of his “now-retired colleagues worked on real-time compression, and he told me that the Alpha AXP processor was very weak on bit-twiddling instructions. For the algorithm that was ultimately chosen, the smallest unit of encoding in the compressed stream was the nibble; anything smaller would slow things down by too much. This severely hampers your ability to get good compression ratios.”

That problem was coded into Windows' compression algorithms and appears to persist to this day, which is why Windows file compression remains feeble.

So why doesn't Microsoft fix this? Chen also points out that “... many (most?) popular file formats are already compressed, so file system compression wouldn't accomplish anything anyway.”

But if nothing else it's nice to know that DEC Alpha lives on, albeit as an irritant, deep within the bowels of Windows. ®


Keep Reading

Googlers show off AI that can help developers protect crypto code from key-slurping side-channel attacks

DEF CON Deep-learning model pinpoints the pain in your AES

After huffing and puffing for years, US senators unveil law to blow the encryption house down with police backdoors

Lawmakers will attempt to bend the laws of mathematics to their will

Meet Morpheus, the AI that'll show you how deep the universe's rabbit hole goes: Code can detect, classify galaxies from 'scope scans

Way easier than getting humans to scour petabytes of images by hand, looking for faraway systems

As if you needed another reason not to use Visual Studio, C++ extension for Visual Studio Code is live

Optional extra pack a bit like Microsoft Plus! but with CMake instead of Pinball

Google says Australian pay-for-news code means it can’t quit the country

Pushes back against accusation of misinformation with argument its only option is to stop indexing almost everything

Intel, boffins invent an AI Clippy for code: Hi, I see you're writing another lock-free bloom filter. Can I help?

Proof-of-concept algorithm-matching system paves way for more complex recommendation engine

Let's roll the 3d6 dice on today's security drama: Ah, 15, that's LG allegedly hacked, source code stolen by Maze ransomware gang

Crooks threaten to leak swiped software blueprints

Come on, Amazon: If you're going to copy open-source code for a new product, at least credit the creator

Developer wishes cloud giant gave more thought to not stepping on toes

Biting the hand that feeds IT © 1998–2020