Microsoft fixes Windows 'idiosyncrasy' that hampered some SMB file transfers

You might find copying data over a network goes at least a little quicker

Microsoft this month fixed what one of its staffers called an "idiosyncrasy" in the end-to-end SMB compression functionality it began rolling out last year for Windows Server 2022 and Windows 11.

The upshot is: you may find your network file transfers run at least a little faster than before.

Starting in June 2021, the Redmond IT giant documented and pushed out a new feature that, when activated by a user, administrator, or application, would automatically compress documents transferred over SMB file sharing and decompress them at the destination.

There was a weird catch. The code would only compress all of the file if at least 100 MiB of the first 500 MiB was lossless compressible. If less than 100 MiB could be compressed, the algorithm would give up. If the file was under 100 MiB, the algorithm would therefore give up. This behavior could be overridden with registry key edits, but who has time for that?

With the optional release this month of the KB5016693 (OS Build 20348.946) update to Windows Server 2022 and a similar update for Windows 11 (KB50116691, or OS Build 22000.918 preview), now Microsoft's code will always attempt to compress the whole file if compression is requested.

"If fewer than 100 MiB was compressible, SMB compression stopped trying to compress the rest of the file," Ned Pyle, a principal program manager in Microsoft's Windows Server High Availability and Storage Group, explained in a blog post on Friday.

"If at least 100 MiB compressed, SMB compression attempted to compress the rest of the file. This means that very large files with compressible data – for instance, a multi-gigabyte virtual machine disk – are likely to compress but a relatively small file – even a very compressible one – would not compress."

He added that the registry overrides for this limitation were "way too esoteric for folks to know," and so no one was using them. This was "an idiosyncrasy in the initial design," he said.

With the latest updates, "if you request compression over SMB, we will attempt to compress the file regardless of its size or seeming suitability for compression. It doesn't matter how you request compression – robocopy, share setting, registry values – we will try to compress the file," he wrote. "Now we just assume if you try to compress, you want to compress."

Pyle's post above includes instructions, including PowerShell and Group Policy details, and a four-minute video on how to set up and use this compression feature.

Compression is quite useful when moving data over networks, as there's less information to move and thus time is saved. This is handy if you have a connection with low bandwidth. Especially in this day and age, the amount of time taken by the CPU cores to compress and decompress the data will be outweighed by the time saving of the smaller transfer. Compressing a large ISO on the fly makes sense; there's no need for it with already squashed files, like JPEGs and ZIPs.

SMB compression becomes even more crucial with the growing trend toward hybrid work models, with users trying to send or receive files over their bandwidth-challenged home networks.

"This does mean that transferring some smaller, less-compressible files will now get some useless CPU time spent trying to compress them, but the solution is to just know your files," Pyle explained. "JPG, ZIP, DOCX, etc., formats that are already compressed see no advantage from SMB compression no matter their size. VHDX, ISO, DMP, and other large files with lots of whitespace will compress quite well."

More details, and the algorithms used, are here. ®

Similar topics

Other stories you might like

Biting the hand that feeds IT © 1998–2022