When Facebook moved its servers to HipHop for PHP – the code transformer it built to convert PHP into optimized C++ – the company's average CPU usage dropped by 50 per cent. And after six months of additional engineering, the tool was about 1.8 times faster.
Now, after another six months, the company says, it has improved performance another 1.7 times. But this is more than just self-congratulation. The project is open source, and it has been open since Facebook first switched its servers to HipHop in February 2010.
Unlike Google, you see, Facebook has been known to promptly open-source some of the most important pieces of its back-end infrastructure.
With a Wednesday blog post, Facebook research scientist Xin Qi charts HipHop's relative throughput improvement over the last six months:
The end result, he says, is that HipHop can handle about 70 per cent more traffic on the same hardware infrastructure.
After transforming PHP into C++, HipHop compiles the code and builds binary files with the GNU C++ compiler, aka g++. The idea is that you can still code with high-level PHP, but then get the performance of C++ – though it does give up certain "rarely used" PHP features.
According to Qi, Facebook and the open source community have juiced the tool in several different ways. HipHop uses a version of the Alternative PHP Cache (APC), and engineers have stripped most of the serialization and unserialization operations. "Semantically, an object is serialized and unserialized when it is stored into and fetched from APC. However, serialization and unserialization are costly operations, commonly dominating the cost of APC data fetching itself," Qi says.
"Thus, we reworked the APC implementation in HipHop, getting rid of almost all the serialization/unserialization operations, while keeping the semantics equivalent to before."
But some serialization is still required, and this has been fine tuned. "Objects still need to be serialized or JSON-encoded in order to transfer them over the wire. To make things faster, we optimized various aspects of these operations, including UTF8/UTF16 conversions, object property accesses, number parsing, and so forth."
Facebook has also reduced the size of the binary code, improved memory allocation, and made several changes to the compiler. "Several phases in the compiler, including parsing, optimization, and code generation, are now parallelized. Hyves contributed changes to the generated C++ code to make it compile faster without losing any run-time efficiency," Qi says.
His crew can build a more than 1GB binary in about 15 minutes (after stripping out debug information). "Although faster compilation does not directly contribute to run-time efficiency," Qi says "it helps make the deployment process better."
The likes of Drupal, MediaWiki, and WordPress are now using HipHop. No one is using BigTable or the Second Coming of the Google File System. Except for Google. ®