SFX studio uses Avere turbocharger to accelerate Isilon rocket

We know that doing that would obviously only work metaphorically

Comment Who would think a rocket needs a turbocharger? Yet data access from Isilon filers is being accelerated by front-end Avere FXT devices in a movie special effects rendering environment.

Avere has previous in the SFX industry, having been used to render effects for the film Gravity, accelerating hardware-accelerated BlueArc filers.

Now the Moving Picture Company (MPC) is using Avere’s FXT Edge Filers to get clustered Isilon scale-out filers performing as MPC’s extreme SFX demands need them to do. It’s not like Isilon filers are slow in everyday corporate work, but they weren’t fast enough for MPC.

The company has crafted effects for Guardians of the Galaxy, Maleficent and mutant-slaying Sentinels in X-Men: Days of Future Past. It has offices in London (Soho), Montreal and Bangalore, where its visual effects (VFX) artists work.

MPC’s London Isilon cluster in 2009. Now MPC augments the core Isilon systems with the Avere kit

Martin Weaver, MPC’s Head of Infrastructure Delivery, says: “Avere’s solution provides cost-effective performance acceleration for rendering and other heavy-I/O workloads.” In other words, there are no commercially-available filers that are sufficient on their own for this workload, which we find surprising.

But then VFX work ramps up storage demands at an incredible rate as picture resolution and effects complexity increases. This is the bleeding edge of filer data access.

Weaver adds: “Moving higher up the echelons of VFX quality dramatically increases the amount of data that goes into rendering an image. That means we’re working with larger and larger datasets to deliver the client’s desired visual results, whether for photo-realistic images, full-frame characters or convincing fluid-based simulations.”

“In the VFX industry where last year’s best becomes this year’s norm, the amount of data that goes into image rendering grows at an even faster rate than Moore’s law predicts on the compute side. Moving from 2K to 2K-stereo or 2K to 4K resolution, for example, means more data and more IOPS and throughput.” Ten years ago, the Isilon kit was leading edge. Now it’s trailing edge and needs turbocharging.

The VFX artists – and there are hundreds of them at each site – use apps such as Autodesk Maya, The Foundry NUKE, and Pixar RenderMan. MPC doesn’t want these costly bods waiting while the storage system finds and delivers the files they need for their work.

The UK render farm is located at Pinewood, outside London, as office space in London costs too much money to house it there. An Avere edge filer was deployed at the Pinewood site “to accelerate NFS performance to the farm and simultaneously reduce network traffic between the Pinewood data centre and our London facility”.

The network bandwidth needed would cost double its current total if the Pinewood Avere kit was taken away.

Other Avere edge filers are used in MPC’s London, Montreal and Bangalore sites, where Avere clusters offload the EMC Isilon core storage to speed rendering processes and provide fast interactivity to artists' workstations.

Scene from X-Men: Days of Future Past

“To prevent general slowdowns as demand increases, we redirect much of our heavy-IO traffic to Avere,” Weaver says. "The Avere cluster effectively boosts I/O performance so that we can keep up with the rendering demand and at the same time take the strain off our core storage systems to ensure responsiveness to other applications.”

In the London office, an Avere cluster is used “to enable faster loads from a third-party software and tool server. Instead of waiting minutes to pull required rendering scripts and other data off the software server, render nodes can start almost instantly. With hundreds of machines simultaneously attempting to initiate the same render, time-savings are dramatic”, compared to relying just on the Isilon kit.

In other MPC facilities, Avere’s deployment means the core storage systems can avoid the use of costly tier-1 disk drives – think 15K rpm jobs – and so save money by using cheaper drives without degrading performance.

Read an Isilon MPC case study here (pdf). Check out Avere’s case study here.

Avere was founded in 2008. Isn't it about time that filer manufacturers recognised the validity of Avere's tiered, IO-type oriented architecture and did a better job of making their filers go faster? It's basically a clever caching scheme. Come on guys: caching isn't rocket science. ®