Behold. The benchmarkers have brainstormed a fresh filer performance measure – SPEC SFS 2014. It looks at end-to-end filer performance instead of looking at component levels, as did the previous SPECsfs 2008 measure.
SPEC SFS 2014 stands for the Standard Performance Evaluation Corporation and its Solution File Server 2014 test. The big news is that it is protocol-independent with no more separation of NFS and CIFS results. Instead, there are four separate application areas - making it more, rather than less, complex.
SPEC SFS committee members contributing to the new benchmark include EMC, Hitachi, Huawei, IBM, Microsoft, NetApp, Oracle and Seagate. Iozone.org provided the source code for the benchmark. The committee said the new test has five new aspects:
- It measures the performance of the entire storage configuration as it interacts with application-based workloads.
- Four different workloads representing specific application areas
- Number of simultaneous builds that can be done (software Build)
- Number of video streams that can be captured (VDA)
- Number of simultaneous databases that can be sustained (Database)
- Number of virtual desktops that can be maintained (VDI)
- Can run over any version of NFS or SMB/CIFS and on any clustered, object-oriented, local or other POSIX-compatible file system
- Methodology that permits vendors to demonstrate their unique storage accelerators
- Users can create their own customised workloads for measuring storage system performance
Where the 2008 test was split into separate NFS and CIFS results, making it simple and easy to interpret, there are four parts to the new test: database, software build, VDA and VDI.
When any suppliers file results they can’t be compared with the existing SPECsfs 2008 results.
SPEC SFS 2014 diagram
SPECsfs 2008 results were divided into relatively few CIFS ones and more populous NFS ones. The general trend of SPECsfs 2008 NFS results shows that one of the simplest tactics to get a fast filer is to stick an Avere FXT accelerator in front of it.
SPECsfs 2008 results - to make a filer go faster stick an Avere box in front of it
We could say that SPECsfs 2008 had run its course and something more up to date and relevant to filer suppliers is needed.
This new benchmark is still a networked filer test but it appears you can’t use it for Big Data, Hadoop-style configurations as they aren’t fully POSIX-compliant. As Hadoop installations may use commodity hardware components it’s perhaps unrealistic to expect a committee of proprietary filer suppliers to devise a benchmark compare performance of their arrays versus ones built from white box servers and JBODs.
Pity though, as big data analytics is going to become more and more important and knowing what storage server configurations to use for it would be most helpful.
You can download the SPEC SFS 2014 benchmark from the SPEC website; the cost is $2,000, with discounts for qualified research and academic institutions. ®