Open Compute Project testing is a 'complete and total joke'

Source questions integrity, neutrality of certification programme

'They're not even looking for things like data integrity'

El Reg: What have you noticed about OCP testing?

Test engineer: As a longtime enterprise engineer, I'm very sceptical of this concept, as the bottom line of this programme simply seems to be building out cheap systems and data centres that have the lowest cost of ownership over their lifetime. So, basically, they're looking for cheap engineering, components, testing, manufacturing, low power consumption and low thermal generating systems. Quality does not seem to be a metric of consideration.

El Reg: How does this differ from your experience?

Test engineer: Most name-brand system, storage and networking companies spend quite a bit of money testing and certifying solutions over the course of several months. They use specialised software tools, internal diagnostics, protocol analysers, error injectors and lots of thermal/electrical/shock/vibe type of certifications.

El Reg: How does OCP testing differ?

Test engineer: The weirdness that I discovered is the OCP certification or testing process is a complete and total joke, which can be completed in about one day or less. They're not even looking for things like data integrity, lock-ups, gross errors or error injection/recovery, which are the first commandments of enterprise testing.

OCP test suite time, taken from page 78 of OCP C&I Storage Certification Specification and Test Plan v0.12

El Reg: Where is OCP testing carried out?

Test engineer: I believe the only OCP certification centre in the USA is at UTSA in San Antonio. This "neutral and third-party" lab is being run by a VP of Rackspace, who is also a UTSA employee. Not exactly neutral or third-party.

And from what I know of this school, it is a computer school that does not have a strong history of computer engineering.

El Reg: Why does this matter?

Test engineer: If it was just a Facebook project, then this probably wouldn't be a story. But many others are jumping on the bandwagon with even financial institutions such as Fidelity and Goldman Sachs saying they will be mostly OCP soon.

Ultimately a social media cloud does not require true enterprise reliability. However, financial institutions cannot experience data corruption, data loss, unexpected errors/error recovery or any level of unexpected performance anomalies. These types of errors could be catastrophic or they could even potentially be used to exploit weaknesses in financial clouds.