SPONSORED Achieving consistent ultra low latency is often top of the list of requirements for those investing in all flash arrays.
The days when it was only big capital markets financial services players operating automated or algorithmic trading applications who were obsessed with sub-millisecond latency have long passed as today cloud, mobile and more and more enterprise applications require this level of responsiveness.
The race to zero has also occupied some of the brightest engineering and software minds in the business.
Efforts to achieve consistent, stable, low latencies require dedication and are ongoing at firms such as Huawei which has been developing flash technologies since 2005.
With the launch of its latest family of OceanStor Dorado V3 all flash arrays the firm has put great emphasis on its in-house developed technology with the direct aim of reducing latency.
Improving performance of the storage media required software be designed from the ground up and the OceanStor Dorado V3 uses Huawei’s proprietary FlashLink technology to address the specific needs of SSD latency.
FlashLink helps achieve improved global wear leveling–treating all NAND components as one memory unit - hot/cold portioning, end-to-end I/O prioritization, full-stripe sequential writes, and other operations particular to flash memory-based systems.
The FlashLink technology when combined with Huawei’s inhouse developed disks and proprietary controllers delivers a latency of just 0.5 ms (See below).
Developed to be used with its own disks and controllers for optimal performance and reliability the FlashLink disk controller collaboration technology can double the latency performance, says Huawei.
Some of the FlashLink features include adoption of global cold/hot data partitioning to reduce the number of garbage collections. [Garbage collection is a form of automatic memory management for freeing available resources. In SSD, this process is used for freeing the blocks by moving the available pages to a new block and erasing the old blocks.]
Intelligent I/O priority adjustment technology ensures response in I/O reads and writes for the host ahead of other devices to help keep latency quick.
The controllers use an optimised cache algorithm which improves overall storage performance.
Huawei says, “By replacing the binary tree algorithm with a hash algorithm it enables data to be inserted instantly – accelerating data writes.”
The cache query algorithm divides a cache index table into three partition index tables, hot data, warm data and cold data. Hot data is the data most used by an organisation, cold data is that which is least used. This accelerates data queries.
FlashLink provides partitioning of hot and cold data. Metadata and data are stored in different partitions inside the SSDs and controllers. This also helps accelerate data reads and writes.
For sequential writes of large blocks, the controller senses large blocks inside SSDs and aggregates multiple single IOs into a large IO before writing it onto an SSD, this will maximise the read write data in unit time. Data read write IOs have the highest priority inside the SSD and the controller. By performing global garbage collection, this reduces the volume of data inside the SSD.
To give full performance to all SSDs and flash, the FlashLink technology optimises flash from three aspects, basic function layer, data service layer and data layout layer.
Huawei developed SSD chips with the core Flash Translation Layer algorithm identifying the position and data volume, improving write speeds and providing a low latency of 40us [Flash translation layer is the software layer of flash memory, which is responsible for the logical to physical address mapping, and also includes functions such as garbage collection, wear levelling and bad blocks management]
OceanStor OS provides CPU partitioning, lock-free threading for multiple cores, time complexity prioritization cache algorithms, and other utilities to ensure latency remains low for host read/write requests in the scheduling and resource allocation operations in the system software.
Considering the cost and limited write cycles of SSDs, inline deduplication and compression are must-haves in any all-flash system. All-flash systems use redirect on write (ROW) mechanisms to achieve the needed wear leveling and deduplication/ compression features. Another must-have feature in enterprise-class storage is end-to-end data integrity field (DIF) to avoid silent data corruption. This feature is used with the Data Integrity Extensions (DIX) function to ensure integrity of data in every process from services to storing on mediums. Snapshot, used to execute local backups, is yet another essential feature to achieving a high reliability program. However, running all these features will draw away from performance to varying extents.
Even with these draw downs on performance FlashLink ensures Huawei’s OceanStor Dorado V3 latency is maintained at ultra low levels. In testing, OceanStor Dorado V3 all flash storage systems running on OceanStor OS and Huawei-developed hardware adopting innovative FlashLink technology and homegrown SSD controllers provide the needed end-to-end optimizations to deliver 0.5 ms stable and predictable latency. In the SPC-1 benchmark, OceanStor Dorado V3 with 4 controllers delivers 1,000,560 IOPs at 0.5 ms latency. At the high end FlashLink technology ensures 4 million IOPS at 0.5 ms consistent latency. That is 0.0005 second.
The 4 million IOPs at half a millisecond latency places the OceanStor Dorado V3 as the leading industry performance in all flash storage. the storage array is designed from the chipset, to controller, to the module interface and back to the OS layer.
The low latency performance of the Huawei OceanStor Dorado V3 systems are testament to the R&D efforts embarked on by the company over the last decade. Huawei’s FlashLink technology is a key element in providing the sub millisecond latency in a stable, consistent manner required by today’s telco, finance and cloud providers.
SPONSORED BY: Huawei