SwiftStack is adding load-balancing to its open source object storage, deepening its S3 integration and adding native file access support.
The latest version of SwiftStack, v4.0, has four new capabilities and three future ones coming:
- It has load-balancing across its nodes which, SwiftStack claims, reduces a need for dedicated network hardware, shortens access latency, lowers bandwidth costs, and enables a larger number of storage nodes
- Integrated third-party metadata indexing and search support, with SwiftStack saying this makes the product analytics-ready
- Desktop client (SwitftStack Drive) support to get access to objects directly from desktops or laptops
- Better management with IPv6 support, capacity planning and data migration tools
The load balancing involves proxy nodes grouped by a common virtual IP (VIP) address, with overlapping groups for failover. A DNS CNAME record abstracts the VIPs, and applications address the DNS name and are randomly assigned different VIPs.
The metadata searching involves SwiftStack automatically sending metadata for each object to Elasticsearch, so it can be indexed and thus made searchable.
SwiftStack Drive mounts a container on your desktop so you can read and write files as you would with any drive.
Three future features are all Amazon-related. They start with the open source development of integrated scale-out file services with S3 and Swift-based object APIs, supporting SMB and NFS. Files can come in over SMB and accessed through object APIs and visa versa. Currently SwiftStack has a file system gateway.
Secondly, there will be central policies to replicate on-premises object data to any S3-compatible cloud.
Thirdly, SwiftStack says it will introduce universal access to scale-out private object storage as well as synchronisation with any Amazon S3-based public cloud. It says the object synchronisation is to enable cloud-to-cloud data locality where applications can make the best use of the data. Once the objects are in the S3-compatible store then other applications, and users who may not have access to SwiftStack, can now utilise the objects.
The "universal access" point probably means widening the population of end-point devices through which a SwiftStack object store could be accessed.
Analytics is flavour of the month. An object storage system typically is not content-aware; that is, not looking into object contents. Such inspection would need awareness of the object's data format which can't be taken for granted.
Think of a file system listing files in a folder. The files could be spreadsheets, images, documents, etc. Unless you have a copy of the creation software for such a file, plus access privileges, you can't access it and have no idea what's inside it. Similarly with objects.
Any general analytics looking into object contents would face the same problem. So the answer is to do it indirectly, via metadata about the objects. It needs users to input metadata about the object and its contents, thus providing the data for an index structure.
In order for such indices and other metadata to be consistent then object upload would need a formal process to generate the metadata. It all starts looking complicated but the object searchability would be the benefit. ®