There is an excellent roster of talks at the upcoming Storage Developers Conference in September. A quick preview of the CDMI-related sessions lined up include:
CDMI and Scale Out File System for Hadoop
Philippe Nicolas, Director of Product Strategy, Scality
Scality leverages its own file system for Hadoop and replaces HDFS while maintaining HDFS API. Scality Scale Out File System aka SOFS is a POSIX parallel file system based on a symmetric architecture. This implementation addresses the Name Node limitations both in term of availability and bottleneck with the absence of meta data server with SOFS. Scality leverages also CDMI and continues its effort to promote the standard as the key element for data access. Scality capitalizes on 2 data protection techniques - Replication and Erasure Coding with Scality ARC - to boost data access, improve data durability and reduce hardware footprint and costs.
- Realize the potential of CDMI
- Illustrate a new usage of CDMI
- Address Hadoop limitations
- Introduce method to improve data durability
Lessons Learned Implementing Cross-protocol Compatibility Layer
Scott Horan, Integration Engineer, Cleversafe, Inc.
Over the past year, we have integrated our storage solution with a number of cloud and object storage APIs, including Amazon S3, WebDAV, OpenStack, and HDFS. While these protocols share much commonality, they also differ in meaningful ways which complicates the design of a cross-protocol compatibility layer. In this presentation, we detail how the various storage protocols are the same, how they differ, and what design decisions were necessary to build an underlying storage API that meets the requirements to support all of them. Further, we consider the lessons learned and provide recommendations for developing cloud storage APIs such as CDMI.
- Features of the various popularly used cloud storage protocols
- Differences between the protocols, how they authenticate, how they handle multi-part objects, how they guarantee integrity
- Overview of the design decisions we made to support all these protocols on top of the same underlying storage
Profile Based Compliance Testing of CDMI: Approach, Challenges & Best Practices
Sachin Goswami, Solution Architect and Storage COE Head, Hi Tech TATA Consultancy Services
Cloud Data Management Interface Specifications are now moving towards profile based categories. This is due to the increased focus where organizations are planning to adopt profile based CDMI in their products, example Service, ID and Self Storage Management profiles. TCS has been focusing on implementing ‘CDMI Automated Test Suite’ and with new developments is focusing towards incorporating profiling to the same. In this proposal we will share the approach and challenges for testing of profile based scenarios towards CDMI profile based compliance, of the cloud products. Also, we will share additional challenges / learning towards testing of CDMI Products for compliance. These learning’s will serve as a best practice and a ready reference for other organizations in developing their own CDMI product suit.
- Understanding CDMI Profile based Specification.
- Understanding the Profile Based testing approach
- Understanding of existing gaps that are present in CDMI Profile based specification.
Open-source CDMI-compliant Proxy: "Stoxy"
Ilja Livenson, KTH Royal Institute of Technology
We present our on-going effort in development of a open-source server (Stoxy - STOrage proXY), which exposes CDMI-compliant interface on the frontend and allows to store and manage data on several public and private cloud backends, incl. AWS and MS Azure. The work is based on a CDMIProxy prototype developed in EU VENUS-C project with a continuos development effort sponsored by EGI Inspire project as well as commercial companies. Presentation will cover the architecture of Stoxy and highlight certain key components, esp. related to the data streaming. In addition, we shall give initial experience from integrating Stoxy with the other products (cloud managers, storage servers) inside the EGI Federated cloud task force.
- Introduction of the open-source product and its use cases
- Experience from implementing the streaming proxy server to public cloud backends
- Experience from integration of CDMI with a federated cloud test-bed
- Interoperability testing with OpenStack Swift
LTFS and CDMI - Tape for the Cloud
David Slik, Technical Director, NetApp
LTFS tape technology provides compelling economics for bulk cloud storage and transportation of data. This session provides an overview of the use cases identified by the joint LTFS and Cloud Technical Working Group, including when tape provides lower-cost alternatives to network and disk-based transportation, and when tape provides lower-cost alternatives to disk-based storage and archiving. This session will introduce standardization efforts underway to allow for simple tape-based bulk data transport to, from, and between clouds, and standardized of how to store rich object data on standard LTFS tapes.
- Learn about the advantages of tape for bulk data movement to, from and between clouds
- Learn about the advantages of tape for bulk data storage within clouds
- Learn about standardization activities around tape-based bulk data transport
- Learn about how CDMI objects can be stored on LTFS tapes
CDMI Federations, Year 4
David Slik, Technical Director, NetApp
In addition to standardizing client-to-cloud interactions, the SNIA Cloud Data Management Interface (CDMI) standard enables a powerful set of cloud-to-cloud interactions. Federations, being the mechanism by which CDMI clouds establish cloud-to-cloud relationships, provide a powerful multi-vendor and interoperable approach to peering, merging, splitting, migrating, delegating, sharing and exchange of stored objects. In last year's SDC presentation, bi-directional federation between two CDMI-enabled clouds was discussed and demonstrated. For year four, we will discuss how CDMI federation when combined with CDMI versioning, enables mobile and web-based applications to synchronize data with clouds, effectively allowing clients to create "mini-clouds" local to the client. This architectural approach allows clients to easily store, cache, and merge cloud-resident content, provide disconnected operation, and provides a foundation for application-specific conflict resolution.
- Learn how clients can use CDMI federation to synchronize local and cloud-resident data stores
- Learn how versioning and globally unique identifiers enables multiple concurrent writers without synchronization
- Learn techniques that enable automated conflict detection and application-specific conflict resolution
- See a multi-client demonstration how of CDMI Federation simplifies application data management
CDMI, The key component of Scality Open Cloud Access
Giorgio Regni, CTO, Scality
Announced during SNIA SDC 2012, Scality Open Cloud Access, aka OCA, was the first converged access methods between file and object mode, from local or remote site. Dewpoint, the Scality CDMI server, demonstrated during previous 3 CDMI annual plugfests during SDC, continues to be the pivotal component of this strategy. During last year, Scality leverage CDMI to build the Scality solution for Hadoop and plan to announce a few other innovations.
- Illustrate utilization of CDMI
- How to build convergent data access methods
COSBench: A benchmark tool for Cloud Storage
Yaguang Wang, Sr. Software Engineer, Intel
With object storage services becoming increasingly accepted as one new offering comparing to traditional file or block systems, it is important to effectively measure the performance of these services. Thus people can compare different solutions or tune their systems for better performance. However, little has been reported on this specific topic as yet. To address this problem, we developed COSBench (Cloud Object Storage Benchmark), a benchmark tool that we are currently working on inside Intel for cloud object storage services. In addition, we will share the status for CDMI supporting, and share results of the experiments we have performed so far.
- To know what's cosbench, and how does it work
- To know what kind of information can get from cosbench
- To know the key different points between CDMI and other object store interfaces like S3
We look forward to seeing everyone at SDC 2013, and at the co-hosted CDMI plugfest!