The Data Storage market is complicated, hard to describe and ever changing. Good for me, otherwise I probably wouldn’t have a job. But not so good if you’re looking to spend an ever increasing amount of money on it, as data growth continues to escalate. A lot is to do with the maturity of the market – at 15-20 years (for disk based systems), much younger than other major IT segments like Servers or Networking. There are also many approaches & combinations to solve your storage problem with typically no one 'right answer'. And no single company dominates the storage / data management market. Even for those of us who work in it every day, it’s very difficult to accurately describe, segment & measure. As a result, people tend to look at the lowest common denominator – hardware*. Despite the fact that 95% of the business value in a storage system is in the software it runs.
From a hardware point of view, there have only really been 2 eras in Storage. The Tape era, which started around 4000BC and ended in 1995. And the Disk era, which we’ve been in ever since. The market dynamics have been pretty stable, with the disk storage specialist companies - EMC, HDS, NetApp - innovating & gaining market share and the broader systems companies – IBM, HP, Sun/Oracle, Dell – losing market share, some making aquisitions to try to compensate. So imagine the frenzy over the past couple of years as things started to change & we start to enter a 3rd storage hardware era - the Flash era. For background on why this is happening, I’d recommend reading Laurence James’ post here: Jumping Jack Flash is a Gas Gas Gas .
(Further background: one of the the best records of storage market dynamics is kept by Jean-Jacques Maleval, at Storage Newsletter: http://www.storagenewsletter.com/news/startups/storage-start-ups-in-2012)
Flash era: architecture options for your Data Centre
There are two main choices to make when thinking about Flash hardware in your data centre – 1) where to place the Flash in your architecture for maximum impact – near the server, in the network or within the storage array? And 2) Do you need an All-Flash Array, or Hybrid Storage Array that combines both Flash and Disk technology? Which is best for you today depends on many factors. Broadly, if you need ultimate performance, then go for a dedicated an all-flash array. For the absolute lowest latency, use server flash. If not, and cost efficiency and flexibility are more important, then a hybrid array approach probably makes most sense - as part of an agile data infrastructure strategy.
In the Market for some Flash hardware?
It’s no surprise that all the major Storage players have built a Flash era strategy – even HP announced one recently! EMC acquired XtremIO and will likely bring a product to market this year. IBM acquired TMS. HDS recently validated the hybrid approach. There are also many Flash array start-ups trying to take advantage of the change – Violin, Nimbus, Nimble, Pure, Whiptail and about 47 more. Given the major storage players have an announced Flash strategy, I would expect many of these to disappear from the market in 2013. Beware of ever lower & unsustainable prices and inflated performance claims as a result.
#NetAppFlash – a software company in a hardware world
NetApp has been shipping ‘hybrid storage arrays’ running Data ONTAP to customers since 2009 – in combination with our Virtual Storage Tiering (VST) approach. I first wrote about the success of this strategy in 2011: Flash & your Cloud Storage Strategy. We have now shipped more than 36,000TB, accelerating 3,000,000TB of disk storage - way more than any other vendor, generating a huge amount of experience, and important economies of scale. And in August last year we announced Flash Accel & a partnership strategy for server caching – see Tim Waldron’s blog post here: Flash: Changing the calculus of cloud.
This morning I am at NetApp’s HQ in Sunnyvale. We have invited journalists from around the world to hear a series of new all-Flash array strategy and product announcements. For both dedicated, high performance environments, and shared infrastructure. For full details see here later. And remember, as we continue to move into the Flash era - the storage media may be evolving, but 95% of the business value & innovation in a storage system is in the data management software it runs.
*The last 5-10 years we have also used storage networking protocol as an artificial way of segmentation, but with Unified Storage now standard, that is now an operational decision, nothing more. Same applies for Flash arrays.