21 Replies Latest reply: Sep 28, 2011 9:05 AM by shawnodell RSS

Automated Storage Tiering and the NetApp Virtual Storage Tier

feresten NetApp Employee Sprinter
Currently Being Moderated

Automated storage tiering (AST) helps data centers benefit from the improved performance of flash-based media while minimizing the cost and complexity by preferentially storing "hot" data on Flash. There are two basic approaches to AST: migration-based AST which essentially automates the process of data migration and caching-based AST which uses well understood caching methods to promote hot data to high performance storage. NetApp strongly believes that the caching-based method is more efficient and will perform better. The NetApp Virtual Storage Tier is NetApp's implementation of caching-based AST. Rajesh Sundaram and I recently wrote an article on AST and the NetApp virtual storage tier.

You can check out the article at http://communities.netapp.com/docs/DOC-10322


Is automated storage tiering on your roadmap? Does VST meet your needs? Why or why not?


Message was edited by: Paul Feresten

  • Re: Automated Storage Tiering and the NetApp Virtual Storage Tier
    rmharwood Cyclist
    Currently Being Moderated



    Nice article, thank you.


    There is one thing that has been nagging at the back of my mind about this. Why did Netapp opt to use flash memory on a PCIe card as cache rather than just architect a system board with more RAM? Obviously flash is cheaper but RAM is yet quicker. I figure there must be some engineering reason behind it.




    • Re: Automated Storage Tiering and the NetApp Virtual Storage Tier
      feresten NetApp Employee Sprinter
      Currently Being Moderated

      Hi Richard,


      Flash is much more cost effective and is also non-volatile. However it's worth pointing out that our Virtual Storage Tier actually involves two cache levels-one of which is DRAM. DRAM is used for buffer cache which is the first level cache and when data is evicted from there it is copied to Flash. So, we do actually leverave RAM but we are able to absorb a higher percentage of high performance IOPS in Flash.



  • Re: Automated Storage Tiering and the NetApp Virtual Storage Tier
    guillermoleira Novice
    Currently Being Moderated



    The article says that the implementation of AST (FAST) is complex, and its performance is not optimal. What I have seen is that the combination of FAST Cache for spikes, and FAST VT for moving less used data to SATA and hot data to FLASH seems to work fine, and without any configuration problems.

    Even if NetApp approach works best (I don't know of any benchmarks comparing both technologies), FAST is very good for sales presentations, as it is very easy to understand, and intuitive. And it will have another advantage very soon, as it seems that it is going to be available in the entry level systems. PAM cards can't be used on 20xx systems.

    I think that NetApp needs a better answer, and needs it soon. Almost every other characteristic of the competetitors can be discussed, but we are having a hard time against FAST.


    Best Regards


    Guillermo Leira

  • Automated Storage Tiering and the NetApp Virtual Storage Tier
    louw.pretorius Novice
    Currently Being Moderated

    After reading this document I was thinking to myself: "Why not add a Shelf of SSD's and use that as another Tier?" - this would make ALOT of sense as we cannot afford a PAM (fastcache) but we might be able to get a shelf with a few SSD's on the budget and the performance could then be increased easily by just adding more SSD's to the aggr/cache tier.


    I hope NetApp is working on something like this as the jump to PAM is just too big for us SME's




  • Automated Storage Tiering and the NetApp Virtual Storage Tier
    jkim NetApp Employee Novice
    Currently Being Moderated

    I've been following HSM/ILM/AST for many years (though not as far back as the roots of mainframe HSM) and the number of customers who think it sounds good has always been a lot higher than the number who actually try it, which itself is a lot higher than the number who actually made it work well enough to justify the cost. The biggest advantage of NetApp Flash Cache/VST is that anyone can implement it easily and nearly everyone benefits from it. That doesn't mean our VST approach is the best for everyone, but I think for >80% of customers it's either the best or at least as good as the alternatives. My estimate of the customer breakdown for open systems network storage customers is like this:


    • Customers interested in AST as a concept: >90%
    • Customers who would actually benefit today from some kind of AST: ~85%
    • Customers who could significantly benefit from NetApp Flash Cache/VST: ~80%
    • Customers who would benefit more from traditional HSM (like EMC ILM/FAST) over NetApp Flash Cache/VST: ~5%
    • Customers who need AST but would not really benefit from either NetApp VST or EMC FAST: ~5%


    So our challenge is that even though 85% of customers would see significant benefits from NetApp VST, there are the approximately 5% that would be better off using traditional style HSM and another 5% who need AST but for whom neither NetApp-style VST or EMC-style FAST is the right approach. It is likely those 10% of customers will be on the bigger side with bigger-than-average data. Another issue is that even though the vast majority of customers who need AST would do just as well or better with NetApp VST than with EMC FAST, most won't know this until they've talked to a peer reference or tested both options themselves.

    • Automated Storage Tiering and the NetApp Virtual Storage Tier
      radek.kubka Hall of Fame F1 racer
      Currently Being Moderated



      As much as I value your detailed dissection of the problem, I still think you are missing the trick (& so does NetApp as a company):


      'Proper' AST (like EMC FAST and/or Compellent implementation) is winning customers' minds - this is to a large extent a marketing, not a technical discussion / issue. Customers do simply believe AST will allow them to address their storage needs with very little upfront planning & no ongoing maintenance. The fact that quite often this belief is wrong is of a secondary importance for their purchasing decision.




      • Automated Storage Tiering and the NetApp Virtual Storage Tier
        jkim NetApp Employee Novice
        Currently Being Moderated

        Radek, I largely agree with you. I should have added to my list above the following

        • Customers who think EMC FAST (or Compellent AST) will really help them, based on marketing material: ~90%
        • Customers who would actually benefit from EMC FAST or Compellent AST: ~80%
        • Customers who would get same or greater benefit (at lower cost) from NetApp VST: ~70%
        • Customers who realize at beginning that NetApp VST is the better choice for their needs: ~15%


        So by my estimates, out of 100 typical customers, 80 might benefit from the EMC/Compellent type AST but only 10 of those would do better with this than if they went with the NetApp VST/Flash Cache solution. But out of the 70 who would do better (same or better performance and cost benefits with simpler implementation) with NetApp VST, only 15 will initially see that. The rest need to be educated with a demo, whiteboard session, eval, or actual implementation. Since most customers are not willing to do a POC with both EMC FAST and NetApp VST, in many cases the decision mgiht be based on which vendor has better marketing.

    • Automated Storage Tiering and the NetApp Virtual Storage Tier
      aaronh NetApp Employee Cyclist
      Currently Being Moderated

      I would like to address the peer reference point with two very public NetApp customers, Suncorp and CERN. Both have had great success using NetApp VST on Oracle database workloads.


      1. CERN quote from the May 11 Edition of Storage Magazine http://go.techtarget.com/r/13941018/5390492/2

      CERN, the European Organization for Nuclear Research in Geneva, uses

      NetApp’s Flash Cache on Oracle RAC databases. “Prior to using Flash Cache,

      we had to size everything based on IOPS regardless of storage utilization,”

      said Eric Grancher of the CERN IT department. “Now, we can optimize both

      IOPS and capacity. We have moved from expensive Fibre Channel drives to

      less-expensive SATA drives. This has resulted in a substantial savings for the

      organization.” Grancher has found the NetApp system to be very adaptive to

      workloads resulting in simple management. His experience has determined

      that overall performance is better when the flash memory is in the storage

      rather than in the servers. “It makes more sense to have the stable NetApp

      systems cache the data rather than the database servers, which are restarted

      more frequently for patching or updates. A data cache on the storage server is

      already ‘warmed up’ and so eliminates the inevitable periods of poor performance

      we would suffer with cold server-based caches after each restart,”

      he said.


      2. Find the Suncorp Epic story here; http://www.netapp.com/au/campaigns/builton/ and here; http://communities.netapp.com/docs/DOC-9947

      On top of their already well-known success with Private Cloud, the Suncorp team applied Flashcache to accelerate processing times in a key Oracle environment. As I understand it, many other business applications are keen to climb on board to get the same business acceleration. Hopefully we can get one of the Suncorp team to share their experience in this thread.

  • Automated Storage Tiering and the NetApp Virtual Storage Tier
    Rodrigo Nascimento NetApp Employee Certified Cyclist
    Currently Being Moderated

    Hi Paul,


    Very nice article!

    I'm trying to develop a presentation (in portuguese) to show VST advantages over others similar solutions in the market.

    Doing my researchs, what I can find out is:

    - Automated Storage Tiering is a piece of something called Information Life-cycle Management;

    - When a customer listen "Automated Storage Tiering", he is realize that a piece of data will be moved from one disk tier to another based on access (when hot: go to perfromance disks; when cold: go to cheaper disks);  <=== IMHO, today is very hard to fight against it.


    IMHO, the VST approach is the most visionary because in a few years, the Flash technologies will be cheaper and we'll have two types of media to store data: one for data that needs performance (Flash & SSDs) and another for data that needs capacity (SATA[or its evolutions]). Because of this, I believe that Automated Storage Tiering is a solution with a date to die. BUT, until this day comes, we have a very hard fight against FAST VP and Compellent approach.


    The customer always says: "Ok! You "promote" the data instantly, but when this data comes "cold" I wanna move it (automatically) to the most cheaper disk tier that I have available."


    How can we fight against it?





    NetApp - Enjoy it!


    PS: Sorry for my poor english, it is not my native language.

    • Re: Automated Storage Tiering and the NetApp Virtual Storage Tier
      leejohn2008 Novice
      Currently Being Moderated

      I am out of the office until 06/23/2011.


      I will be on business trip to Beijing from June 19, 2011 to June 23,2011.

      I may not be able to reply your mail on time.

      For pSeries problem, please contact Eric Cheung.

      For NAS problem, please contact Trever Ip.



      Note: This is an automated response to your message  ""Automated Storage

      Tiering and the NetApp Virtual Storage Tier" [NetApp Community > Tech

      OnTap]" sent on 6/22/11 1:18:36.


      This is the only notification you will receive while this person is away.

  • Automated Storage Tiering and the NetApp Virtual Storage Tier
    leongkoklow Sprinter
    Currently Being Moderated



    Exciting to hear NetApp has the Automated Tiering capabilities. But, what I am looking for is Tiering the storage from different congtroller, let say I have 1 pair FAS3240 of SAS and another FAS3170 with all SATA, Can this be archiveable as I try to serach for any console but failed. Even , in the TR I could not find any.

    FYI, I have played with IBM Smart Tiering which come with a GUI that you can import the storage from different vendor.

    Is netapp developing something similiar and not yet announce!




More Like This

  • Retrieving data ...


  • Correct Answers - 10 points
  • Helpful Answers - 5 points