Currently Being Moderated

Part of - Learning NetApp - A storage admin's guide



This blog post has lots of links in it to enable you to develop your own knowledge.  The definitive guide is written by NetApp and is available to customers and partners via  Log in, select documentation from the banner, product documentation, {Operating System}, version number and then "Storage Management Guide".  Be sure to get the correct version for your system as the documentation is different between versions.  For example DoT 7.2 does not have 64bit aggregates and SIS sizes have changed between releases.


The information below is for those choosing to ease into the topic.


In NetApp storage, disks are grouped together into aggregates and these aggregates are formatted with the WAFL file system.  Aggregates are then logically divided up into usable units called Flexvols (flexible volumes), which can be further divided with Qtrees and or LUNs.


NetApp Objects





NetApp does not manufacture the HDDs used in its systems directly but works with several partners, such as Seagate, Hitachi and Samsung to name a few.  (Disk vs Disc)  These disks are manufactured to industry standards for physical size, interface (SATA, FC & SAS) and only the NetApp label and propitiatory disk candies indicate they are different.  Firmware is a key differentiation between generic HDDs and NetApp supported disks.  The more disks you have in your estate, the increased chance of having to update the firmware on the individual disks.  MyAutosupport customers will get alerts as to when disk firmware requires upgrading or you can manually check via the Disk Firmware page.  When you select the firmware required for download, it will open a link to detailed step-by-step instructions for the upgrade process.  Older systems may require downtime but disk firmware upgrades are normally non-disruptive but read the instructions carefully.


Disks are either 'spinning rust' HDD or solid-state devices SSD.  There are no moving parts in SSDs, resulting in orders of magnitude performance increases over HDD technology but they are expensive, have lower storage capacity and contain components which wear out. SSDs also suffer from problems like write amplification however, all these issues can be managed with software or clever design.  The technology used in HDDs is taken for granted but imagine what is required to read a news paper on the ground as you pass over it at 100 mph!  The Future of the disk drive.  As a NetApp admin, you will not be required to know how a Hdd works but as an engineer you should be interested enough to google it.  Beware the rabbit hole goes deep...


Back in the real world, most NetApp HDDs come in two flavors "cheap and deep" or fast.  Slow (but with large storage capacity) being a disk with a rotational speed of less than 7,500 RPM and fast disk being greater, typically 10,000 (10k) rpm or 15,000 (15k) rpm.  The spin speed is important as it directly relates to the average seek time for the head.  Performance of the individual disks is enhanced by cache technology in the filer and DoT operating system.  Plus purchased enhancements Flash cache & Flash pools if required.


Disk performance can be simplified to a few very rough rules of thumb (When you learn more, you will understand why these rules are 'flexible')

  • Sequentail work loads are significantly faster than random
  • A single slow SATA disk delivers 50 random IOPs
  • A single fast FC or SAS disk delivers 250 random IOPs
  • Performance scales with disks, ie 10 SATA disks deliver 500 IOPs


Disk capacity is an industry wide issue with marketing teams describing the disk 'raw' capacity when the actual usable capacity is always less.  There are many technical reasons why the RAW and usable sizes are different but NetApp's Right Sizing is something you will need to know about.  Right sizing is the process of making sure that same raw size disks from different manufactures work together in aggregates by rounding down the capacity, ensuring the addressable space is the same.


     Top tip - Don't annoy your account manager by pointing out the price difference between a single NetApp disk based on the cost of a shelf divided the number of disks contained, to what you could purchase the "same" generic disk for locally.  Instead focus on the total system cost compared to other similar enterprise storage arrays and the value they represent to the business, when thinking about the large numbers on the PO form.



NetApp Product Documentation - Disk Shelves


Disks are mounted either internally with smaller systems or external via disk shelves.  Shelves connect either as a loop, of up to 6 shelves or in a stack of up to 10 shelves per stack.  Loops are an older method of connecting disks based on 1, 2 or 4 Gb Arbitrated loop, also known as FC-AL and require an FC TARGET port on the filer.  Ports on the system board can be changed (requires filer reboot) between target and initiator modes of operation but ports on HBAs tend to be fixed, so be sure to know what types you have (fcadmin command) when thinking about expanding your system.  Stacks are based on SAS 3 or 6 Gb technology and supports a much higher number of disks.


In a system with just a single filer head and only single path cabling, disk identification is simple but becomes more 'complex' when HA pairs and multipath cables are used.  However the 1st time you are tasked with replacing a failed disk on your own, you will be grateful for both learning how the disk ID system works and anything you can do to make it simpler.  Like clearly labeling shelves and cables, logically arranging kit in the data centre and numbering each stack by powers of 10. 


The NetApp documentation is where you need to look for further information.

    • Hardware Universe - (Improved version of "System Configuration Guides")
    • Universal SAS and ACP Cabling Guide
    • FC Hardware Service Guide
    • AT Hardware Service Guide
    • Multipath and Dual-Path Cabling Information
    • Disk Shelf Installation and Setup


Use filerview, system manager & CLI for the following

      • How many disks are in your system
      • What type of disk are they, interface | size | speed | manufacture | firmware version
      • Is the system cabled for multipath
      • How many unowned disk are in the system
      • How do you change the disk ownership in a HA system
      • Reduce the spare disks in the system to just 1, what happens to the NetApp disk maintenance sub system (if it is enabled)
      • Remove all the spare disks - Turn off the alerts created



An aggregate is created from a minimum of 2 disks when RAID-4 is used or 3 with RAID-DP, however the more disks in an aggregate, the better it will perform. 


These disks must be of a common type SATA, FC or SAS but the rules say it is possible to mix in some combinations (see storage guide), I however always stay away from this type of thing.  Mainly because the system will operate at the lowest common speed within the aggregate and support engineers will always suspect these types of configurations when trouble shooting weird problems.  "Late for work, that is because you have SAS & FC 600Gb disks in your aggregate..."


If your estate is pre 2006 (ish) you may need to know about 'NetApp's Traditional volumes' -see storage management guide- but I have not come across this type of configuration in production and will skip past them.  Therefore aggregates come in either 32 or 64 bit versions, with the later being a feature of newer DoT versions and having a higher capacity.  With DoT 8.1, it became possible to upgrade from 32 to 64 bit but be sure to read the small print.  Depending on which version of DoT your system has and who created the aggregates, you may have an aggregate snap reservation of 5% and hourly aggregate snapshots.  Personally I set the aggregate snap reverse to zero and use the released storage capacity for production.  This is because an aggregate restore is a slow process and resets EVERYTHING in the aggregate, so I protect critical data with snapmirror.  It is possible to mirror data at the aggregate level using sync mirror and start talking about metro cluster or fabric metro cluster options but that is beyond the scope of this guide.


When working with aggregates it is simple to both create them and then later grow them by adding disks but these are one way processes, so always take the time to think about the consequences of your actions. The classic 'n00b' mistake is to grow an aggregate by a single disk to solve a capacity issue, which then results in a performance problem.  At a minimum grow aggregates by 3 disks but always try and add a full RAID Group worth of disks.  Aggregates can be renamed easily and over the years I have grown to like the naming standards which include the disks which the aggregate is made from.  ie  sata1000_aggr1


Use filerview, system manager & CLI for the following

      • Create a 2 disk RAID-4 aggregate, with a RAID group size of 6
      • Fail the data drive in the aggregate, notice the status, replace the failed disk
      • Add a single drive to the aggregate
      • Convert the aggregate to RAID-DP
      • Fail the data drive in the aggregate, notice the status, replace the failed disk
      • Rename the aggregate
      • Create two flexvols in aggregate and snapshot aggregate
      • Create two move flexvols called "Fired if lost" & "Need to know"
      • Perform an aggregate restore and review volumes available



RAID Groups

The primary method of protection against disk failure in NetApp storage is RAID (redundant array of independent disks, originally redundant array of inexpensive disks) and RAID-DP is what you should be using.  Engineers talk about RAID 4 but I would not spec it in my designs...  Jay & Carlos have already documented everything you need to know on this great blog post.


Remember - NetApp will shutdown after 24 hours if degraded raid group is not repaired


Or to put it another way

Ken Foster has written a very clear post here -


Back to index


Filter Blog

By date:
By tag: