37 Replies Latest reply: Jan 21, 2014 5:43 AM by radek.kubka RSS

Understanding aggregate and lun

rodrigo.passos
Currently Being Moderated

Hi

 

What is aggreggate and Lun concepts?

 

Thank you

  • Re: Understanding aggregate and lun
    ekashpureff
    Currently Being Moderated

    Rodrigo -

     

    An aggregate is the physical storage. It is made up of one or more raid groups of disks.

    You can see the Data ONTAP 'Storage Management Guide' for documentation.

     

    A LUN is a logical representaion of storage. It looks like a hard disk to the client. It looks like a file inside of a volume.

    You can see the Data ONTAP 'Block Access Management Guide for iSCSI and FC' for documentation.

     


    I hope this response has been helpful to you.

     

    At your service,

     


    Eugene E. Kashpureff
    ekashp@kashpureff.org
    NetApp Instructor and Independent Consultant
    http://www.linkedin.com/in/eugenekashpureff

     

    (P.S. I appreciate points for helpful or correct answers.)

  • Re: Understanding aggregate and lun
    ken.foster
    Currently Being Moderated

    i like to explain it like this.  its rough but pretty straight forward.

     

    Raid groups are protected sets of disks.  consising of 1 or 2 parity, and 1 or more data disks.  We don't build raid groups, they are built behind the scene when you build an aggregate.  for example:

     

    in a default configuration you are configured for RaidDP and a 16 disk raid group (assuming FC/SAS disks).  So, if i create a 16 disk aggregate i get 1 raid group.  if i create a 32 disk aggregate, i get 2 raid groups.  Raid groups can be adjusted in size.  For FC/SAS they can be anywhere from 3 to 28 disks, with 16 being the default.  you may be tempted to change the size so i have a quick/dirty summary of reasons.

     

    Larger RG                                              Smaller RG                     

    Better Space utilization                         worst space utilization                          (think of the ratio of parity to data disks)

    worst protection                                      better protection                                   (think of the ratio of parity to data disks)

    better performance                                worst performance                                (roughly more data disks gives you better performance)

    more risk                                                 less risk                                                  (smaller raid groups rebuild faster)

     

     

     

     

    Aggregates are collections of raid groups.  consist of one or more Raid Groups.

     

    I like to think of aggregates as a big hard drive.  there are a lot of similarities in this.  When you buy a hard drive you need partition it and format it before it can be used.  until then its basically raw space.  Well, thats an aggregate.  its just raw space.

     

     

    A volume is analogous to a partition.  its where you can put data.  think of the previous analogy.  an aggregate is the raw space (hard drive), the volume is the partition, its where you put the file system and data.  some other similarities include the ability to have multiple volumes per aggregate, just like you can have multiple partitions per hard drive.  and you can grow and shrink volumes, just like you can grow and shrink partitions.

     

     

    A qtree is analogous to a subdirectory.  lets continue the analogy.  aggregate is hard drive, volume is partition, and qtree is subdirectory.  why use them? to sort data.  the same reason you use them (at least i hope you use them) on your personal PC.  There are 5 things you can do with a qtree you can't do with a directory and thats why they aren't just called directories.  Oplocks, security style, Quotas, Snapvault, Qtree SnapMirror.

     

     

    last but not least, LUNs.  As Eugene mentioned its a logical representaion of space, off your SAN.  but the normal question is when do i use a LUN over a CIFS or NFS share/export.  i normally say it depends on the Application.  Some applications need local storage, they just can't seem to write data into a NAS (think CIFS or NFS) share.  Microsoft Exchange and SQL are this way.  they require local hard drives.  So the question is, how do we take this network storage and make it look like an internal hard drive.  the answer is a LUN.  it takes a bit of logical space out of the aggregate (actually just a big file sitting in a volume or qtree) and it gets mounted up on the windows box, looking like an internal hard drive.  the file system makes normal SCSI commands against it.  the SCSI commands get encapsulated in FCP or iSCSI and are sent across the network to the SAN hardware where its converted back into SCSI commands then reinterpreted as WAFL read/writes.

     

    Some applications know how to use a NAS for storage (think Oracle over NFS, or ESX with NFS datastores) and they don't need LUNs.  they just need access to shared space and they can store their data in it.

    • Re: Understanding aggregate and lun
      saranraj456
      Currently Being Moderated

      what is the use of aggregate?

      • Re: Understanding aggregate and lun
        ken.foster
        Currently Being Moderated

        Aggregates are the raw space in your storage system.  you take a bunch of individual disks and aggregate them together into aggregates.  But, an aggregate can't actually hold data, its just raw space.  you then layer on partitions, which in NetApp land are called volumes.  the volumes hold the data.

         

         

        you make aggregates for various reasons.  For example:

         

        performance boundaries - a disk can only be in one aggregate.  so each aggregate has its own discreet drives.  this lets us tune the performance of the aggregate by adding in however many spindles we need to achieve the type of performance we want.  This is kind of skewed by having Flash Cache cards and such, but its still roughly correct.

         

        Shared Space boundary - All volumes in an aggregate share the hard drives in that aggregate.  there is no way to prevent the volumes in an aggregate from mixing their data on the same drives.  i ran into a problem at one customer that, due to regulatory concerns, couldn't have data type A mixed with data type B.  the only way to achieve this is to have two aggregates.

         

         

        for volumes - you can't have a flexible volume without an aggregate.  Flex Vols are logical, Aggregates are physical.  you layer one or more flex vols on top (in side) of an aggregate

         

         

        hope this helps

        • Re: Understanding aggregate and lun
          saranraj456
          Currently Being Moderated

          Thank you fpr your response!!!

           

            I couldnt get that performance boundary point in that explanation.

            We can get that performance boundary  in raid group also know by selecting the no.of spindles right.?.

            Then what was the difference by doing it in aggregate?

            Kindly clarify.

          • Re: Understanding aggregate and lun
            ken.foster
            Currently Being Moderated

            An aggregate is made of Raid Groups.

             

            lets do a few examples using the command to make an aggregate  "aggr create aggr1 16".  if the default raid group size is 16, then the aggregate will have one raid group.  But, if i use the command "aggr create aggr1 32"  now i have two full raid groups, but still only one aggregate.  So, the aggregate gets the performance benefit of 2 RGs worth of disks.  notice we did NOT build a raid group.  Data ONTAP built the RG based on the default RG size.

             

            i explain this in more detail in a previous post in this thread.

    • Re: Understanding aggregate and lun
      kjen
      Currently Being Moderated

      Ken,

       

      Thanks for  the clear and concise explanation. I am about to carve out a SAN out of out NetApp setup (been running NAS all this time).

      I am currently deciding what is the better method (iSCSI vs FCP). The NetApps have just been installed with FCoE adapters that supports both.

      Can anyone shed some light on weighing between the 2? Also, are there any quick reference guides for setting up a FC or iSCSI SAN besides the Block Access Mgmt Guide? Please help...we need to get this done soon. All input appreciated.

       

      Steve

    • Re: Understanding aggregate and lun
      DINJMD735
      Currently Being Moderated

      Great explanation!..Thanks

  • Re: Understanding aggregate and lun
    saranraj456
    Currently Being Moderated

    what happen when we give a volume directly to the user instead of LUN?

     

    What was the difference between these two?

    • Re: Understanding aggregate and lun
      ken.foster
      Currently Being Moderated

      Volumes are access via NAS protocols, CIFS/NFS

       

      LUNS are accessed via SAN protocols, iSCSI/FCP/FCoE

       

       

      in the end you can put data in a LUN, you can put data in a Volume.  its how you get there thats the question.

       

       

      other than that not sure about the question.

      • Re: Understanding aggregate and lun
        saranraj456
        Currently Being Moderated

        is it possible to give a volume directly in a san environment?

        • Re: Understanding aggregate and lun
          ken.foster
          Currently Being Moderated

          no

          • Re: Understanding aggregate and lun
            saranraj456
            Currently Being Moderated

            Ken,

             

            aggregate-raw space

            volume-providing file system

            lun-?

            by this hierarchy what kind of function lun would provide?

            • Re: Understanding aggregate and lun
              ken.foster
              Currently Being Moderated

              luns are logical.  they go inside a volume, or in a qtree.

               

               

              from a netapp perspective they are really just one big file sitting inside of a volume or qtree.

               

              from a host perspective, they are like a volume, but use a different protocol to access them (purists will argue with that but i'm simplifying).  LUNs provide a file system, like Volumes provide a file system, the major difference is who controls the files system.  with a LUN the storage system can't see the file system, all it sees is one big file.  the host mounts the file system via one of the previously mentioned protocols and lays a file system down inside.  the host then controls that file system.

               

               

              I normally determine LUN/Volume usage by looking at the Application.  Some apps won't work across a network, Microsoft SQL and Exchange are two examples of this.  They require local disks.  LUNs look like local disks.  Some applications work just fine across the network, using NFS, like Oracle.  In the end its normally the application that will determine whether you get your filesystem access through a LUN or a Volume.

               

               

              some things like Oracle or VMware can use either LUNs or NFS volumes, so with them its whatever you find easier or cheaper.  I'm an NFS bigot so I tend to push that but in the end either works.

              • Re: Understanding aggregate and lun
                saranraj456
                Currently Being Moderated

                Ken,

                 

                   From netapp perspective,volume will provide filesystem.

                   here the file system you are talking about is host side filesystem or WAFL ?

                • Re: Understanding aggregate and lun
                  ken.foster
                  Currently Being Moderated

                  the underlying filesystem is always WAFL in the volume.

                   

                  when you share out a volume it looks like NTFS to a windows box, or it looks like a UNIX filesystem to a unix box but in the end its just WAFL in the volume.

                   

                  with a LUN its a bit different.

                   

                  you first make a volume, then you put a LUN in the volume.  the volume has WAFL as the file system, the LUN looks like one big file in the volume.

                   

                   

                  you then connect to the storage system using a SAN protocol.  the big file we call a LUN is attached to the host via the SAN protocol and looks like a big hard drive.  the host then formats the hard drive with NTFS or whatever File system the unix box is using.  That file system is actually NTFS, or whatever.  its inside the LUN, which is big file inside of a Volume, which has WAFL as its file system.

                   

                   

                  hope that makes sense.

                   

                  and also I have two other recommendations.  Netapp and Fastlane have a few free Flash self paced training videos you can watch.  Netapp also has a youtube channel that has some cool stuff on it.  and I will also suggest that you may want to take a few training classes.  Most of this thread is covered in the Data Ontap fundamentals class and the SAN classes.

                   

                  In the interest of Full Disclosure, I am a contract instructor for Fastlane.

  • Re: Understanding aggregate and lun
    DIVISLABS
    Currently Being Moderated

    Thank you all for elobarate information provided.

  • Re: Understanding aggregate and lun
    kjen
    Currently Being Moderated

    Ken,

     

    Thanks for  the clear and concise explanation. I am about to carve out a SAN out of out NetApp setup (been running NAS all this time).

    I am currently deciding what is the better method (iSCSI vs FCP). The NetApps have just been installed with FCoE adapters that supports both.

    Can anyone shed some light on weighing between the 2? Also, are there any quick reference guides for setting up a FC or iSCSI SAN besides the Block Access Mgmt Guide? Please help...we need to get this done soon. All input appreciated.

     

    Steve

  • Re: Understanding aggregate and lun
    LICOHEN83
    Currently Being Moderated

    HI Ken,

    Thanks very much for your very detailed, informative explanation.

    It was very helpful.

  • Re: Understanding aggregate and lun
    VANESSA.AREVALO
    Currently Being Moderated

    Hi Ken,

    Please help me understand more how LUN and volume works. Given this scenario; the volume and LUN size gives different results from each other. Volume reaches it's 100% size but on LUN it is only using 30% with same results from the virtual machine.

  • Understanding aggregate and lun
    EMPOWERISM
    Currently Being Moderated

    Thanks Radek,It is very usefull for me,

    I created Luns with Santricity but it is not visible in vmware and I have to restart my host to see LUNs,any solution or document?