8 Replies Latest reply: Oct 10, 2012 4:39 PM by adaikkap RSS

Snap Mirror Compression  and OnCommand 5

rdenyer001
Currently Being Moderated

Does anyone know if its possible to enable Snap Mirror Compression  with  OnCommand 5 ?

I have found a document TR-3790  "SnapMirror Network Compression"   which explains how to use compression with DFM 3.8 but it relies  on  the /etc/snapmirror.conf  file  and the compression=enable  option.

 

Since I have upgraded to OnComand 5.0  my  /etc/snapmirror.conf files seem to have been purged of any  snapmirror relationships that are setup and managed via OnCommand.

  • Re: Snap Mirror Compression  and OnCommand 5
    Currently Being Moderated

    I'm not aware of any changes in OC5 that would cause it to purge snapmirror.conf, but perhaps something changed that I'm not aware of.

     

    If you enable SnapMirror compression by editing snapmirror.conf, those changes can be erased if you use the hostPreferredAddr[12] options. This is a known problem and should have been mentioned in the TR.

  • Re: Snap Mirror Compression  and OnCommand 5
    adaikkap
    Currently Being Moderated

    Hi Richard,

    Can you get the output of the following command for your source and destination filers of snapmirror ?

     

    dfm host get -q <filerid/name>

     

    Regards

    adai

    • Re: Snap Mirror Compression  and OnCommand 5
      rdenyer001
      Currently Being Moderated

      Here  is the output from dfm host get -q <filerid/name>

      Destination

      C:\Windows\system32>dfm host get -q HOST1
      host=HOST1.dc.local
      hostLogin=XXXX
      hostPassword=********
      hostLoginProtocol=ssh
      hostPreferredAddr1=
      hostPreferredAddr2=
      hostPrimaryAddress=XX.X.X.XX
      hostRLMAddress=
      hostConsoleAddress=
      cpuTooBusyThreshold=95
      cpuBusyThresholdInterval=00:15:00
      hostPingMethod=all
      hostAdminTransport=http
      hostNdmpLogin=XXXX
      hostNdmpPassword=********
      hostAdminPort=80
      perfAdvisorTransport=httpOnly
      prefsnmpVersion=1
      useHostsEquiv=No
      maxActiveDataTransfers=160
      perfDataExportEnabled=No
      defaultvFilerInterface=
      processHostIP=
      autoClientStatEnabled=No
      hostClockSkewedThreshold=1 minute
      hostClockNearlySkewedThreshold=30 seconds

       

      Source
      C:\Windows\system32>dfm host get -q HOST2
      host=HOST2.dc.local
      hostLogin=XXXX
      hostPassword=********
      hostLoginProtocol=ssh
      hostPreferredAddr1=
      hostPreferredAddr2=
      hostPrimaryAddress=XX.X.X.XX
      hostRLMAddress=XX.X.X.XX
      hostConsoleAddress=
      cpuTooBusyThreshold=95
      cpuBusyThresholdInterval=00:15:00
      hostPingMethod=all
      hostAdminTransport=http
      hostNdmpLogin=XXXX
      hostNdmpPassword=********
      hostAdminPort=80
      perfAdvisorTransport=httpOnly
      prefsnmpVersion=1
      useHostsEquiv=No
      maxActiveDataTransfers=160
      perfDataExportEnabled=No
      defaultvFilerInterface=
      processHostIP=
      autoClientStatEnabled=No
      hostClockSkewedThreshold=1 minute
      hostClockNearlySkewedThreshold=30 seconds

      • Re: Snap Mirror Compression  and OnCommand 5
        adaikkap
        Currently Being Moderated

        Hi Richard,

                     I created a compression enabled relationship on the filer and imported the same into a dataset in DFM 4.0.2D1.

        C:\>dfm version

        dfbm.exe         4.0.2.7809 (4.0.2D1)

        dfdrm.exe        4.0.2.7809 (4.0.2D1)

        dfpm.exe         4.0.2.7809 (4.0.2D1)

        dfm.exe          4.0.2.7809 (4.0.2D1)

        dfmcheck.exe     4.0.2.7809 (4.0.2D1)

        dfmconfig.exe    4.0.2.7809 (4.0.2D1)

        dfmconsole.exe   4.0.2.7809 (4.0.2D1)

        dfmmonitor.exe   4.0.2.7809 (4.0.2D1)

        dfmperf.exe      4.0.2.7809 (4.0.2D1)

        dfmscheduler.exe 4.0.2.7809 (4.0.2D1)

        dfmserver.exe    4.0.2.7809 (4.0.2D1)

        dfmwatchdog.exe  4.0.2.7809 (4.0.2D1)

        eventd.exe       4.0.2.7809 (4.0.2D1)

        grapher.exe      4.0.2.7809 (4.0.2D1)

         

        C:\>dfm controller list

        ID   Type                     Host Name                           Host Address                              ProductId    Deleted

        ---- ------------------------ ----------------------------------- ----------------------------------------- ------------ --------

          90 Active/Active Controller fas-sim-1                           192.168.98.10                             0099931871   No

          91 Active/Active Controller fas-sim-2                           192.168.98.11                             0099931872   No

         

        C:\>dfm host get -q 90

        host=fas-sim-1

        hostLogin=

        hostPassword=********

        hostLoginProtocol=rsh

        hostPreferredAddr1=

        hostPreferredAddr2=

        hostPrimaryAddress=192.168.98.10

        hostRLMAddress=

        hostConsoleAddress=

        cpuTooBusyThreshold=95

        cpuBusyThresholdInterval=00:15:00

        hostPingMethod=echo_snmp

        hostAdminTransport=http

        hostNdmpLogin=

        hostNdmpPassword=********

        hostAdminPort=80

        perfAdvisorTransport=httpOnly

        prefsnmpVersion=1

        useHostsEquiv=No

        maxActiveDataTransfers=

        perfDataExportEnabled=No

        defaultvFilerInterface=

        processHostIP=

        autoClientStatEnabled=No

         

         

        C:\>dfm host get -q 91

        host=fas-sim-2

        hostLogin=

        hostPassword=********

        hostLoginProtocol=rsh

        hostPreferredAddr1=

        hostPreferredAddr2=

        hostPrimaryAddress=192.168.98.11

        hostRLMAddress=

        hostConsoleAddress=

        cpuTooBusyThreshold=95

        cpuBusyThresholdInterval=00:15:00

        hostPingMethod=echo_snmp

        hostAdminTransport=http

        hostNdmpLogin=

        hostNdmpPassword=********

        hostAdminPort=80

        perfAdvisorTransport=httpOnly

        prefsnmpVersion=1

        useHostsEquiv=No

        maxActiveDataTransfers=

        perfDataExportEnabled=No

        defaultvFilerInterface=

        processHostIP=

        autoClientStatEnabled=No

         

         

        C:\>

         

        Ran Multiple updates and found the conf entry to be intact.

         

        Now upgraded my dfm server to 5.0 and ran multiple update and still find my conf entry to be intact in snapmirror.conf.

         

         

        C:\>dfpm dataset list -x compression

        Id:                              392

        Name:                            compression

        Protection Policy:               Mirror

        Application Policy:

        Description:

        Owner:

        Contact:

        Volume Qtree Name Prefix:

        Snapshot Name Format:

        Primary Volume Name Format:

        Secondary Volume Name Format:

        Secondary Qtree Name Format:

        DR Capable:                      No

        Requires Non Disruptive Restore: No

         

        Node details:

         

           Node Name:           Primary data

           Resource Pools:

           Provisioning Policy:

           Time Zone:

           DR Capable:          No

           vFiler:

         

           Node Name:           Mirror

           Resource Pools:

           Provisioning Policy:

           Time Zone:

           DR Capable:          No

           vFiler:

         

         

        C:\>dfpm dataset list -m compression

        Id         Node Name            Dataset Id Dataset Name         Member Type                                        Name

        ---------- -------------------- ---------- -------------------- -------------------------------------------------- -------------------------------------------------------

               383 Primary data                392 compression          volume                                             fas-sim-1:/comp_vsm_src

               386 Mirror                      392 compression          volume                                             fas-sim-2:/comp_vsm_dst

         

        C:\>dfpm dataset list -R compression

        Id         Name                        Protection Policy           Provisioning Policy Relationship Id State        Status  Hours Source                       Destination

        ---------- --------------------------- --------------------------- ------------------- --------------- ------------ ------- ----- ---------------------------- ----------------------------

               392 compression                 Mirror                                                      389 snapmirrored idle    0.0   fas-sim-1:/comp_vsm_src      fas-sim-2:/comp_vsm_dst

         

        C:\>dfpm backup list compression

        Backup Id Backup Version        Retention Type Retention Duration (in seconds)   Node Name            Description                         Properties(Name=Value)

        --------- --------------------- -------------- --------------------------------- -------------------- ----------------------------------- -------------------------

               12 22 Dec 2011 00:42:08  unlimited                                        Mirror

               10 22 Dec 2011 00:42:08  unlimited                                        Primary data

               11 22 Dec 2011 00:41:54  monthly                                          Mirror

                9 22 Dec 2011 00:41:54  monthly                                          Primary data

                8 22 Dec 2011 00:19:35  unlimited                                        Mirror

                7 22 Dec 2011 00:19:21  weekly         3024000                           Mirror

                5 22 Dec 2011 00:19:21  weekly         3024000                           Primary data

                3 22 Dec 2011 00:16:51  daily          604800                            Mirror               test

                1 22 Dec 2011 00:16:51  daily          604800                            Primary data         test

         

        C:\>dfm run cmd fas-sim-2 " rdfile /etc/snapmirror.conf"

        Job: 1

        Host: fas-sim-2

        Command:  rdfile /etc/snapmirror.conf

        Status: done

        Submitted: Thu Dec 22 00:45:28 2011

        Attempts: 1

        Login Protocol: rsh

        Use RLM Port: No

        Started: Thu Dec 22 00:45:28 2011

        Completed: Thu Dec 22 00:45:29 2011

        Stdout: #Regenerated by registry Tue Sep 27 08:58:29 GMT 2011

        Stdout: fas-sim-1:/vol/qsm_src/qsm_qt1 fas-sim-2:/vol/qsm_dst/qsm_qt1 restart=never 0-59/3 * * *

        Stdout: fas-sim-1:/vol/qsm_src/qsm_qt2 fas-sim-2:/vol/qsm_dst/qsm_qt2 restart=never 0-59/4 * * *

        Stdout: fas-sim-1:/vol/qsm_src/- fas-sim-2:/vol/qsm_dst/qsm_dst_non_qtree_data - 0-59/3 * * *

        Stdout: fas-sim-1:vsm_src fas-sim-2:vsm_dst restart=never * 0-23 * *

        Stdout: comp_con=multi(192.168.98.12,192.168.98.13)

        Stdout: comp_con:comp_vsm_src fas-sim-2:comp_vsm_dst compression=enable - - - - -

         

        C:\>

         

         

        C:\>dfm version

        dfbm.exe         5.0.0.7636 (5.0)

        dfdrm.exe        5.0.0.7636 (5.0)

        dfpm.exe         5.0.0.7636 (5.0)

        dfm.exe          5.0.0.7636 (5.0)

        dfmcheck.exe     5.0.0.7636 (5.0)

        dfmconfig.exe    5.0.0.7636 (5.0)

        dfmconsole.exe   5.0.0.7636 (5.0)

        dfmmonitor.exe   5.0.0.7636 (5.0)

        dfmperf.exe      5.0.0.7636 (5.0)

        dfmscheduler.exe 5.0.0.7636 (5.0)

        dfmserver.exe    5.0.0.7636 (5.0)

        dfmwatchdog.exe  5.0.0.7636 (5.0)

        eventd.exe       5.0.0.7636 (5.0)

        grapher.exe      5.0.0.7636 (5.0)

         

        C:\>

         

        Can you provide the steps to reproduce  ?

         

        Regards

        adai

        fas-sim-2> rdfile /etc/snapmirror.conf

        #Regenerated by registry Tue Sep 27 08:58:29 GMT 2011

        fas-sim-1:/vol/qsm_src/qsm_qt1 fas-sim-2:/vol/qsm_dst/qsm_qt1 restart=never 0-59/3 * * *

        fas-sim-1:/vol/qsm_src/qsm_qt2 fas-sim-2:/vol/qsm_dst/qsm_qt2 restart=never 0-59/4 * * *

        fas-sim-1:/vol/qsm_src/- fas-sim-2:/vol/qsm_dst/qsm_dst_non_qtree_data - 0-59/3 * * *

        fas-sim-1:vsm_src fas-sim-2:vsm_dst restart=never * 0-23 * *

        comp_con=multi(192.168.98.12,192.168.98.13)

        comp_con:comp_vsm_src fas-sim-2:comp_vsm_dst compression=enable - - - - -

        fas-sim-2>

        • Re: Snap Mirror Compression  and OnCommand 5
          rdenyer001
          Currently Being Moderated

          Thanks Adai,

          Most of my Snapmirror relationships have been created directly from the "NetApp Management Console"  and dont have  entries in the snapmirror.conf file.

          So I guess there are two options 1. wait until the NetApp management tools catch-up with ALL the features availible on a NetApp system or  2. Edit my snapmirror.conf  files  to add the compression option

           

          Regards,

          Richard

          • Re: Snap Mirror Compression  and OnCommand 5
            adaikkap
            Currently Being Moderated

            Hi Richard,

                 I think there is some misunderstanding. Let me try to clarify some.

            1. PM does not support creation of compression enabled snapmirror relationships.But it interoperates
            2. PM does not make a snapmirror.conf entry for all relationship its creates, except for those that are created using preferred interface and uses connections.

             

            Case 1:Where PM creates the relationships:

            ================================

            So if you create a normal QSM or VSM relationship using PM it neither makes a  snapmirror.conf entry nor schedules in the same as below.

            fas-sim-1:/vol/qsm_src/- fas-sim-2:/vol/qsm_dst/qsm_dst_non_qtree_data - 0-59/3 * * *

             

            Instead the schedule are driven by DFM server scheduler service which does the scheduling as per the policy you use in PM.

             

            If you have the option hostPreferredAddr1/2 set in DFM for the source and destination filer of Snapmirror.And create a VSM or QSM relationship using PM then PM creates a connection and make these entries in snapmirror.conf

             

            fas-sim-1_fas-sim-2_1208568606=multi(192.168.98.12,192.168.98.13)

            fas-sim-1_fas-sim-2_1208568606:comp_vsm_src fas-sim-2:comp_vsm_dst compression=enable - - - - -

             

            (Note if 1 is set on primary then 1 in secondary is used. Its the respective interface and not 1 in source and 2 in destination is used)

             

            Even in this case the schedule are driven by DFM server scheduler service and not ontap.This applies to both QSM and VSM.

             

            So PM is not capable of creating a compression enabled QSM or VSM relationship.

             

            The same applies to SV also where we don't create the snapvault snap schedules. In order to have the SV traffic in a preferred interface the following options in filer should be set.

            ndpmd_preferred.interface.

             

            Case2:Where relationships are imported.

             

            First and foremost rule:

            PM does not delete any snapmirror.conf or snapvault schedule when the corresponding relationships are imported into a Dataset.This enables PM to interoperate with Compression enabled

            snapmirror.

             

            So to have a compression enabled relationship in PM have the relationship created in the filer and the corresponding conf entries.Import them into a data and have the schedule in PM

             

            fas-sim-1_fas-sim-2_1208568606=multi(192.168.98.12,192.168.98.13)

            fas-sim-1_fas-sim-2_1208568606:comp_vsm_src fas-sim-2:comp_vsm_dst compression=enable - - - - -

             

            It is advised to remove the schedule in the snapmirror.conf to remove the redundant scheduling. Make sure that hostPreferredAddr1/2 are not set on the source and destination filer of the compression enabled snapmirror filers in DFM.

             

             

             

            When a relationship is imported say like below which has snapmirror.conf entry as follows

            fas-sim-1:/vol/qsm_src/- fas-sim-2:/vol/qsm_dst/qsm_dst_non_qtree_data - 0-59/3 * * *

             

            and hostPreferredAddr1/2 are correspondingly set on the source and destination filer then PM makes an entry in the snapmirror.conf as follows

             

            fas-sim-1_fas-sim-2_1208568606=multi(192.168.187.12,192.168.187.13)

            fas-sim-1_fas-sim-2_1208568606:/vol/qsm_src/- fas-sim-2:/vol/qsm_dst/qsm_dst_non_qtree_data - - - - -

             

            Still the old entry remains.

             

            The reason why hostPreferredAddr1/2 should not be set for compression enabled snapmirror is because before every transfer we rewrite the snapmirror.conf with the above two entry in this process compression options is deleted as there is no support for it in NDMP or api.

             

             

            Hope this clarifies.

             

            Regards

            adai

            • Re: Snap Mirror Compression  and OnCommand 5
              localizedinsanity
              Currently Being Moderated

              I had originally asked....how would you go about retrofitting existing PM created relationships to enable compression?  I subsequently worked that out....here are the steps...

              * add dfm host set <hostname> hostPreferredAddr1=x.x.x. to each source node's configuration within DFM

              * manually edit the /etc/snapmirror.conf file to populate it using the output of "snapmirror status" for source/destination pairings

                   * remember to add the "compression=enable" in the options field and to keep the schedule - - - -

                   * remember to configure your connection string settings

                   * I use a text editor to fully create the file structure, then paste it into the CLI using "wrfile /etc/snapmirror.conf" & ctrl-c

               

              Once you do those steps, Protection Manager will use compression the next time it updates.  This process assumes you have pre-existing datasets with policies defined and mirrors baselined/updating.  If you need to configure new relationships, Protection Manager will populate the snapmirror.conf file (thanks to the hostPreferredAddr1 setting) and all you need to do is edit the configuration file to change the source node name to the connection string and add the compression option. 

               

              NOTE - an initialize process will NOT be compressed because Protection Manager cannot enable compression as pointed out in previous comments.  Unless you create the relationship using System Manager without initializing, make the changes to snapmirror.conf, manually initialize, then import as outlined in Adai's post, you cannot initialize using compression.  I hope this helps the next person to ask the question!

               

              Thanks to Adai for getting me on the right track with his posting about preferred addresses.

More Like This

  • Retrieving data ...