60 Replies Latest reply: Oct 21, 2013 8:59 PM by adaikkap RSS

Quota notification email

Igor Stojnov Cyclist
Currently Being Moderated

Is it possible for a filer to send out an e-mail directly to the user, when they reach soft/hard quota limit? A script, perhaps?

 

Examples and guides would be most welcome!

 

Thanks,

Igor

  • Re: Quota notification email
    msaravan NetApp Employee Cyclist
    Currently Being Moderated

    Hi lgor,

     

    I am not sure about scripts or filer.  But we have a product called Operations Manager (part of DFM suite) that can do more than what you expect.

     

    The below are the events Operations Manager can generate right today after monitoring your storage system's quota setup:

     

    user-disk-space-quota-almost-full

    user-disk-space-quota-full 

    user-disk-space-quota-ok   

    user-disk-space-soft-limit-exceeded 

    user-disk-space-soft-limit-not-exceeded

    user-files-quota-almost-full   

    user-files-quota-full    

    user-files-quota-utilization-normal  

    user-files-soft-limit-exceeded       

    user-files-soft-limit-not-exceeded     

    volume-quota-almost-overcommitted

    volume-quota-not-overcommitted        

    volume-quota-overcommitted

     

    Regards,

    Saravanan

    • Re: Quota notification email
      Igor Stojnov Cyclist
      Currently Being Moderated

      Helo Saravanan,

       

      Yes, I'm looking at DFM right now. It looks a bit complicated to navigate and configure. Under Setup menu, parameters regarding quota notifications can be found in Default Thresholds, Events And Alerts and Users. And you can also choose a specific qtree, and click on Edit Quota Settings and work there. This discussion here also mentioned having to create notification groups and it seems it's not very easy to grasp...

       

      Other than this guide http://now.netapp.com/NOW/knowledge/docs/DFM_win/rel371/pdfs/opsmgr.pdf is there any other document describing step-by-step setup for user quota alerts?

       

      P.S. Just a thought... since Operations Manager is a part of Base Pack software pack, this may work for users which have purchased NetApp recently. Old users may not have that option.

      • Re: Quota notification email
        txskibum2000 Certified Sprinter
        Currently Being Moderated

        Need to create an email account for the user in DFM.  domainname\username email address is blahblah@domain.com  This is in quota user settings.  I can send a screenshot if you like.

         

        I would like to change the default email address from the sender of the email notification.  Right now, emails being sent to users come from "IT Administrator".

         

        Dale

        • Re: Quota notification email
          Igor Stojnov Cyclist
          Currently Being Moderated

          A screenshot would be helpful, thanks!

           

          Igor

        • Re: Quota notification email
          rmharwood Cyclist
          Currently Being Moderated

          I didn't read the linked articles but we're doing quota notifications with DFM. See the "dfm quota mailformat" command - it allows you to specify an email template file, and in that file you can specify your own mail headers. For example, at the top of the file I have:

           

               From: <abc@xyz.com>

               Subject: Your personal drive is nearing its quota limit

           

               email content.....

           

          You place the desired sender's email address in the From: field.

          • Re: Quota notification email
            txskibum2000 Certified Sprinter
            Currently Being Moderated

            Thanks rmarwood..  I will try that.

            igor, see attachment.

            • Re: Quota notification email
              msaravan NetApp Employee Cyclist
              Currently Being Moderated

              Hi Igor,

               

              I have given step-by-step procedure below which you can use to configure quota alerts for individual users.

               

              1=> Set your email server @ Setup -> Options -> Events and Alerts

               

              2=> Look at the default quota thresholds @ Setup -> Options -> Default Thresholds

               

              3=> Set your e-mail domain-name @ Setup -> Options -> Users. You can leave this field empty if you specify complete e-mail address at User settings page @ step7.

               

              4=> Check that you set the credentials for the storage systems (which have user quotas) being monitored in DFM using dfm host set <storage system> hostlogin=xxxx hostpassword=xxxx. This is needed for the discovery of User quotas. Once the credentials are set, re-discover the storage system using 'dfm host discover <storage_system>'. This is required to keep DFM to have up-to-date information with respect to storage system.

               

              5=> Wait for few mins (till userquotas mon completes) and launch "User Quotas All" Report (Go to Control Center -> Home -> Group Status -> File SRM/Quotas -> Report -> User Quotas, All)

               

              6=> "User Quotal, All" page will show you all the usernames for whom quotas are configured.

               

              7=> Click on one of the Usernames which will launch "User Details" page. Click "Edit Settings" link available in the bottom panel "QuotaUser Tools". It will launch "Edit User Settings" page where you can specify the e-mail address for the user.

               

              8=> You need to repeat the above step for all the users. If you are not comfortable with UI, you can do this using CLI.

               

              dfm quota user list
              dfm quota user set <username> userEmail=abc@xyz.com

               

              A simple script might help you here.

               

               

              Hope these steps help you to fix your problem.

               

              Regards,

              Saravanan

              • Re: Quota notification email
                Igor Stojnov Cyclist
                Currently Being Moderated

                Thanks for the reply Saranavan,

                 

                I'll give it a try today!

                 

                One thing though... while testing quotas, I noticed that FilerView immeditelly registers when some exceeds their quota. However, DFM doesn't... it's only after I manually issue "dfm host discover <storage_system>" command that it registeres the change, and even than it'll be after a few minutes.

                 

                Is there a way to have DFM keep up with the changes in real-time, automatically?

                 

                Thanks,

                Igor

                • Re: Quota notification email
                  msaravan NetApp Employee Cyclist
                  Currently Being Moderated

                  Hi Igor,

                   

                  Filerview is something like you are on the box directly. So, whatever changes you do on CLI (or via some operations), that will be reflected immediately in Filerview.

                   

                  But, DFM is a Storage Management product. It is designed in a way that the userquotas are monitored only once in 24 hours (1 day). i.e 24 hrs is a default timer set for userquota mon.

                  Again, this option is configurable using "dfm options set".  If you reduce this value, it means that DFM will do more polling which in turn will consume few more CPU cycles of your storage system.

                   

                  [root@10 log]# dfm options list | grep -i userQuotaMonInterval
                  userQuotaMonInterval                  1 day
                  [root@10 log]#


                  Regards,

                  Saravanan

                  • Re: Quota notification email
                    msaravan NetApp Employee Cyclist
                    Currently Being Moderated

                    On your other question :

                     

                    >> Is there a way to have DFM keep up with the changes in real-time, automatically?

                     

                    Please file a RFE for this.  We can make this possible in DFM using SNMP Traps feature. The flow should be :

                     

                    1. Pre-Requisite : On your storage system, you need to configure DFM server as snmp traphosts
                    (snmp traphost add <DFM_Server_IP>)

                     

                    2. When DFM server receives traps (quotaExceeded, quotaNormal) from these storage systems, we can have DFM to trigger userquota monitor which will fetch you the latest data. DFM would send mails to the users when their  quotas are exceeded.

                     

                    Regards,

                    Saravanan

                  • Re: Quota notification email
                    Igor Stojnov Cyclist
                    Currently Being Moderated

                    Hello Saravanan,

                     

                    There's something a miss with e-mail parameters... Here's your checklist from before:

                     

                    1=> Set your email server @ Setup -> Options -> Events and  Alerts  - DONE.

                    2=>  Look at the default quota thresholds @ Setup -> Options ->  Default Thresholds - DONE.

                    3=> Set your e-mail domain-name @ Setup -> Options  -> Users. You can leave this field empty if you specify complete  e-mail address at User settings page @ step7.  - DEFINED UNDER STEP 7.

                    4=> Check that you set the  credentials for the storage systems (which have user quotas) being  monitored in DFM using dfm host set <storage system>  hostlogin=xxxx hostpassword=xxxx. This is needed for the discovery of  User quotas. Once the credentials are set, re-discover the storage  system using 'dfm host discover <storage_system>'. This is  required to keep DFM to have up-to-date information with respect to  storage system. - DONE.

                    5=> Wait for few mins (till userquotas mon completes)  and launch "User Quotas All" Report (Go to Control Center -> Home  -> Group Status -> File SRM/Quotas -> Report -> User Quotas,  All)

                    6=>  "User Quotal, All" page will show you all the usernames for whom quotas  are configured. - IT'S ALL THERE.

                    7=> Click on one of the Usernames which will launch  "User Details" page. Click "Edit Settings" link available in the bottom  panel "QuotaUser Tools". It will launch "Edit User Settings" page where  you can specify the e-mail address for the user. - DONE.

                    8=> You  need to repeat the above step for all the users. If you are not  comfortable with UI, you can do this using CLI. - OK.

                     

                    Also, under Setup -> Options -> Monitoring, I've set User Quota Monitoring Interval at 10 seconds to see how it goes. Having done that, I filled up test user's home folder to the limit. FilerView registered it, some 30 seconds later DFM also registered that event, but no e-mails were sent out.

                     

                    I've decided to test the e-mailing ability by setting up a random alarm for some critical event @ Setup ->Alarms. When I clicked on Test, an e-mail was sent out to test user detailing that it's an Alarm Test. So, mail settings are properly configured.

                     

                    I than added another alarm, this time specifically for "User Disk Space Quota Almost Full" event and added both myself and the test user as e-mail recipients. Surely enough, 20 seconds after I filled up the home folder for the second time, we received an email each! This means that the quota report can successfully trigger alarms and follow their e-mail settings, but "User Disk Space Quota Almost Full" is a global event... any user can trigger it... I need DFM to contact specific users in  regards to their specific quotas.

                     

                    Is there something else I need to configure for User Settings to work?

                     

                    Thanks,

                    Igor

              • Re: Quota notification email
                Igor Stojnov Cyclist
                Currently Being Moderated

                Since DFM uses SMTP for communicating with storage, could this be a SMTP issue?

                • Re: Quota notification email
                  msaravan NetApp Employee Cyclist
                  Currently Being Moderated

                  No, DFM is not using SMTP for storage system communication.

                   

                  DFM uses SNMP and HTTP (NetApp proprietary version) protocols for 95% of its opeartions.

                   

                  For UserQuotas, DFM is using HTTP (NetApp proprietary version) protocol.

                   

                  Regards,

                  Saravanan

                  • Re: Quota notification email
                    Igor Stojnov Cyclist
                    Currently Being Moderated

                    Sorry Saravanan,

                     

                    I meant to say SNMP. :-)

                     

                    So, if I wanted to have a more frequent DFM update for UserQuotas I'd have to set userQuotaMonInterval to a lower value (e.g 1 minute, 30 secs), and that would be updated through HTTP? Is that enough to trigger a quota alarm message to be sent to a user's e-mail address?

                     

                    I ask this because I think we're getting close to what I need -- to have a local user (with a quota on his home folder) receive an automated e-mail when they exceed soft/almost full/hard limits. Doesn't have to be reflected immediatelly, like on the box, but if I can get it to work within 1 minute or even 30 secs it would be great!

                     

                    >>Again, this option is configurable using "dfm options set".  If you  reduce this value, it means that DFM will do more polling which in turn  will consume few more CPU cycles of your storage system.

                     

                    Since userQuotaMonInterval works through HTTP, I suppose this woldn't be a significant increase in CPU load?

                     

                    Thanks,

                    Igor

                    • Re: Quota notification email
                      rmharwood Cyclist
                      Currently Being Moderated

                      Here is my recommendation. Do not use the built-in quota notification system in DFM to generate near real-time alerts of users getting near to or going over quota. It is designed to run on a periodic basis because it asks your storage controllers for a quota report. This generates a lot of work for your DFM server and your storage controller, depending upon the complexity and scale of your systems. You MAY get away with bringing your quota monitoring interval down to an hour or so but I would still be wary of what additional load this generates.

                       

                      As others have said you really should look into SNMP traps. Read the section in the DFM admin guide about these. It describes the pros and cons of changing monitoring intervals and suggests that SNMP traps - event notifications from the storage controller to an SNMP trap listender regarding a specific event - are better for near or real time monitoring. It's the difference between having your kids ask you every minute "are we there yet" or you telling them when you arrive.

                       

                      I have not looked into the details but I'm quite sure the storage controller will send an SNMP trap when a user quota limit is hit. You (or someone) will have to examine the Netapp SNMP MIB to see what the specific event is so that you can watch for it. What you can do then is to configure an alarm in DFM that triggers when the trap is received. You then have the alarm call a script that sends email to the user as required. This approach isn't simple: it requires knowledge of SNMP and scripting.

                       

                      As someone else pointed out, it would be nice if DFM could natively handle SNMP quota messages and send alerts to users in the same way that quota monitoring already works. I haven't checked - do Netapp have a mechanism for submitting RFEs?

                       

                      Cheers,

                      Richard

                      • Re: Quota notification email
                        Igor Stojnov Cyclist
                        Currently Being Moderated

                        Hello Richard,

                         

                        The systems are two 2020A models set in different parts of the building, cross-replicating data for DR purposes. User documents, various types of mailboxes, archive, etc. Purely NAS device so far. As for quota reports, I wasn't aware they're so demanding in terms of resources. I thought it farily was a simple process/query?

                         

                        Yes, I read about SNMP listener in DFM. Haven't tried it yet, but if all SNMP traps look like this, they're not saying much other than something happened... Am I mistaking? Do SNMP traps carry more information that this?

                         

                        If my notion of SNMP traps is true - and following your paradigm - it would be like receiving a "there was arrival" message. Doesn't say who arrived or where. Perhaps a SNMP trap informing DFM that a quota event happened, follow by an fresh quota monitoring query from DFM would be most effective.

                         

                        On the other hand, what I'm attempting to pull off is a pretty handy thing. It would be nice if this could be incorporated in FilerView, under Quotas, and save us all a lot of network traffic.

                         

                        Cheers,

                        Igor

                        • Re: Quota notification email
                          rmharwood Cyclist
                          Currently Being Moderated

                          Some SNMP traps carry more information than others, I couldn't say one way or the other if the quota traps contain enough information. Before we had DFM we were intending on using syslog to trap quota messages, until we realized that the syslog message provides a Windows SID rather than a username and no qtree name.

                           

                          You can always try to decrease the monitoring interval for quota checks and see what happens. Every environment is different so it may be that your systems can tolerate such a load without adverselt affecting anything. Note that DFM also maintains its own database of user quota information so that information has to be updated also each time a check is run.

                           

                          I doubt very much that the storage controller itself will ever have the functionality to do what you want. The more that you give it to do that isn't directly related to serving NAS/SAN stuff the less effective it is going to be at those tasks.

                           

                          Cheers,

                          Richard

                          • Re: Quota notification email
                            Igor Stojnov Cyclist
                            Currently Being Moderated

                            Well, I've decreased the monitoring interval but I still can't get DFM to send out an email to the test user.

                             

                            - Under Volume Tools I opened the volume's Quota Settings, and set Enable User Quota Alerts to yes.

                            - I did the same under Qtree Tools: Enable User Quota Alerts... yes.

                            - The User Settings are already configured

                            - DFM gets fresh quota reports in regular intervals...

                             

                            ... but unless I set a general Alarm for an event, no e-mails are sent out!

                             

                            DFM simply doesn't work! :-(

                             

                            Cheers,

                            Igor

                            • Re: Quota notification email
                              msaravan NetApp Employee Cyclist
                              Currently Being Moderated

                              Hi Igor,

                               

                              I just tested this in my lab setup and I saw that it is working and I got e-mails too. My setup has both DFM Server and Storage system part of same NIS and DNS domains. But, I dont know whether that really matters. This is one of the legacy features of DFM Server and it should work. 

                               

                              Since this doesnt work for you, can you file a burt on DFM server product. We need to understand your environment better and some one from Development team can look into your issue.


                              Regards,

                              Saravanan

                              • Re: Quota notification email
                                Igor Stojnov Cyclist
                                Currently Being Moderated

                                Hi Saravanan,

                                 

                                I used 2 environments:

                                - a local one, completely virtualized with ONTAP 7.3.2 simulator

                                - a remote on, with real physical servers and a test filer, through VPN

                                 

                                Both are quite simple with regular settings, and both failed to perform.

                                 

                                Would it be too much bother to send me screenshots of pages detailing the parameters you've set for your lab? I just want to make sure I'm not missing something before I file a burt... I need to make this work before we offer this to our client next week.

                                 

                                Thanks,

                                Igor

                              • Re: Quota notification email
                                Igor Stojnov Cyclist
                                Currently Being Moderated

                                I'm wondering if this is a license issue... What licenses do I need to have under Setup -> Options -> Licensed Features?

                                 

                                Thanks,

                                Igor

  • Re: Quota notification email
    lovik_netapp Certified Cyclist
    Currently Being Moderated

    Ok so here's my 2 cents, select the qtree and go to edit qtree page then put email address of user in email field. It works for me (tried in 3.7 as well 3.8) so it should work for you as well.

     

    Now I know that it's big deal if you have to modify each and every qtree to add email address but this is the last thing I can think of as of now.

     

    Don't forget to keep the alarms on for quota exceeded notification.

    • Re: Quota notification email
      Igor Stojnov Cyclist
      Currently Being Moderated

      Hello Lovik,

       

      The alarms are not the problem, they work fine. Like I wrote in this discussion before, I created a "User Disk Space Quota Almost Full" alarm and tested it successfully. The problem with alarms is that they are "global" events... for instance, any  user can trigger my test alarm simply by exceeding their quota. What I need is for DFM to send emails to specific  users when they exceed their own quotas.

       

      But I've taken your advice - and tried both this time. I've set up a  quota for both a test user and a qtree, and then exceeded them. Surely enought, I got this report:

       

      http://img227.imageshack.us/img227/9728/capture1m.jpg

       

      You can see that I've set up the user's email address so he can be notified:

       

      http://img291.imageshack.us/img291/1221/capture2ak.jpg

       

      Same goes for the owner of the qtree:

       

      http://img20.imageshack.us/img20/5631/capture3ujv.jpg

       

      The mail server is setup OK and I've tested the email address DFM is using to send out notifications by setting up various alarms... This is pretty basic setup and as Saravanan said, it should work automatically. Still, I haven't managed to get DFM to send out a single quota  notification email.

      • Re: Quota notification email
        lovik_netapp Certified Cyclist
        Currently Being Moderated

        Ok here are some of my observations.

         

        "The problem with alarms is that they are "global" events... for instance, any  user can trigger my test alarm simply by exceeding their quota"

         

        for this you can create new group in OM and add that system/vol/qtree/user part of that group and then modify the alarm to limit its reach only up-to that group.

         

         

        "What I need is for DFM to send emails to specific  users when they exceed their own quotas"

         

        This should be taken care by user mail address which you have added in owner email field or if you go in options and quota alerts in OM you will see there is an option to add domain name for quota notification mail, here you have to use your organization's FQDN like 'gmail.com' rather than only ''gmail"

         

        after all if nothing works there is a tool in toolchest called as Trapnotify - http://now.netapp.com/eservice/toolchest?toolid=425, you may want to try that, however i don't know how good they are as i have never used it.

        • Re: Quota notification email
          Igor Stojnov Cyclist
          Currently Being Moderated

          Thanks for your input Lovik, but using groups isn't as effective as regular email user notifications that OM is supposed to be sending.

           

          As for Trapnotify, I've tried it out yesterday but after running it I get an error saying "Bad arg length for Socket::inet_ntoa, length is 0, should be 4 at trapnotify line 316". I found the line in question but I've no idea what I'm looking at, I'm also quite inexperienced in scripting. :-)

           

          Regards,

          Igor

  • Re: Quota notification email
    Igor Stojnov Cyclist
    Currently Being Moderated

    Are there at least any LOG files that I can check to see whether Ops Manager attempts to send out e-mails or not?

    • Re: Quota notification email
      adaikkap NetApp Employee Grand Marshal
      Currently Being Moderated

      Hi

           Can you check the following options.

      See if userEnableAlerts is yes by default its enabled.

      [root@lnx ~]# dfm options list userEnableAlerts
      Option           Value
      ---------------- ------------------------------
      userEnableAlerts yes

       

      The dfm sends the username for which the quota exceeded, its the mailserver that should resolve the domain name.If the mailserver domain name is different set the below option so that dfm sends the username with domain.

      By default its blank

       

      [root@lnx ~]# dfm options list userEmailDefaultDomain
      Option                 Value
      ---------------------- ------------------------------
      userEmailDefaultDomain

       

      Similarly see if the value set to SMTP server resolves. ping and see if your smtp server is aliases to mail by doing the following.

       

      ping mail in the command line else change it to appropriate value.

       

      [root@lnx~]# dfm options list SMTPServerName
      Option          Value
      --------------- ------------------------------
      SMTPServerName  mail
      [root@lnx ~]#

       

      If the events are generated then there is not other reason than listed above why the alert is not sent to the user.

       

      Regards

      adai

      • Re: Quota notification email
        Igor Stojnov Cyclist
        Currently Being Moderated

        Hello Adai,

         

        Well, I've tried both approaches.

         

        First, to specify Email Default Domain which would be appended to e-mail address. On my test domain LAB, I've a user named Joe with a home folder at /vol1 volume. I've entered my own e-mail address name (igor) and specified our regular email domain (osap.rs). Alerts are enabled, but I've also enabled user quota alerts on the volume level as well. SMTP server has been defined and pinged, and a there's an account with no password which I use for mail tests ready and waiting. I've tested alarms with it and emails are getting through alright...

         

        Here's the checklist:

         

        C:\>dfm options list userEnableAlerts
        Option           Value
        ---------------- ------------------------------
        userEnableAlerts yes

         

         

        C:\>dfm quota user get LAB\joe
        QuotaUser:                            LAB\joe
        Email Address for Quota Alerts:       igor
        User Quota Full Threshold (%):        90
        User Quota Nearly Full Threshold (%): 80

         

         

        C:\>dfm options list userEmailDefaultDomain
        Option                 Value
        ---------------------- ------------------------------
        userEmailDefaultDomain osap.rs

         

         

        C:\>dfm options list SMTPServerName
        Option          Value
        --------------- ------------------------------
        SMTPServerName  mail.ourspace.si

         

         

        C:\>ping mail.ourspace.si
        Pinging mail.ourspace.si [84.52.141.183] with 32 bytes of data:
        Reply from 84.52.141.183: bytes=32 time=30ms TTL=118
        Reply from 84.52.141.183: bytes=32 time=42ms TTL=118
        Reply from 84.52.141.183: bytes=32 time=32ms TTL=118
        Reply from 84.52.141.183: bytes=32 time=29ms TTL=118
        Ping statistics for 84.52.141.183:
            Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
        Approximate round trip times in milli-seconds:
            Minimum = 29ms, Maximum = 42ms, Average = 33ms

         

         

        C:\>dfm volume get storage:/vol1
        Volume: storage:/vol1
        Volume Full Threshold (%): 90
        Volume Nearly Full Threshold (%): 80
        Volume Full Threshold Interval: 00:00:00
        Enable User Quota Alerts, not available for vserver volumes: Yes
        etc...

         

         

        As usual, I went on and exceeded the quota and the event was registered in OM under User Quota Events. If I understand correctly, DFM should've used igor and osap.rs parameters, to send out a quota alert to igor@osap.rs via set SMTP server. Still, nothing happened.  (I have screenshots but I cannot insert them here for some reason)

         

         

        Then I tried tried the default approach, leaving the Email Default Domain blank and specifying a full e-mail address for the user:

         

        C:\>dfm options set userEmailDefaultDomain=none
        Cleared default email domain for quota alerts.

         

        C:\>dfm options list userEmailDefaultDomain
        Option                 Value
        ---------------------- ------------------------------
        userEmailDefaultDomain

         

        C:\>dfm quota user set -s lab\joe userEmail=igor@osap.rs
        Changed email address for quota alerts for user LAB\joe (125) to igor@osap.rs.

         

        C:\>dfm quota user get LAB\joe
        QuotaUser:                           LAB\joe
        Email Address for Quota Alerts:       igor@osap.rs
        User Quota Full Threshold (%):        90
        User Quota Nearly Full Threshold (%): 80

         

         

        I've exceeded the quota once again but with the same results. User alerts are not getting through...

         

        Can you send me the checklist from your lab? Perhaps an example of what a generated user alert e-mail looks like?

  • Re: Quota notification email
    connoisseur Sprinter
    Currently Being Moderated

    Question about vFiler considerations in this issue!

     

    We use vFilers to seperate NAS servers between different customers, and therefore each and every customer have different domain.

    Now  I have activated quota for a customer i get reply on that the email can´t be verified.. "email domain/user can´t be verified"

     

    This customer has like 5000 users so it will be a massive work to modify each and every user like the steps above in this tread. (Still if scriptet we need someone to mach the username to the correct email)

     

     

    Also, is it not possible to have the email notification only show the share name?

    Must it show complete volume/qtree path?

     

    Thanks

More Like This

  • Retrieving data ...

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points