19 Replies Latest reply: Jul 25, 2013 12:15 PM by SSWALLOWEBC RSS

Orphaned datastores in vCenter ("vmdk_sqlnap")

alliantenergy-njk
Currently Being Moderated

Not sure if this is a SnapDrive problem, SMSQL problem, VSC problem, or SMSP problem.  I think it is SnapDrive...

 

We recently upgraded the various SnapManager products and SnapDrive to allow us to upgrade to vCenter 5.0 U1.  Here is what we currently have:

 

vCenter 5.0 U1

VSC 4.1

SnapDrive 6.4.2

SnapManager SQL 6.0

SnapManager SharePoint 6.0 with Agent Patch 3 which supports SMSQL 6.0

NFS volumes in vCenter

 

According to the Compatibility Matrix, these are all supported.

 

Whenever a SnapManager for SharePoint job finishes, it leaves orphanded datastores in vCenter - one on each SP SQL server.  This used to happen once every 2 or so weeks randomly for one or two servers and I just lived with it,  Now it is doing it every night in all three envrionments and I have cleanup 7 orphans every morning.  This started happening after the upgrade.

 

The names of the datastores are something like this:

OriginalDSName (Backup vmdk_sqlsnap__sqlservername_12-07-2012_10.21.56)

 

I also get these errors in SMSQL logs which Google found no hits for:

 

Dismounting LUN C:\Program Files\NetApp\SnapManager for SQL Server\SnapMgrMountPoint\MPDisk001\ of SnapShot [smsql snapshot]...

SnapDrive operation to dismount LUN in snapshot copy failed. SnapDrive error code: 0xc0041071

SnapDrive operation to dismount LUN in snapshot copy failed. SnapDrive error code: 0xc0041085

[SnapDrive Error]: Unable to locate a LUN to perform requested operation.

(SnapDrive Error Code: 0xc0041085)

Re-trying to force dismounting LUN...

SnapDrive operation to force dismount LUN in snapshot copy  failed. SnapDrive error code: 0xc0041085

[SnapDrive Error]: The LUN may not be connected, because its mount point cannot be found.

(SnapDrive Error Code: 0xc0041085)

Unknown Error, Error Code: 0xc0041085

 

Does anyone know how to prevent this?  Is there a setting I need to enable or a hotfix to install?

Nelson

  • Re: Orphaned datastores in vCenter ("vmdk_sqlnap")
    stemmer
    Currently Being Moderated

    H Nelson,

     

    yes, we have the same problem at one customer side.

     

    The environment is:

       FAS 3240A

       Ontap 8.1.2 7-mode

       Snapdrive 6.4.2

       SnapManager SQL 5.2

       VSC 4.1

       NFS based VMDKs

     

    Same first Snapdrive errror on dismounting than yours, but secondary error code different:

       [01:13:22.818]  Database was detached successfully.

       [01:13:22.824]  Dismounting LUN C:\mnt\verify\MPDisk001 of SnapShot [sqlsnap__muc1s0043_12-12-2012_00.02.29__daily]...

       [01:13:47.341]  SnapDrive operation to dismount LUN in snapshot copy failed. SnapDrive error code: 0xc0041071

       [01:14:00.923]  SnapDrive operation to dismount LUN in snapshot copy failed. SnapDrive error code: 0x80010105

       [01:14:00.927]  Re-trying to force dismounting LUN...

       [01:14:01.128]  SnapDrive operation to force dismount LUN in snapshot copy  failed. SnapDrive error code: 0xc0040221

       [01:14:01.140]  [SnapDrive Error]: The LUN may not be connected, because its mount point cannot be found. (SnapDrive Error Code: 0xc0040221)

     

       [01:14:01.140]  Dismounting LUN C:\mnt\verify\MPDisk002 of SnapShot [sqlsnap__muc1s0043_12-12-2012_00.02.29__daily]...

       [01:14:10.416]  SnapDrive operation to dismount LUN in snapshot copy failed. SnapDrive error code: 0xc0041071

       [01:14:21.637]  SnapDrive operation to dismount LUN in snapshot copy failed. SnapDrive error code: 0x80010105

       [01:14:21.638]  Re-trying to force dismounting LUN...

       [01:14:21.810]  SnapDrive operation to force dismount LUN in snapshot copy  failed. SnapDrive error code: 0xc0040221

       [01:14:21.810]  [SnapDrive Error]: The LUN may not be connected, because its mount point cannot be found. (SnapDrive Error Code: 0xc0040221)

     

    Same try to force dismount.

    The error is shown on every dismount in the log, but the first dismounts are in reality succesfull. That´s why the force dismount failed, because than it´s already disconnected.

    But after 7 succesfull dismounts (all 7 show the snapdrive error in log) the 8 one really miss.

     

    It looks like a timeing problem !

     

    History:

    Last week we had Snapdrive 6.3P2 and Ontap 8.1.1 and every thing is running fine.

    After an NFS outage during 2 VSC virtual machine restores, we decide to upgraded to Ontap 8.1.2 on last friday.

    After that update SnapManager SQL can do succesful backups but cannot do verify´s because of the Snapdrive dismount errors.

    There is no 6.4.2D1 or 6.4.2P1 at the moment, but this error smells like different function giveback codes on 8.1.2

    The force dismount of snapdrive logs immediatly the errors (after 1 sek.)

     

    Have you opened a case ?

    Have you also Ontap 8.1.2 running ?

    Is there feedback from NetApp ?

     

    On our Exchange VM with iSCSI connected Raw Devices the old Snapdrive 6.3P2 works fine on that 8.1.2 based NetApp.

    So i think it´s a problem with vmdk based SnapManager/Snapdrive installations and Ontap 8.1.2

     

    Greetings

    Roland

     

    Nachricht wurde geändert durch: Roland Kurz

    • Re: Orphaned datastores in vCenter ("vmdk_sqlnap")
      alliantenergy-njk
      Currently Being Moderated

      Hi Roland,

       

      Thanks for the information in your post.  I am 99% sure we are running 8.1.2.  I am on vacation this week, so I am going off of memory but I will check when I return to the office.  We had to upgrade many things in order to support the new vCenter 5.0 U1 install.  I know we upgraded Ontap for sure, and I am pretty sure it was 8.1.2.

       

      I opened a case with NetApp because when we upgraded SnapManager for SQL, we broke SnapManager for SharePoint - and they had to provide us a patch to make it all work.  I asked them if I could tag this second problem (orphaned vCenter datastores) onto that case and they told me to open a new one.  I was starting my vacation so I did not have time to open the new case yet.

       

      Nelson

    • Re: Orphaned datastores in vCenter ("vmdk_sqlnap")
      alliantenergy-njk
      Currently Being Moderated

      Sorry, I was wrong.  We upgraded to 8.1.1 not 8.1.2.  I am not a storage admin (I am a Windows Server admin that just happens to be responsible for vCenter and SnapManager for SharePoint) so I am not sure on why they selected 8.1.1 instead of 8.1.2 when they upgraded or the impact this has.

       

      Thanks

      Nelson

      • Re: Orphaned datastores in vCenter ("vmdk_sqlnap")
        NET_KOHLBACH
        Currently Being Moderated

        Hi,

         

        I have the same problems!

        Detaching database [LOHNDATEN__Clone].

        Database was detached successfully.

        Dismounting LUN C:\Program Files\NetApp\SnapManager for SQL Server\SnapMgrMountPoint\MPDisk001 of SnapShot [sqlsnap__w2k8-kronos_12-22-2012_20.00.37__daily]...

        SnapDrive operation to dismount LUN in snapshot copy failed. SnapDrive error code: 0xc0041071

        SnapDrive operation to dismount LUN in snapshot copy failed. SnapDrive error code: 0xc0041085

        [SnapDrive Error]: Unable to locate a LUN to perform requested operation. (SnapDrive Error Code: 0xc0041085)

         

        The environment is:

           FAS 2240A

           Ontap 8.1.2 7-mode

           Snapdrive 6.4.2

           SnapManager SQL 6.0

           VSC 5.1

           NFS based VMDKs

  • Re: Orphaned datastores in vCenter ("vmdk_sqlnap")
    alliantenergy-njk
    Currently Being Moderated

    A suggestion from NetApp support was to configure SMSQL to use a directory without spaces for the verification mount point.  Currently it is set to:

     

    C:\Program Files\NetApp\SnapManager for SQL Server\SnapMgrMountPoint

     

    This is the default set by the installation and is not something I set or configured when I installed the product.  This path worked in the past.  We changed it to a directory on the root of C: with no spaces in the name and tested in our development environment:

     

    C:\smsql_mnt_pnt

     

    The backup suceeded and left no orphaned disks in vCenter.  I going to make the changes in production and see if they work tonight.

    Nelson

    • Re: Orphaned datastores in vCenter ("vmdk_sqlnap")
      alliantenergy-njk
      Currently Being Moderated

      Correction.  Spaces in the name is not the problem, it is the length of the mount point path that is the problem (and in this case, spaces just make the problem worse because it adds to the length).  According to NetApp support, if the the total path is longer than 160 characters, it fails.  So in production last night, we still have 4 servers leave orphans.  The location of the data files in conjunction with the DB file names and the mount point path of "C:\Program Files\NetApp\SnapManager for SQL Server\SnapMgrMountPoint" is longer than 160 characters and the new mount path I uses is also too long.  So now I am trying this:

       

      C:\MP

       

      If this is still too long, I will have to have the DBA move the SP data files from "D:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\DATA" to something like "D:\Data".

      Nelson

      • Re: Orphaned datastores in vCenter ("vmdk_sqlnap")
        James Castro
        Currently Being Moderated

        Did this work?

        • Re: Orphaned datastores in vCenter ("vmdk_sqlnap")
          alliantenergy-njk
          Currently Being Moderated

          Sort of...

          It fixed both QA and DEV.  However, we have seen mixed results in Production.  It seems at least one server still has such long paths that it is failing.  The names of the some of the SP databases are so long that even with a short mount-point path, it is still too long.  But then, the Friday backup worked without a problem.  So I am going to let it run a few more days and try to figure any rhyme or reason to when it fails or works.  Before it was 100% consistent.  So we are seeing improvements...but maybe that is worse in the long run because now I cannot consistentely recreate the problem...

          Nelson

          • Re: Orphaned datastores in vCenter ("vmdk_sqlnap")
            alliantenergy-njk
            Currently Being Moderated

            Oh - and I was wrong about the "D:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\DATA" path.  That is where our system databases are stored.  Our SP databases are already in a short path location: S:\MSSQL\Data.  So I guess I could shorten a bit more by making it something like "S:\DB" but I am not sure how much of a difference that will really make.  MS putting the GUID in the name of the DB files is not really helping us

            SessionStateService_d78200b5804d4e19885f5e2064ed9523.mdf

            Just one example...

  • Re: Orphaned datastores in vCenter ("vmdk_sqlnap")
    chqlsnetapp
    Currently Being Moderated

    We are having the same problem. Anyone managed to get a few pointers? I'd relate it to the new version of vcenter for now.

  • Re: Orphaned datastores in vCenter ("vmdk_sqlnap")
    GGS-SUPPORT
    Currently Being Moderated

    Has anyone a solution for this Problem?

    TY. Walter

  • Re: Orphaned datastores in vCenter ("vmdk_sqlnap")
    CADOLT-FT
    Currently Being Moderated

    Hi there,

     

    we encounter similar problems with our Snapmanager for SQL installation.

     

    Are there any updates?

     

    Rgds,

    Michael

  • Re: Orphaned datastores in vCenter ("vmdk_sqlnap")
    SSWALLOWEBC
    Currently Being Moderated

    I'm noticing no activity on this post for a while.  Has everyone given up or has there been a resolution not posted?

More Like This

  • Retrieving data ...