5 Replies Latest reply: Jun 27, 2013 2:41 AM by REGIS.GARRUCHET RSS

Performance issue when provisioning qtrees in a dataset

REGIS.GARRUCHET Sprinter
Currently Being Moderated

Hi,

 

IHAC where we have some serious performance issues when provisioning new qtrees in a dataset.

 

The DFM server is on 4 vCPU VM with 8 GB of memory. There is no contention on the ESX server.

DB size is about 100 MB

I tried to grow dbCache to 1 GB without no change in behaviour. I disabled PerfAdvisor with still no change in behaviour.

 

Web GUI and NMC GUI are very responsive and CPU is OK most of the time.

 

The DFM server monitors only 4 controllers. At the current time, there is only 2 vfilers, one only active. The active vfiler contains 16 data volumes (containing 25 qtrees each) managed in a dataset.

The dataset follows a DR Mirror then Backup Protection Policy without a single issue.

All snapmirrors are done once a day in less than 2 minutes

All snapvaults are done once a day in less than 15 minutes.

 

The only problem we have is when trying to provision a new qtree (or sometimes when changing the quota of an existing one).

When using the NMC GUI, it times out after some time and does nothing.

When using a dfpm CLI command, it works but it takes ages.

What I see in the logs: A Conformance Dry Run occurs immediately but nothing happens for 7-10 minutes, then the provisioning job is launched and done in a few seconds (vol creation, qtree creation, share creation, then creation of protection volumes when necessary). Then we have to wait 5-10 more minutes before the CLI is available again.

 

We don't see that on another dataset .

 

I think we will try to add a new dataset but it will add complexity while there is no need for a new one to manage snapmirror and snapvaults since they alway complete in a few minutes.

 

My question:

- is this behaviour normal or not since it is the only performance issue we with OnCommand and CPU is not full at all ?

- is there a way to at least minimize the time taken to edit existing qtrees ?

 

The dfm diag is attached.

Here is an extract from the conformance log show what happens when provisioning a new qtree (15 minutes):

 

Apr 19 12:06:02 [dfmserver: INFO]: [2576:0x10d8]: Dry run: action: Selected volume xxxx:/xxxx_Data16 from dataset node to process the provisioning request.

Apr 19 12:07:00 [dfmserver:DEBUG]: [2576:0x1104]: Conformance checker started scanning at Fri Apr 19 11:57:38 2013

Apr 19 12:07:00 [dfmserver:DEBUG]: [2576:0x1104]: Finished scanning dataset xxxx_Data with protection policy: DR Mirror, then back up, unresolvable tasks: 0, resolvable tasks: 2, resolvable-need confirm tasks: 0

Apr 19 12:14:21 [dfmserver: INFO]: [2576:0x1604]: Successfully exported path /vol/xxxx_Data16/PG-xxxx over CIFS.

Apr 19 12:14:24 [dfmserver: INFO]: [2576:0x1604]: Modified CIFS share PG-xxxx with permission Change for user Everyone.

Apr 19 12:14:24 [dfmserver: INFO]: [2576:0x1604]: Successfully modified share PG-xxxx with path /vol/xxxx_Data16/PG-xxxx.

Apr 19 12:14:27 [dfmserver: WARN]: [2576:0x5c0]: Starting a re-run of provision checker task for dataset: 1046

Apr 19 12:14:27 [dfmserver: INFO]: [2576:0x1604]: Running provision request on dataset 1046

Apr 19 12:14:27 [dfmserver: INFO]: [2576:0x1604]: Running conformance checker after executing provisioning requests

Apr 19 12:14:27 [dfmserver: INFO]: [2576:0x1604]: Dataset 'xxxx_Data' (1046) is busy in check and reserve run as conformance tasks are in progress. Deferred conformance run scheduled as username 'NT AUTHORITY\SYSTEM' with user confirmation set to 1.

Apr 19 12:21:19 [dfmserver:DEBUG]: [2576:0x1718]: Conformance checker started scanning at Fri Apr 19 12:14:13 2013

Apr 19 12:21:19 [dfmserver:DEBUG]: [2576:0x1718]: Finished scanning dataset xxxx_Data with protection policy: DR Mirror, then back up, unresolvable tasks: 0, resolvable tasks: 1, resolvable-need confirm tasks: 0

 

More Like This

  • Retrieving data ...

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points