4 Replies Latest reply: Feb 18, 2014 2:59 PM by akerr RSS

cinder extend error with iscsi

youngs
Currently Being Moderated

We have been doing Openstack POC..

we have an Error during trying to extend cinder volume in cDOT iSCSI FlexVol. ..(  ubuntu 12.04 KVM .. Havana version .. cDOT 8.2P3, flexclone enabled, iscsi FlexVol 3TB , available size 2.9TB )

Cinder volume in cDOT NFS FlexVol was OK  . Also manual LUN extend was OK in FAS.

It is very critical issue and I can’t find any same cases in google.

I think that it is maybe our Openstack iSCSI Driver bug. ( I also tested it with devstack in our LAB and got a same problem..  )

Please help..

 

2014-02-17 16:00:09.679 9716 ERROR cinder.volume.manager [req-00241cbd-6535-4459-8e95-451cc01b3bb5 3712ec7a451f452ebf0903bbff517c85 ca966fc251414a5caf81f3c62a994527] volume 1a2ff3ac-df8e-45a4-81d8-dbef238e39f2: Error trying to extend volume
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager Traceback (most recent call last):
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager   File "/opt/openstack/cinder/cinder/volume/manager.py", line 877, in extend_volume
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager     self.driver.extend_volume(volume, new_size)
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager   File "/opt/openstack/cinder/cinder/volume/drivers/netapp/iscsi.py", line 609, in extend_volume
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager     self._do_sub_clone_resize(path, new_size_bytes)
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager   File "/opt/openstack/cinder/cinder/volume/drivers/netapp/iscsi.py", line 713, in _do_sub_clone_resize
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager     self._destroy_lun(new_path)
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager   File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager     self.gen.next()
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager   File "/opt/openstack/cinder/cinder/volume/drivers/netapp/iscsi.py", line 708, in _do_sub_clone_resize
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager     self._clone_lun(name, new_lun, block_count=block_count)
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager   File "/opt/openstack/cinder/cinder/volume/drivers/netapp/iscsi.py", line 988, in _clone_lun
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager     self.client.invoke_successfully(clone_create, True)
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager   File "/opt/openstack/cinder/cinder/volume/drivers/netapp/api.py", line 222, in invoke_successfully
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager     raise NaApiError(code, msg)
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager NaApiError: NetApp api failed. Reason - 13115:Integer value 46137344 out of range [1..2^24] for block-count
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager

  • Re: cinder extend error with iscsi
    akerr
    Currently Being Moderated

    How are you attempting to extend the volume(command line cinder client, Horizon GUI, or custom).  Can you provide the exact steps to reproduce?  What was the original size of the volume and the requested extended size of the volume?  I have attempted this myself using the cinder client with stable/Havana devstack on ubuntu 12.04.4LTS and could not reproduce.  My flexvol was 50GB in size, but allowed to automatically grow, and my cDOT is running 8.2P3.

     

    akerr@openstack4:~$ tail /etc/cinder/cinder.conf

     

    [cmodeiSCSI]

    netapp_password = xxx

    netapp_login = admin

    netapp_vserver = test

    netapp_storage_family = ontap_cluster

    netapp_storage_protocol = iscsi

    netapp_server_port = 80

    netapp_server_hostname = x.x.x.x

    volume_driver = cinder.volume.drivers.netapp.common.NetAppDriver

     

    akerr@openstack4:~/devstack$ cinder create 1

    +---------------------+--------------------------------------+

    |       Property      |                Value                 |

    +---------------------+--------------------------------------+

    |     attachments     |                  []                  |

    |  availability_zone  |                 nova                 |

    |       bootable      |                false                 |

    |      created_at     |      2014-02-17T15:22:47.170709      |

    | display_description |                 None                 |

    |     display_name    |                 None                 |

    |          id         | d8531963-525a-4d24-94da-e04a8c3be1d3 |

    |       metadata      |                  {}                  |

    |         size        |                  1                   |

    |     snapshot_id     |                 None                 |

    |     source_volid    |                 None                 |

    |        status       |               creating               |

    |     volume_type     |                 None                 |

    +---------------------+--------------------------------------+

    akerr@openstack4:~/devstack$ cinder list

    +--------------------------------------+-----------+--------------+------+-------------+----------+-------------+

    |                  ID                  |   Status  | Display Name | Size | Volume Type | Bootable | Attached to |

    +--------------------------------------+-----------+--------------+------+-------------+----------+-------------+

    | d8531963-525a-4d24-94da-e04a8c3be1d3 | available |     None     |  1   |     None    |  false   |             |

    +--------------------------------------+-----------+--------------+------+-------------+----------+-------------+

    akerr@openstack4:~/devstack$ cinder extend d8531963-525a-4d24-94da-e04a8c3be1d3 2

    akerr@openstack4:~/devstack$ cinder list

    +--------------------------------------+-----------+--------------+------+-------------+----------+-------------+

    |                  ID                  |   Status  | Display Name | Size | Volume Type | Bootable | Attached to |

    +--------------------------------------+-----------+--------------+------+-------------+----------+-------------+

    | d8531963-525a-4d24-94da-e04a8c3be1d3 | available |     None     |  2   |     None    |  false   |             |

    +--------------------------------------+-----------+--------------+------+-------------+----------+-------------+

     

    Screen Shot 2014-02-17 at 10.37.30 AM.png

    • Re: cinder extend error with iscsi
      akerr
      Currently Being Moderated

      I have been able to reproduce this issue as well and have opened https://bugs.launchpad.net/cinder/+bug/1281279 to track the issue.

       

      The failure will appear after a pseudo-random number of extends against the same volume.  The exact number of times it takes seems somewhat dependent on the original size of the volume and the size of the extensions.  The following sequence appears to always cause the failure:

       

      akerr@openstack4:~/devstack$ cinder create 4

      +---------------------+--------------------------------------+

      |       Property      |                Value                 |

      +---------------------+--------------------------------------+

      |     attachments     |                  []                  |

      |  availability_zone  |                 nova                 |

      |       bootable      |                false                 |

      |      created_at     |      2014-02-17T20:25:44.961265      |

      | display_description |                 None                 |

      |     display_name    |                 None                 |

      |          id         | 5ec14ba8-2da0-4e19-8594-9bee9a6ab641 |

      |       metadata      |                  {}                  |

      |         size        |                  4                   |

      |     snapshot_id     |                 None                 |

      |     source_volid    |                 None                 |

      |        status       |               creating               |

      |     volume_type     |                 None                 |

      +---------------------+--------------------------------------+

      akerr@openstack4:~/devstack$ cinder extend 5ec14ba8-2da0-4e19-8594-9bee9a6ab641 8

      akerr@openstack4:~/devstack$ cinder extend 5ec14ba8-2da0-4e19-8594-9bee9a6ab641 20

      akerr@openstack4:~/devstack$ cinder extend 5ec14ba8-2da0-4e19-8594-9bee9a6ab641 30

      akerr@openstack4:~/devstack$ cinder list

      +--------------------------------------+-----------------+--------------+------+-------------+----------+-------------+

      |                  ID                  |      Status     | Display Name | Size | Volume Type | Bootable | Attached to |

      +--------------------------------------+-----------------+--------------+------+-------------+----------+-------------+

      | 5ec14ba8-2da0-4e19-8594-9bee9a6ab641 | error_extending |     None     |  20  |     None    |  false   |             |

      +--------------------------------------+-----------------+--------------+------+-------------+----------+-------------+

  • Re: cinder extend error with iscsi
    alexmc
    Currently Being Moderated

    16777216 is 2^24 blocks. The API is being asked for something that’s way too large at 46137344 blocks.  What is the space request you’re making for the LUN extend?

  • Re: cinder extend error with iscsi
    akerr
    Currently Being Moderated

    Youngs,

     

    I have identified the bug and I have submitted a patch for review in the community.  Initially this is going into master (icehouse) but I think we should be able to back port easily enough to Havana.  In the mean time, you can view the patch and make the code changes manually if you would like to test it out: https://review.openstack.org/74524

More Like This

  • Retrieving data ...