Jive SBS Recent Blog Comments Syndication Feed

RE: Multimode VIF Survival Guide

trey.layton@netapp.com — Tue, 10 Nov 2009 22:12:46 GMT

Stuart

Thanks for the detail. In your first example with two dynamic multimode VIFs you would actually role these two dynamic multimode VIFs into what we call at NetApp a second level single mode VIF. You would then prefer one of the dynamic multimode VIFs over the other. This would render one of the dynamic multimode VIFs offline until all the interfaces in the preferred VIF went offline. If such a case were to happen then all IP and MAC information would float to the secondary (standby) dynamic multimode VIF. Many customers run in this configuration today but they don't like the idea of having one multimode VIF inactive. To that end many network manufactures have created switch clustering technologies which allow a etherchannel to be actively spanned across two physical chassis. In the case of your 6500 this is called Multi-Chassis Etherchannel and has some hardware requirements for Supervisor and Linecards. I have written an article on these various technologies and have provided that link below. You would need to use this type of technology on the switch side to achieve an all active interface scenario which spans multiple switches. If you are not able to deploy these diverse switch spanning etherchannel technologies then you must deploy the two VIFs in an active and passive state.

With regard to our description in the Network Administration Guide for ONTAP about how Dynamic MultiMode VIFs are able to in effect sense inactivity, this is unfortunately not some technology that NetApp has invented and is unique to NetApp, it is actually how the LACP standard is written and the features which have been provided with this 802.3 specification.

That is the standard states binding or aggregating links may be executed under manual control through direct administrator manipulation. Additionally, automatic determination, configuration, binding and monitoring may occur through the use of the Link Aggregation Control Protocol. The LACP uses peer exchanges across the bound links to determine, on an ongoing basis, the aggregation capability of the various links, and continuously provides the maximum level of aggregation capability achievable between a given pair of systems.

We spend alot of time talking about static versus dynamic. Static is administratively bound or aggregated interfaces and the only means to determine failure is loss of link. Dynamic is LACP and the means to determine failure extend beyond link loss through peer exchange. This is why every reference in the site states that when you can use LACP do so because it has many more mechanisms to determine failure. Loss of link is the most absolute form of failure but we all know that links don't always just loose link. They sometimes half fail and the automatic binding mechanism in LACP has been developed to leverage those features. ONTAP does not use peer exchanges for configuration binding but does for all the other variables mentioned above.

I encourage you to read this link regarding the diverse switch ether-channeling technologies.

I hope this helps in your deployment efforts,

Trey

Link I mentioned above:

http://communities.netapp.com/blogs/ethernetstorageguy/2009/09/23/virtual-port-channels-vpc-cross-stack-etherchannels-multichassis-etherchannels-mec--what-does-it-all-mean-and-can-my-netapp-controller-use-them

RE: Multimode VIF Survival Guide

community@netapp.com — Mon, 09 Nov 2009 00:46:15 GMT

Hi Trey,

OK, let's say I care more about high-availability than I do about throughput. To that end, I install two switches in each of my data centers. Is there a way to use Dynamic Multimode VIFs to survive what I call "Lights are on; no one is home"? Meaning, the cases in which the Sup card fries (or a hapless administrator assigns one of the interfaces feeding the filer to the wrong VLAN). Link stays up ... but the relevant filer ports are 'isolated' from one another.

From the bottom of p. 128 in http://now.netapp.com/NOW/knowledge/docs/ontap/rel732/pdfs/ontap/nag.pdf

"Dynamic multimode vifs can detect not only the loss of link status (as do static multimode vifs), but also a loss of data flow. This feature makes dynamic multimode vifs compatible with high-availability environments."

In the example below, e0a + e1b belong to one Dynamic Multimode VIF, while e0b and e4a belong to a second Dynamic Multimode VIF. If the Sup card in c6k-a-switch fries, then I would predict that LACP Hellos would quit flowing from the switch to the filer, and that Ontap would deactivate vif1. [And if a hapless administrator assigns Port1 to the wrong VLAN, then ... hmm, I don't think LACP would detect this problem, and bad things would happen.]

                        head-a
                     vif1           vif2
                 e0a   e1b   e0b   e4a
                /        |         |        \
               /         |         |          \
              /          |         |           \
             /           |         |            \
            /            |         |             \
           /             |         |              \
          /              |         |               \
         Port1      Port2     Port3       Port4
        cat6k-a-switch--------cat6k-b-switch
                 |                          |
                 |                          |
                 |                          |
             router-a                router-b
                      \              /
                        \           /
                      corporate network

As an aside, I've tried tackling this problem using Single-Mode VIFs, without success. In this scenario, if I assign, say, Port1 to the wrong VLAN, svif1 stays up and Ontap continues trying to use e0a. If I simulate a Sup card frying (by rebooting cat6k-a-switch), the results are intermittent -- I believe that Ontap deactivates individual NICs when link goes down ... but of course, during most of the reboot, link is up, but the switch isn't forwarding frames ["Lights are on; but no one is home"], and Ontap doesn't realize this and tries to use NICs plugged into Ports 1 & 2, with unpleasant results.

                        head-a
                   svif1          svif2
                 e0a   e1b   e0b   e4a
                /        \      /           \
               /          \    /             \
              /            \ /               \
             /              \/                 \
            /               / \                 \
           /               /   \                 \
          /               /     \                 \
         Port1      Port2     Port3       Port4
          cat6k-a-switch--------cat6k-b-switch
                 |                          |
                 |                          |
                 |                          |
             router-a                router-b
                      \              /
                        \           /
                      corporate network

At any rate, I have two questions:

(a) From the point of view of High-Availability, would you buy my story around using Dynamic Multimode VIFs, as above? Or would you suggest I investigate another approach?

(b) What does the "can detect not only loss of link status ... but also a loss of data flow" sentence mean? I don't see how LACP helps anybody detect loss of packets flowing across a link. Perhaps this means something like "If you administratively remove a switch port from the aggregate, LACP will detect the change in the channel definition and remove that path from the filer's view of the aggregate." ? Or does Ontap track the packet receive counter on each NIC and use changes in the behavior of that counter to influence whether or not it deactivates/activates a NIC?

--sk

Stuart Kendrick

RE: Multimode VIF Survival Guide

community@netapp.com — Fri, 16 Oct 2009 03:45:08 GMT

I think that I've partially answered my own question:

It doesn't make sense to have more active iSCSI connections than there are physical links on the host; you're certainly not going to get more bandwidth than the physical links will support.

However, it would be important to select the pairs in such a way that the XOR hash doesn't select the same physical link on the LACP EtherChannel between the switches and the filer. From what I've been able to determine, the interface selection is done according to the following formula:

(Source_IP) XOR (Dest_IP) MOD (Interface_Qty) = selected interface index

So in the case of my example above, my possibilities are:

.80<->.63     50 xor 3F = 6F     6F mod 4 = 3
.80<->.64     50 xor 40 = 10     10 mod 4 = 0
.80<->.65     50 xor 41 = 11     11 mod 4 = 1
.80<->.66     50 xor 42 = 12     12 mod 4 = 2
.180<->.63    B4 xor 3F = 8B     8B mod 4 = 3
.180<->.64    B4 xor 40 = F4     F4 mod 4 = 0
.180<->.65    B4 xor 41 = F5     F5 mod 4 = 1
.180<->.66    B4 xor 42 = F6     F6 mod 4 = 2

So in this case, 10.17.10.80<->10.17.10.64 would route traffic over the first physical link in the EtherChannel, and 10.17.10.180<->10.17.10.65 would route traffic over the second physical link.

Is this correct, according to your understanding? It would also make sense to hash a second host's IPs so that you could pick pairs that would select links three and four, yes?

Thanks,

Chuck

RE: Multimode VIF Survival Guide

community@netapp.com — Thu, 15 Oct 2009 03:29:01 GMT

Trey:

What are the implications of the alias IP addresses for iSCSI MPIO?

I have my filer configured as follows:

hostname HOUFAS2050A
vif create lacp HOUFAS2050A -b ip e0a e0b e1a e1b
vlan create HOUFAS2050A 10 
ifconfig HOUFAS2050A-10 10.17.10.65 netmask 255.255.255.0 partner HOUFAS2050B-10 mtusize 9000 trusted -wins up
ifconfig HOUFAS2050A-10 alias 10.17.10.66 netmask 255.255.255.0
ifconfig HOUFAS2050A-10 alias 10.17.10.63 netmask 255.255.255.0
ifconfig HOUFAS2050A-10 alias 10.17.10.64 netmask 255.255.255.0
route add default 10.17.10.1 1

My host has two NICs dedicated to iSCSI, with the following addresses:

10.17.10.80
10.17.10.180

Everything is on the same subnet, same VLAN, connected via a cross-stack LACP EtherChannel on my 3750-E stack.

You said, "Simply placing the additional addresses will not exploit the advantage of additional addresses. You must ensure that the hosts which mount data from the NetApp controllers utilize all of the addresses." But the examples you gave were both NFS. Is this true for iSCSI in this scenario? If I were to set up multipath I/O from each host interface to each filer ip address, I'd have eight total iSCSI connections. Is this optimal, redundant, or less-than-optimal?

Thanks!

Chuck

RE: Multimode VIF Survival Guide

community@netapp.com — Wed, 14 Oct 2009 23:22:46 GMT

[deleted duplicate post]

RE: How to start a user group?

summersj@netapp.com — Thu, 08 Oct 2009 00:47:22 GMT

You should also add that can go to the "Group" section of the community and start an online group with-in the community to support your events as well.

http://communities.netapp.com/groups/

RE: Fractional Reservation - LUN Overwrite

ckranz@b2net.co.uk — Wed, 07 Oct 2009 20:14:24 GMT

Definitely agree with your description of events. I've seen it happen on a few other odd cases, but generally you're right, it's be caused by the AFS growing quicker than expected and as such using up extra space, or causing the FR to reserve more space than expected.

I agree the simpler you can explain FR, the better and easier it is to understand. But a lot of the rules around FR and snapshots tend to be a little flexible, and tend to be at the whim of the AFS. Ultimately the Active File System is the most important thing to the filer, and the filer gives this ultimate priority.

RE: Fractional Reservation - LUN Overwrite

community@netapp.com — Wed, 07 Oct 2009 19:46:43 GMT

Hmmmmm,

Snapshots can, and do regularly grow into the Fractional Reserve space.

the way I'm teaching it (and the way it's documented) a snapshot creation would fail if the space left after the snapshot blocks the currently active file system is less than the FR space.

The only way I can imagine the snapshots 'growing into the FR' is, when (at the time of taking the snapshot) everything was still OK, but afterwards the AFS grew (and thereby also the FR) and the snapshots are now using up space that the (now bigger) FR 'should have protected'. Can't blame the snapshot, though. It's just that by definition the FR is a moving target (until reaching it's maximum size of x% of the volume size as opposed to x% of the AFS size...).

Was that what you observed?

Sebastian

P.S. And yes, it's high level, but catchy...