Looking and reading at documentation regarding usage of ATTO SAS-FC bridge on Metrocluster I've always read about the need to have 4 bridges (as in the pic here) to connect and double the path to the stack SAS up to completation of one of this stack (seven DS4243 shelves).
I should setup a fabric MC where I will use both SAS and SATA disks in my DS4243 shelves: exactly as in the pic with one site with both 1xSAS+1xSATA shelves and the second site only with 1xSAS.
It now happens that configuring in PE/CE the needed hardware it give me EIGHT bridges for somewhere somebody says that this is best practice to have separate stack for SAS and SATA...what?!? This violates the principle that the SAS backend can do: a multiplexed one of four 3 Gbps links each
As far I know a stack SAS can be formed by both SAS and SATA disks mixing the DS4243 (up to ten shelves) and the ATTO manual too repeat this. No issue at all.
Why should my customer but other 4 ATTO bridges (9000 € each price list!)?
I think technically speaking you are correct.
PartnerEdge always used to have its own ways of 'best practices' around different things, like throwing in extra SAS HBAs on 3200 platforms.
I believe you can always build MetroCluster setup manually (i.e. as a non-MC configuration) & add additional components via "Add parts" - just run the outcome through your NetApp resource to make sure they are happy with that as well!
We recently raised exactly the same query with the NetApp Partner Helpdesk this week and received the reply below.
As an observation, if there is any single person within the channel whose advice I would trust, it is Radek.
Hope this helps,
.... The challenge is that the specification it spits out includes 8 of the new SAS Bridges, my assumption is that you can mix both SAS and SATA disk within a stack (pg 19 of the attached PDF) and would only require 4 of the SAS bridges in total.
yes, that's correct. SAS and SATA shelves can be mixed in the same stack, which would require only four SAS Bridges in total.
According to the MetroCluster Product Frequently Asked Questions,
MC (DOT 8.1.0 release) do support SAS and SATA drives in separate DS4243 shelves (but in the same stack). However, this is not supported as a configured system at POS. If someone wants a combination of SAS and SATA drives with DS4243 the same stack, they will have to order an MC configuration with SAS drives and then order DS4243 with SATA drives as add-on or vice-versa.
I hope this answers your question.
I'm in the process of buying a metro cluster solution. Today we've both SAS and SATA disks in the system. Our NetApp Partner advice us not to mix SAS and SATA in the same stack and came up with a solution including 8 ATTO 6500N bridges (exactly as the drawing in the gmilazzoitag posted earlier in the thread). But looking to the tech. spec the bridge has 2 SAS ports A & B. Can't I use both SAS ports in the bridge? port A for a stack of SAS disks and port B for a stack of SATA disks. Then it should be enough with 4 bridges. Or is that impossible?
There is even more interesting question here - why create separate SAS & SATA stacks?
In theory mixing different disk types in the same stack is supported, but there is a very vague recommendation from NetApp about "minimising the number of cross-over points". So if we have a mixed stack ending with the "wrong" disk type, a future expansion may be problematic due to creating a second cross-over point.
That being said, I haven't seen any firm statement saying 1 cross-over is fine, but 2 (or 3, or 4) are bad.
Any views on it?
Back in the days of FC backends you where forced to use seperate stacks for FC & S-ATA, so some ppl tend to stick with it and so does the QuoteEdge. Usualy a stack should be owned by one filerhead only as well. Both of these recommendations do not stand for SAS cabeling anymore. The reason why you cannot use both ports on a bridge is that ONTAP cannot handle more than 2 pathes to a single disk and for failover purposes if one of the bridges dies, so 4 is the minimum. Be aware that disk autoassign wont work in such a configuration, at least not now, maybe they will fix it in a newer ontap version.
Both of these recommendations do not stand for SAS cabeling anymore.
Well, that's what common sense tells us.
However, there is this paragraph in Storage Subsystem FAQ:
"HOW MANY CROSSOVER POINTS BETWEEN SATA AND SAS CAN I IMPLEMENT IN MY DS4243 STACK?
The term “crossover point” refers to the number of times a DS4243 storage shelf makes the transition from
SATA to SAS (or vice versa) within a single stack. Multiple crossover points per stack are supported.
NetApp recommends minimizing the number of crossover points when initially deploying a storage
configuration. As storage configurations grow, it is likely that additional crossover points will be introduced
into the storage configuration. This is acceptable; however, by reducing the number of crossover points in
your stacks, you reduce the variability of the storage configuration. As is true in all computer systems today,
increased variability can add difficulty when troubleshooting issues and also raises the possibility of
encountering unknown issues."
As woolly, as you can get
When using DS4243 disk shelves, electronics etc. are the same, so mixing shouldnt be an issue here. Besides SAS is a point to point protocol, so even mixing within 1 shelf shouldnt be an issue (other vendors do so).
But you do not need more then one crossover point anyway if you choose your shelf ids wisely. You can have 10 shelves in a stack and shelfids should be unique, so start with 10, 20 and 30.
if you expand:
So only 1 crosover needed.
SAS is a point to point protocol, so even mixing within 1 shelf shouldnt be an issue (other vendors do so).
Couldn't agree more - one million dollar question: why NetApp can't (don't want to) do it?
you do not need more then one crossover point anyway if you choose your shelf ids wisely.
In an ideal world - yes. But usually you have all types of constraints, like physical space in a rack, cable lengths, etc. Or you just have one "mixed" stack & can't introduce another one (e.g. due to having only 4 ATTO bridges ). I agree multiple cross-over points shouldn't make any difference for the overall health of the solution, yet this statement in the Storage Subsystem FAQ hints it is somewhat treated with suspicion.
For SAS/S-ATA i could image they dont want to add a gazillion possible options to the quoting edge tool resulting in less experienced users creating not field feasible solutions (as happens still alot with FAS2220 machines, eg 12 disks with snaplock in an ha-pair)
For the crossover stuff, i think they just dont have any experience yet if it causes errors or not or maybe they have a problem with acp in that mix.
I have a little bit different view why is netapp recommending 8 bridges.
I can not provide direct link to documentation, but as far as I remember, netapp best practice recommends to separate local and remote disk pools into different stacks. So because we are talking about metrocluster configuration which should utilize 2 disks pools in 2 stacks and you need 2 bridges per stack for redundancy, you end up with 4 bridges per location, 8 bridges for whole metrocluster solution.
In our production environment, we have installations with only 4 bridges, so we are mixing disks for different pools in 1 stack.