I think I'm more confused than when I started
I think I'm close to understanding but something between what you're saying and what I thought from discussions with others aren't matching up.
Below are some scenarios based on this discussion. I've tried to explain what's going on as best as I could but there's a couple scenarios which I don't know the answer to.
In all cases I'm most interested in what happens when there's a drive failure. Let's assume all my volumes and thin/dedupe so if something needs to grow I need to be able to allocate space. If there's a controller failure I assume there's no issue other than a performance hit.
- 2 nodes, 32 drives, 14+2 > no express layout because each node owns 16 drives. 14+2 requires 16 drives which doesn't meet the requirements of express layout (per 3par docs, a set size needs >50% of available drives on a node for express layout to be used). If a drive fails I'm not sure what happens in this case. Can CPGs/VVs/LDs continue to grow without any issue? I see a node not having enough drives in this case so I'm in big trouble.
- 2 nodes, 16 drives, 14+2 > each node owns 8 drives so to create a 14+2 the node will need to share with its partner hence the 3par will do an express layout. If I lose a drive I'm in big trouble because I need 16 drives for a full stripe but only have 15 in the system. I won't be able to grow any CPG/LDs/VVs in this situation and the array will set everything offline the next time things need to grow to avoid corrupting all my data.
- 2 nodes, 24 drives, 14+2 > each node owns 12 drives. 14+2 is > 50% of available drives on the node so express layout is used. Because 16 doesn't go into 12 evenly I'll have chunklets overlapping on the disks which isn't ideal in terms of performance. Will I have a problem if I lose a drive?
The optimal set size in this scenario would be 4+2 (or 5+1) as they divide evenly by 12 and there's no danger in terms of grown if a drive were to fail using these smaller set sizes.
- 4 nodes, 32 drives, 14+2 > each node own 8 drives so express layout is needed. Now this is where I'm getting confused. Is the node pair 0-1 and 2-3 or 0-2 and 1-3? (ie. is the partner node in the same chassis on the other chassis?)
To make the 14+2, each node takes 8 drives it controls and 8 drives its partner controls.
If a drive fails and things need to grow, will I be ok will I have a problem? From your reply it seems like things will be ok but performance might be degraded since only 1 of 2 nodes can service the i/o if an LD/CPG needs to grow?
- 4 nodes, 32 drives, 6+2 (or 7+1) > no express layout as each node owns the exact number of drives needed to build the raid set. If there's a disk failure will I have a problem? In my mind I see the node not having enough drives to meet the raid size but I think the fact that there's a partner node might come into play here so I'm not sure what the answer is.
A couple other questions that came to mind trying to understand all this:
If things weren't thin would all this be a non issue since the space would have been allocated already?
What's the point of having all the spare chunklets if it seems like the loss of a disk can take down the array? I would think the spare chunklets would take over for the disk and the raid stripe would be created using all the spare space set aside (since the 3par puts aside enough space space to cover an entire disk)
Thanks