HPE Storage Users Group

Hey all, I found your user group and saw a great amount of knowledge and wanted to hit you up with this question. I've been educating myself on HP 3PAR and found this particular topic not well described. I like details, so the more the merrier. Two platforms I'm most familiar with are the HDS and Netapp lines. With the HDS VSP and Netapp FAS I know that the RAID5/6/DP writes are not written until parity is calculated in cache and then propagated to disk to alleviate most of the issues with read/write penalties for those RAID types.

I've read that 3PAR has a dedicated ASICs (similar to HDS) for the IO movement and parity calculations but I haven't read about if parity is calculated after written to disk or before. Any takers?

HP has a rather detailed resource in their 3PAR architecture white paper (link). On the subject at hand, this is about as detailed as I see them get:

"The HP 3PAR ASIC within each controller node performs parity calculations (for RAID 5 and RAID MP/Fast RAID 6) on the data cache."

For note, the data cache is connected to the ASIC, not the processor. It isn't explicitly written as to what the order is, but I do believe parity is written either before, or in-line as driven by the data in the cache.

Parity is worked out by the Asic before the data is written as a wide strip to the disks.

Kind of similar to netapp but it's done by the dedicated asic instead.

I figure by the terminology in that document that would be the case but it wasn't very clear. Thanks for clearing that up for me all.

So as an addendum to my original inquiry. I've read some HP SEs talking customers out of RAID5 because of read/write penalties. Why would they do this with most of that issue is alleviated by writing the parity at the same time as the data, unless there is a verification after writing to disk. I would see a read penalty there but not a very annoying double read and double write penalty with older RAID5 implementations. Yes, RAID10 would still be faster but the penalty for space especially with SSD disk. Anyone know of perf stats showing the differences between SSD in RAID5 versus RAID1/10?

Raid 50 is the preferred raid level for SSD, only if you're very heavily biased toward writes and really need to take the system to the limit would you use Raid10. The parity calculations are performed inline as the data cache is flushed to back end disk through the ASIC.

The reason some prefer Raid 10 in a write intensive environment (especially disk based) is there is no read before write penalty. With Raid 5 even though we do write back caching, some clever stuff at the ASIC, wide striping and also perform write coalescing in cache to try to ensure full stripe writes there are no guarantees that this can be accomplished. It all depends on the workload mix and since there's typically no way to predict that at the outset, from a sizing perspective we always use worse case on Raid 50. The ASIC provides scalability since the parity calculation is effectively free, but the real issue with parity based raid on writes is the cost in disk I/O, which can potentially look like the following.

1. Read of existing data
2. Read of existing parity
3. Write of new data
4. Write of new parity

So that's potentially 4 back end I/Ops per front end write vs only 2 for raid 10. Additional reads in a SSD environment are typically trivial but the greater the bias toward writes the more potential back end IOps you soak up, and all systems have a limit. BTW the above affects all parity based implementations to a greater or lesser extent, Raid10 will always, always be faster on writes no matter who's Kool-Aide (Raid-DP) you've been drinking.

Besides the above there are still plenty of people who's default best practice regardless of the product, capabilities or circumstances will always be Raid10 for everything, and no amount of persuasion will get them to shift that position.

Thank you for the additional detail. And yes, I fully agree with the RAID-DP stab.. They seem to think it doesn't matter, but in the end all parity calculations take computational power of some sort.

One more question, in regards to your statements: Why would the 3Par read the disk before writing what has been cached? I can understand reading after something is written to the media (sanity check as it were) but why before?

Remember this isn't a write anywhere layout, which also has it's own challenges (fragmentation, buffer space and deferred processes like garbage collection, defrags etc) so you can't always assume a write to empty space. But assuming the write coalescing in cache doesn't achieve a full stripe write (full overwrite) then there may be a need to preserve a portion of the old (on disk) data (partial write), hence the need to read the data first. The need for the parity read is based on a calculation of both modified and existing parity data as part of the new parity to be written to disk. As stated above there are all kinds of tricks to remove some of this overhead in a real system, but from a mathematical model the only safe method is to use the 4 x IOps worst case per write.

Great article here on Raid 5 here
http://rickardnobel.se/how-raid5-works/
And an equally good followup that discusses the specifics of the read modify write process
http://rickardnobel.se/raid-5-write-penalty/

Also I'd encourage you to take a look at the concepts guide http://h20565.www2.hp.com/hpsc/doc/publ ... -c03606434