Currently Being Moderated

Deduplication and The Long Arm of the Law

Posted by parks in Ask Dr Dedupe on May 13, 2009 12:05:02 PM

Check out this press release and you'll see a couple interesting facts:


  • NetApp has delivered deduplication to over 7,200 customers and over 37,000 systems
  • Cohasset evaluated deduplication with SnapLock and found that it meets SEC regulations for electronic records management


Each of these items is a milestone of sorts.  Having been part of the original dedupe launch team back in the Spring of 2007, I can tell you that none of us would have imagined that within two years we would be approaching 40,000 deployments - in fact our "stretch" goal was around a thousand systems in the first year - we beat that by about 10,000 systems.


The success of deduplication at NetApp is a testament to the vision of our development team - they recognized yet another way to leverage the intelligence of Data ONTAP, and to our customers, who were willing to put their faith in our design and prove that production data could be dedupe'd just as easily as backup data.


The second part of today's announcement is Cohasset's evaluation of dedupe for compliance records.  Cohasset Associates is the acknowledged authority in electronic records management.  For decades, storage vendors have turned to Cohasset to make sure their archival and compliance products passed the sniff test. 


To validate products, Cohasset leans heavily on SEC rule 17a, probably the most stringent test of any government agency for legal record compliance.  The SEC is interested in two key criteria - A) Is the record stored in a state that cannot be altered by users? and B) Can the record always be recovered back to its original state?  When Cohasset evaluated NetApp deduplication, there were two things that stood out that eventually resulted in two thumb's up from them:


1) Before deduplication - a byte-level comparison takes place to eliminate data corruption as a result of hash collisions


2) After deduplication, the only change to the original data are re-addressed data pointers, which cannot be altered or otherwise manipulated by users, and does not involve any external hash tables or containers.


Deduplication of legal records has been a controversial topic for some time.  Hopefully our announcement today makes this topic is somewhat less controversial...and before I close I'd to make sure I say thanks to the thousands of NetApp dedupe users that have made this the fastest-growing feature in NetApp history!


Filter Blog

By author: By date:
By tag: