Learn more about SanDisk and Kaminario indexing 100 million emails and attachments at a processing rate of 7.2M messages per day
SanDisk, a top-tier provider of flash storage technology, with multinational businesses across retail consumer goods, OEM relationships with phone and computing device manufacturers, and a provider of enterprise data center and cloud solutions, faced mounting eDiscovery challenges that, if not managed properly, would put the company at risk with time-consuming litigation matters, non-compliance of Records and Information Management (RIM) Governance, and eDiscovery-related cost overruns.
SanDisk Company Profile:
To alleviate and address these eDiscovery challenges, SanDisk chose to deploy an end-to-end solution comprised of eDiscovery and global archiving software running on enterprise-class flash storage.
SanDisk/Kaminario: indexed 100 million emails and attachments at a processing rate of 7.2M messages per day
The SanDisk eDiscovery solution has improved the time and cost spent by the legal department in managing eDiscovery issues and enhanced the legal department’s ability to mitigate litigation risks. As part of the solution, a centralized global archive was also deployed to help employees retain, manage and access email.
With over 5,900 employees supporting SanDisk business in over 100 countries for consumer products, OEM relationships, and enterprise & cloud solutions, SanDisk eDiscovery challenges are complex and diversified.
SanDisk, founded in 1988, and with over 25 years of operations, accumulated eDiscovery challenges where indexing, searching, sorting, and archiving processes have become time-consuming and operationally expensive.
SanDisk employees send and receive more than 500,000 to 1 million emails per day. SanDisk IT maintains a corporate email archive that stores over 21 Terabytes (TB) of data. The elimination of local Microsoft Outlook Exchange PST File archiving also required the need to transition to a centralized approach for compliant data retention.
SanDisk email, at 25,000 pages per gigabyte (GB), is more than 0.5 billion pages of documents, which reside in the SanDisk email archive. Costs of searching and reviewing an email archive of this particular size needed to improve due to traditional approaches would not be able to perform at the current volume.
Cost reduction was necessary, considering an attorney generally takes 167 hours to read one GB of email (about 20,000 emails).
The SanDisk eDiscovery solution needed to implement the following:
Initially, SanDisk IT Department planned on the deployment of traditional spinning hard disk storage for the SanDisk eDiscovery infrastructure. After performance testing and eDiscovery trials in preparation for the complex data migration, the traditional HDDs proved to not meet the performance requirements of SanDisk and would increase the migration risk and considerably lengthen the project implementation timeline.
Based on trial runs, SanDisk IT estimated that to load, index and archive these quarter of a billion emails (plus the attachments) onto traditional HDD storage, an unacceptable lead time of 50 days (processing rate of 2 million emails per day) would be required.
HDD storage was deemed unusable, as its use would jeopardize the release of the eDiscovery capabilities to the SanDisk legal team, and eventually, the company-wide deployment of the powerful email search and archive functionality.
To accelerate the eDiscovery timetable, SanDisk CIO, Ravi Naik, assigned a team of engineers to evaluate the performance of several all-flash storage arrays that would serve as an alternative to HDDs for this particular ultra-high performance application. The hypothesis that drove this decision was that low latency (< 0.5 millisecond) flash storage would virtually eliminate the immense email indexing challenge.
SanDisk IT and SanDisk attorneys deployed HP Autonomy eDiscovery solution so they could migrate data from Microsoft Exchange and the legacy EVault archiving platforms into HP Autonomy.
In total, more than 250 million messages were processed — and then eventually reduced to 100 million unique indexed messages (generating 11TB of data) on the Autonomy platform.
Kaminario all-flash storage array was selected which comprises thirty two 800GB capacity SanDisk Optimus Extreme™ Solid State Drive (SSD) from SanDisk. The combination of this enterprise-class SSD with the Kaminario all-flash array offers predictable, sustained performance, superior reliability and high endurance. The Kaminario architecture is scalable and features proprietary technology, including N + 1 redundancy to ensure the high resilience and uptime of SSDs.
With the SanDisk/Kaminario all-flash solution, the data dilemma was solved. Against a production workload, HP Autonomy indexed 100 million emails and attachments in just 12 days, averaging 7.2M messages per day—a result that surpassed even the most optimistic estimates. That processing time was a fraction of the 50 days (two million emails a day) that it took conventional HDDs to complete the same migration task. The all-flash array storage solution achieved ultra-high performance, high IOPS (input/output operations per second) rates, low latency and high resilience. The architecture was optimized for data protection and it proved to have high reliability and high availability.
The world of digital content grows exponentially every day. From mobile devices to data centers, SanDisk delivers the peace of mind that data is readily available and reliable, even in the most challenging environments.
It’s all part of delivering on our mission to enrich people’s lives through digital storage anytime, anywhere. Since 1988, companies and consumers around the world have counted on our storage solutions in many areas, including:
Data Center - In the data center or in the cloud, our leading-edge flash technology enables fast, reliable access to mission-critical data.
Computing - Our solid state drives offer energy-efficient, compact, and durable alternatives to traditional hard-disk drives for desktops, laptops, and ultra-thin PCs.
Mobile - High performance flash storage is essential in smartphones, tablets, and other mobile devices. We deliver embedded memory solutions to every major mobile device manufacturer.
Consumer Electronics - From cameras to USB drives to MP3 players, the quality and performance of SanDisk technology is in many of your favorite devices.
Specifications are subject to change. © 2014 - 2016 Western Digital Corporation or its affiliates. All rights reserved. SanDisk and the SanDisk logo are trademarks of Western Digital Corporation or its affiliates, registered in the U.S. and other countries. Optimus and Optimus Extreme is a trademark of Western Digital Corporation or its affiliates. Other brand names mentioned herein are for identification purposes only and may be the trademarks of their holder(s). Kaminario_eDiscovery_short – 06.22.2016. Western Digital Technologies, Inc. is the seller of record and licensee in the Americas of SanDisk® products.