Intelliflo Case Study
Intelliflo (www.Intelliflo.com) has been providing information technology services to financial services companies since its formation in 2004. Its leading web-based business management software, Intelligent Office (iO), helps large and small financial businesses improve efficiency and increase profits in an ever-changing market. Intelligent Office supports more than 1,600 firms and 14,650 users with assets under management of more than £243 billion as at 31 December 2015.
In March 2015 Intelliflo was listed among the top 25 best-performing privately owned technology companies in the UK mid-market. The list is compiled by Megabuyte’s independent and highly regarded research team and is based on financial performance and long-term potential.
Intelligent Office traffic and data has gone through staggering growth in recent years. This growth has been driven by the delivery of expanded functionality in each release, the introduction of a mobile platform for end retail investors, and a consistently growing customer base. The application executes 10,000 database transactions per second during peak hours and handles 2.5 million web requests and as many as 250 million database queries each day. The application is highly data-driven and relies heavily on reading and caching various data elements to provide a richer application experience. Query response times significantly affect a given adviser’s productivity and the time spent doing administrative work on the site. During business hours, the data read and write ratio is 80 percent to 20 percent respectively. A massive amount of data is moved, processed, replicated, and transformed in the background. This enables users to access their data in near real time through report, extract, search, and business intelligence tools.
Romanth Nirmal, Database Manager, Intelliflo
Intelliflo was seeking a new solution to address a number of challenges. First, a new product introduction, Personal Finance Portal 2.0 (PFP) had the potential to reach millions of end retail investors, compared with the 14,650 financial advisers using iO. The company expected unprecedented traffic to the web site 24x7 upon the launch of PFP and needed additional headroom to provide the burstable capacity for the additional database traffic.
Second, the current storage area network (SAN) was becoming extremely busy with growing traffic. The Intelliflo engineering team wanted to prepare the SAN for the IOPS and throughput performance demands that were projected for large upcoming releases and anticipated customer growth. The SAN often showed large spikes during heavy batch processing and topped out at one gigabyte of throughput as databases, the VMware infrastructure, and the documents volume competed for I/O.
As a result of these challenges, the Intelliflo database team was tasked with taking their database infrastructure to the next level. “We needed our database response times to be lightning fast and our infrastructure to provide significant burst ability in terms of IOPS and throughput,” explained Romanth Nirmal, Database Manager at Intelliflo. At the same time, Intelliflo decided to withdraw their database footprint from the expensive SAN to avoid costly upgrades in the future, considering that even a replacement SAN may not be able to continue to deliver the required performance. “Our approach was to provide database traffic management across both the production and disaster recovery sites. Microsoft SQL Server 2014 AlwaysOn and InifiniBand, together with Fusion ioMemory PCIe cards, were the clear answer to the challenges we were having.”
Rob Walton, Chief Operating Officer, Intelliflo
Romanth had downloaded SanDisk’s SQL Welcome Kit in early 2015 to explore— through case studies, blogs, and white papers—how Fusion ioMemory could support the Intelliflo infrastructure. Intelliflo had been running Microsoft SQL Server 2008 R2 on HP servers and wanted to move to SQL Server 2014 on new HP servers with a smaller footprint. The goals were to deliver greater application performance to achieve better customer experience; consolidate core count by increasing CPU utilization and efficiency of fewer cores; and ultimately realize SQL Server licensing cost savings.
Although the Intelliflo team seriously considered a competitor’s product, the ioSphere® Management Suite software proved to be a key differentiator which allowed the team to monitor both the performance and endurance of the Fusion ioMemory PCIe cards throughout their lifecycle. Additionally, the higher capacity of Fusion ioMemory PCIe flash storage enabled a solution to be architected such that larger multiples of smaller disk-attached storage could be avoided—thus decreasing overall complexity. Furthermore, SanDisk could provide on-site and responsive technical support as both the developer and manufacturer of the product.
After a successful initial Proof of Concept evaluation supported by the SanDisk Enterprise team, Intelliflo procured nine Fusion ioMemory 2.6TB PX600 PCIe application accelerators to increase application performance and customer satisfaction. With a database approaching five terabytes in size, the total dataset required for the four servers (one master and three slaves) was 20TB. Eight of the cards have been deployed in the production environment, while the ninth card serves as a standby ‘swap-out’ spare.
The infrastructure enhancements consisted of a four-node active/passive HP server cluster, on HP ProLiant DL380 G9 servers with 18 cores—a substantial reduction from the previous core count of 48. Two Fusion ioMemory PX600 2.6TB PCIe cards were installed in each node. Windows 2008 and Microsoft SQL Server 2008 R2 were upgraded to Windows 2012 R2 and Microsoft SQL Server 2014 with AlwaysOn availability groups, while an InifiniBand card between the two database servers ensured ultralow latency. The team also removed complex transaction replication topologies, and moved user searches and data warehouse pipelines to leverage AlwaysOn read slaves. Last, the team enabled Hot + Hot traffic to both the production and disaster recovery sites.
“We recognize that financial advisers see the most value in spending time with their clients, not sitting in front of a computer screen, which is why we’ve invested in technology that significantly speeds up the time it takes to open different functions,” Rob Walton, Intelliflo’s COO told us.
Romanth Nirmal, Database Manager, Intelliflo
SAN HP EVA P6400
The re-architecture and database modernization project was carried out to increase efficiency for iO users—by reducing the amount of time they need to spend interacting with the wide range of available tools and services. As a result of the implementation, Intelliflo has been able to improve web and database application response times, as well as to reduce batch processing time—by as much as 90 percent.
“Looking at our most critical database instance of our ten SQL Servers, the overall application wait on database response on a ten-minute sample interval experienced a 75 percent reduction in wait time,” commented Romanth. “The CPU decreased from 40 percent to eight percent, and disk latency became a thing of past. The read/write disk latency literally went from 35 milliseconds to zero.”
The following Ignite charts illustrate the results outlined by Romanth Nirmal.
Web Response Results
Similar results were detected with respect to web site response times. “Splunk shows that the number of web requests taking longer than one second diminished by 42 percent,” Romanth continued. “Requests taking less than one second became even faster. As a net result, most of the user-facing web pages got a 50 percent performance boost.”
Various batch jobs also saw significant improvements in terms of processing time— up to a 90 percent improvement. Because processing time was reduced, report and fund price updates had a smaller impact on end-user traffic.
With respect to the network, the team realized a 99.98 percent reduction in latency between the production instance and old mirroring instance. “A 300 millisecond latency was reduced to just 40 microseconds,” said Romanth. “This is a massive enabler in terms of introducing Synchronous Read Slaves without compromising user experience. Without InfiniBand, the traditional network card would be unable to harness the transaction log influx that is produced by the Fusion ioMemory cards.” The new architecture has also extended the life of the SAN. By moving the databases off the SAN, application servers are given more I/O priority and greater performance across the architecture.
“The response from our clients to the enhanced service has been instantaneous, with many advisers reporting positive feedback about the increased speed of delivery and commenting on how it’s saving time for them and their support staff,” said Walton. “Delivering these improvements is just one of the ways Intelliflo continues to lead the field through innovation, enabling our clients to work with maximum efficiency.”
When asked about future plans, the Intelliflo team is optimistic. “With the additional performance headroom that the Fusion ioMemory cards have provided, we don’t need to worry about any performance constraints resulting from more client demand,” said Romanth. “We are able to add additional feature sets with confidence.”