How to Survive the Holiday Data Tsunami

Enterprises are gearing up for online sales during the 2014 holiday season. Deloitte forecasts 13.5% to 14% year-over-year growth in online holiday sales. A recent National Retail Federation (NRF) survey shows a record 56% of holiday shoppers plan to make online purchases this year with the average shopper expecting to do nearly half (44.5%) of their entire purchases online.

Mobile will account for a significant portion of this growth as emarketer predicts that 19% of U.S. retail ecommerce sales will come from purchases made on mobile devices. With heavy spikes in data traffic in-store, companies must make sure their complete online commerce operations are up for the task.

Despite their original intentions, consumers’ interest in shopping online can be sabotaged by the user experience. Heavy transaction loads can lead to delays in system processing or system crashes, causing frantic holiday shoppers to switch to other sites, even abandoning full shopping carts if response times fail to meet expectations.

Likewise, long latency times can cause customers to unwittingly order out-of-stock merchandise or adversely affect shipping times. According to Forrester Research, last year 15% of online holiday shoppers didn’t receive their packages on time due to the inability of IT systems to handle increased volumes of orders.

Systems integration is a critical component that is often overlooked when deploying ecommerce and mcommerce systems. Online storefronts don’t stand alone. They need to be integrated with the company’s back-end systems and external services to support the complete process, from inventory management to payment processing, accounting, shipping, customer service, returns and even analytics.

Many organizations choose point-to-point API or database integration as an afterthought, rather than strategically selecting a robust process-based integration platform that knows how to deal with peaks and tremendous load in a secure fashion. By using a robust application integration tool, businesses can ensure that their ecommerce solutions are robust, secure and scalable to manage even the highest transaction loads, so they can maximize holiday sales and customer satisfaction.

So prepare for the holiday season by making sure your integration platform meets these five critical requirements:

Vendor Neutrality. No integration solution exists in a vacuum: by definition its value lies in its ability to connect different systems. A vendor-agnostic platform allows you to connect your best-of-breed ERP and CRM along with a wide range of third-party services, such as exchange rates, credit checks, and order fulfillment services. Make sure that your integration solution provides optimized connectors for your specific ERP and CRM systems and can manage and control APIs for all of your relevant systems. Middleware solutions that rely on one specific type of integration architecture can lock you into manual programming at times when you most need to be able to maintain and adjust system integration flows.

Scalability.  With increases expected in traffic you need your system to be able to grow dynamically and automatically to process multiple simultaneous requests. With an in-memory data architecture, when there are peaks in traffic paired nodes of memory and processing are available to keep the system running smoothly.  As processing requirements increase, the management system automatically recruits more nodes, adding scale elastically when it’s needed.

Cloud Readiness. Support for different architectures, including any combination of on premise and cloud integrations to support integration between systems where ever they physically reside.

In-Memory Computing. By storing data and business processes in-memory, in-memory data grids transfer data much faster and provide robust fail over capability that can guarantee message delivery in the event of a node or system failure. An integrated process workflow running on an In-Memory Data Grid architecture can easily access, process and present real-time business information since it is not dependent on the data processing of any one system. Don’t forget that if integration between systems isn’t real-time, like with batch processing at end of the day, a business can end up with dissatisfied customers and heavy loads on customer service responding to availability and shipping issues.

Security. Effective security including PCI compliance is not only essential to avoid liability; it is also a critical component of consumer confidence and reputation. Ability to run integration services from behind the firewall will provide an added measure of security when compared to cloud-only solutions. Don’t ignore the security of your integration layer where data can be exposed while transferring between systems. Look for integration solutions that include authentication, rights management and encrypted data transport.

A system integration ecosystem with the necessary built-in flexibility and capacity can help you maximize online sales by enabling you to successfully manage the data deluge this holiday season and as your business continues to expand. A robust integration platform will not only enable rapid integration between all the applications, services and databases in your e-commerce /m-commerce system (regardless of where they are located), it  will also make sure that the entire system performs optimally, so holiday orders will be fulfilled successfully and companies will have happy customers earning repeat orders down the road.

Glenn Johnson is the senior  vice-president of Magic Software Americas.