This article originally appeared in the June 2011 issue of Customer Interaction Solutions

IBM (News - Alert) has helped organizations manage data for many decades by providing them with relational databases. But in the last year or two there’s been a data boom, and lot of the data cropping up as part of that doesn’t fit neatly into relational databases. That makes it more difficult for organizations to locate and leverage data effectively and efficiently. To address these new realities and requirements, IBM has committed to spend $100 million to research technologies and services that will enable clients to manage and exploit data as it continues to grow in diversity, speed and volume; rolled out some related new products; and done several acquisitions.

These new initiatives are important, given 83 percent of 3,000 CIOs recently surveyed indicated that applying analytics and business intelligence to their IT operations is the most important element of their strategic growth plans over the next three to five years, according to the 2011 IBM Global CIO Study.

However, while we keep hearing about how there’s a gold mine of data out there just waiting to be uncovered, the fact that the volume of data is already huge and is growing at an unprecedented rate, and the fact that much of that data is unstructured, can sometimes make it very difficult to locate specific data and identify trends. IBM estimates 2.5 quintillion bytes of data are created by mobile devices, sensors, social media and out sources on a daily basis, and between 2011 and 2016 enterprise data is set to increase more than 650 percent – with 80 percent of that data expected to be unstructured.  

"Navigating big data to uncover the right information is a key challenge for all industries," says Arvind Krishna, general manager of information management at IBM Software Group. "The winners in the era of big data will be those who unlock their information assets to drive innovation, make real-time decisions, and gain actionable insights to be more competitive."

That’s why last year at about this time, IBM introduced a big data platform that helps enterprise customers handle large loads and different varieties of data, which may be structured or unstructured, coming from social media, machine data, data from call records (in case of a telco), and/or other sources, says Anjul Bhambhri, vice president of big data products at IBM.

InfoSphere BigInsights and Streams software introduced by IBM last year enable organizations to “integrate and analyze tens of petabytes of data in its native format and gain critical intelligence in sub-second response times,” according to IBM. The company says its BigInsights software “incorporates Watson-like technologies, including unstructured text analytics and indexing that allows users to analyze rapidly changing data formats and types on the fly.” InfoSphere Streams software, meanwhile, analyzes incoming data and flags in real time changes that may signify a new pattern or trend. This addresses such data as blogs, Twitter (News - Alert) feeds, stock information, video and more. IBM announced this spring that it is has expanded its big data platform to run on third-party distributions of Apache Hadoop, beginning with Cloudera. 

There are many current and potential uses for these kinds of solutions, and they go far beyond marketing and sales applications that might first come to mind when you think about this kind of thing. For example, the University of Ontario Institute of Technology is leveraging this IBM technology in its work monitoring, doing research on, and comparing the health of premature babies in Australia, Canada and China.

In addition to the new, Apache Hadoop-based analytics software, IBM last year came out with 20 new service offerings that provide business and IT people with predictive analytics capabilities so they can access, configure and design their operations to take advantage of their data. Specific services under this umbrella include Cloud Workload Analysis, which helps IT staff prioritize cloud deployment and migrations plans; Server and Storage optimization and analysis tools, which promise 80 percent faster implementation times; a Data Center Lifecycle Cost Analysis Tool, which can reduce total data center costs by up to 30; and Security Analytic services, which identify known events and automatically handle them without human intervention.

While IBM already had a strong foundation in the analytics space, the company has invested heavily in the data analytics space and continues to bring new organizations and solutions into the fold. As of last year at this time, IBM had invested more than $14 billion in 24 analytics acquisitions. And in the last few months it has acquired at least two more companies in this category. Those companies are Varicent and Vivisomo. 

The Varicent solutions bring analytics to front-end business operations including finance, human resources, IT and sales departments. Vivisimo is known for its discovery and navigation software, which helps companies access and analyze big data. Vivisimo has more than 140 customers in the consumer goods, electronics, financial services, government, life sciences, and manufacturing verticals, among others; its clients include such large organizations as Airbus, the U.S. Air Force, Procter & Gamble, and LexisNexis (News - Alert).

"IBM sees enormous opportunity to apply advanced analytics to sales functions that can help organizations uncover new, untapped growth opportunities,” says Bhambhri. “We anticipate business analytics revenue for IBM will reach $16 billion by 2015.”

IBM reported first quarter revenue in its software division increased 7 percent and profit was up 12 percent; IBM's business analytics revenue was up 14 percent. Analytics and big data are certainly a good place to be right now, as IDC estimates the market for big data technology and services will reach $16.9 billion by 2015. As discussed, IBM is clearly moving on that opportunity both through internal development and via M&A.

“The Varicent acquisition – along with Algorithmics, Clarity Systems, Netezza, OpenPages, SPSS (News - Alert) and Vivisimo deals – supports our long-term strategy of strengthening our portfolio of big data and business analytics solutions and industry expertise."

Bhambhri offers an example of how a retail organization can uncover new, untapped growth opportunities using the services and tools that IBM provides. All retailers get point-of-sale data, which helps brands and retailers understand what kinds of things shoppers are buying so they can do proper ordering and inventory. However, says Bhambhri, there’s always lag time between the time the sale happens and the time the point-of-sale information becomes available; it’s typically around 72 hours. That lag time wasn’t much of an issue in the past, but it can prevent retailers from moving quickly enough to act on trends in today’s age of the connected consumer. For example, let’s says a YouTube (News - Alert) video talking about how drinking juice is good for you went viral. That could trigger a temporary increase in the sale of juicers, so retailers might want to consider quickly putting out promotions for their juicers while the interest in such products is high. If they waited the 72 hours or more to receive and then analyze customer buying habits, retailers can completely miss out on this kind of opportunity.  

“Specifically for retail, bringing in social media is going to be very key,” Bhambhri says. “Not doing it can hurt the business.”

Of course, it’s no small task to comb through and make sense of data derived from social media. Bhambhri says wading through social media data, which she refers to as noisy data, to find valuable information can be like finding a needle in a haystack. That’s because a single discussion on social media may mention 50 topics, but only one topic (a mention of a specific product, retailer or buying intention) about which the retail would have an interest. The beauty of big data, however, is that somebody can capture all that data and use multiple applications to extract useful information from that same data and, if desired, combine it with data from other sources.

Bhambhri notes that organizations like retailers already have a lot of existing data repositories or applications, like CRM, so it’s important to be able to be able to leverage those resources as well. By adding Vivisomo, which has federated capabilities for structured and unstructured data, IBM can bring in more big data sources and allow for more analysis, she says. However, she adds, IBM believes bringing analytics to the data (even if that data is in multiple databases or locations) rather than bringing the data to the analytics can render faster results. That’s especially important in this age of big data, she says.

While IBM offers a full suite of big data and analytics hardware, software and services solutions, Bhambhri notes that customers don’t have to buy everything from IBM, and that IBM will support analytics packages from other suppliers.