May 2001

Listening and understanding are integral to the health of any relationship, and the relationship you have with your customers is no exception. When you observe your customers' interactions with your company, you are listening to their needs and wants, and by doing so you can understand them better. Armed with this information, you can improve the health of your business. Put simply, data mining helps you understand who your customers are by how they behave. With this information, companies are able to target marketing promotions to drive revenue and build market share. "It is now conventional wisdom among marketers, and IT personnel who support them, that investments in customer data have a demonstrable return," said Phillip Russom, director of data warehousing and business intelligence, Hurwitz Group.

Often, companies are surprised by who is really driving their business. Let's look at Wine.com, for example. Wine.com intended its Web site to be a marketplace for educated wine enthusiasts. Research, however, proved that the majority of the company's customers were looking for the proverbial "how-to" on what wines to buy. The good news for Wine.com was it was alerted through its data-mining initiatives to a potential market share that was twice as large as originally expected. Additionally, data mining provides companies with the ability to specifically target customer segments and their respective needs. Looking ahead, Wine.com could develop more successful marketing campaigns designed to attract educated wine drinkers, on top of maintaining their less-informed, yet larger, customer base.

The development and implementation of a successful data-mining solution can be broken down into five steps: setting goals, data collection, data preparation, analysis and prediction, and measurement and feedback.

Goals
As with any project, the first step is to set reasonable and measurable goals. For a broad data-mining initiative, these goals should be phased in over time and start off simple. Shooting for the moon on the first project is risky. Companies don't change overnight and neither do their business processes. So, let's examine the basics:

Customer profiling and segmentation. Break down your customer base into recognizable and manageable groups. By profiling your customers and dividing them into segments, you can target those groups more specifically to their needs. Additionally, you can measure actual behavior of those groups over time against expected results. Segmentation can be done using cluster analysis (demographic, psychographic and behavioral characteristics) and should be done at least once a year or whenever significant changes are made in your company's business model. Try to avoid demographic-only segments and build segments based on behavioral characteristics. These will better define who your customers really are.

With customer segments in place within your database, you can now gain additional knowledge of your business and your customers with every promotion by measuring response by segment, and comparing it to your expected results. Referencing back to Wine.com...would a targeted promotion to educated wine drinkers have been well received by the majority of their customer base?

The next step is to ask more specific questions about your customers. Who are my best customers? Can I acquire more of them? How can I get more business from my existing customers? Conversely, who are my worst customers? Can I salvage that relationship? Should I? Is there some other product or service I can provide? These are more specific, so let's group them into applications:

Retention and attrition. It is inevitable that businesses lose customers. You can determine those customers with the highest propensity to slow or discontinue business by profiling those customers who have left in the past. This is just another application of a likelihood model. A retention model examines the potential for a customer to be retained after some event. An attrition model considers the likelihood that an active customer will become inactive. These are proactive goals for companies to set. With this information, customers can be actively promoted through direct marketing or can be flagged for special attention when interacting with your company via customer service or your Web site.

Risk avoidance. How can you avoid acquiring unprofitable customers? Data mining can predict, with some accuracy, which prospects will become customers. However, it is also particularly helpful to determine which will be profitable customers. It is also important to recognize that some new customers will default and force your company to write off losses. Therefore, risk management and marketing groups must work together when new business programs are put in place. While marketing is looking for increased revenue, risk management examines the new business that turns out to be unprofitable. Both disciplines should be evaluated together and result in marketing efforts that are based on the insights gathered. Look to your risk management group to see how they can participate proactively in your next marketing campaign. Use models that are built to identify profiles of new customers who will default (another type of likelihood model) and existing customers who will turn bad. These models examine purchase behavior, payment behavior, credit profiles and other factors.

Cross-sell. Cross-selling and upselling are central to successful CRM. Interacting with your customers is a prime opportunity to market additional products or services. What have our customers been buying? What are their interests? What products or services have they been looking at or inquiring about? We can look at the data in different ways. First, what products or services have the best cross-sell potential judging from past sales data? In other words, which products have been purchased by a single customer most often? Next, what are the customer profiles that best match various product groups? When you identify customers who have a tendency to purchase like products, but you haven't approached them yet, you have a great opportunity to promote that product to your existing customers. That's what CRM is all about. We've seen Amazon.com do this very well. When you visit their site, you'll see, "Welcome, here are books and CDs that might interest you." This can be accomplished by modeling product purchases by individuals over time and grouping them.

Profitability. Profitability was mentioned earlier, but how can you measure and affect the lifetime value (LTV) of your customers? Any basic data mining analytics package will include ways of calculating LTV and using it to segment customers. Even recency-frequency-monetary (RFM) models, which are founding members of customer analytics, will contribute greatly toward understanding your customers so that you may communicate with them more effectively. Understanding the key components of profitability will help you understand when and if your customers will become profitable.

Shopping patterns. With electronic storefronts, it is now possible to study your customers' actual behavior. This information can be used to personalize content for the customers as well as to understand your company's Web site efficiency (e.g., Where are people leaving the site? At what point are shopping carts abandoned?). Purchasing behavior was touched upon earlier, but shopping patterns include what customers looked at or considered purchasing. Technology is now available to do this, including tracking the customer's path through the site to what was in a discarded shopping basket. Tracking this information is far more difficult for a brick-and-mortar company. With this knowledge, your marketing efforts can respond to request-for-inquiry and other nonpurchase communications from your customers.

Data Collection
Types of data. First, you need data to build any segmentation or predictive model. You probably have some basic purchase information such as what products were purchased, when and at what price. You may even have some demographic information, but probably not unless you purchase enhancement data. If you want demographic profiles, you'll need demographic data (age, income, location, etc.). These data will also allow you to build better predictive models. You should also maintain promotional history (which customers received which offers).

Which data? The data you study should match your goals. For segmentation models, you'll need to sample the customer base that is representative of your current business. However, predictive models require some prior history of behavior. If we want to predict the likelihood of responding to a particular offer in an effort to acquire new customers, then we need a prior campaign to reference. This model will look at determining which variables separated respondents from nonrespondents and then transfer those findings to your new prospect pool. To make that work, the prior promotion and your new promotion need to be the same or similar. Some of the factors that can vary the response to a promotion include offer, price, copy, channel and season. Care is needed to keep the historical data as unbiased as possible.

Data Preparation
Data quality. Most resources say that data preparation is 80 percent of building a model. Data problems lead to the decrease in value of any customer data warehouse, on top of impeding the value of any model. Data quality issues come in many forms. Missing data is one example. What if you have age demographics for only 20 percent of the file? You'll have difficulty profiling your customers by age. Another problem is poorly coded data. Your customer database needs to have standards set regarding date formats, text case and redundant codes. Over the years, data may have been added to the database from several different input sources. If your data were not standardized before loading them into your data warehouse, take the time to do it. The investment is well worth it.

Data preparation. Different models require data in certain formats. Some require all data to be continuous and ordinal. Others require all categorical data fields or need binary constructs. Some tools do this special preparation of the data for you. In simpler methods, you must do this yourself. Either way, the quality of the statistical model will depend on this data preparation.

Analysis And Prediction
Building the model. After collecting the necessary data to meet the intended goal, different methods are used to construct good models. At the root of these are statistical methods ranging from classification trees, like CHAID, to regression models to neural networks. Any of these methods include sound techniques to fit any goal. Often, the trick is using the appropriate method to fit the goal, and a good analyst will know which to choose. They will also know how to prepare the data. Again, some tools take the mystery out of this, but it is recommended to avoid total "black box" methods that eliminate the contributions of smart people in your office who understand the data intimately and know your business and customers.

Testing the model. Test! Test! Test! When you build a predictive model (e.g., Who is most likely to respond to my campaign?), it should be based on a prior campaign in which you know who actually responded and who didn't. After you build the model, how do you know it will work? To test this, remove a sample of the prior campaign, or a "holdout." Some of that holdout will have respondents and non-respondents. After you build the model, you score the holdout and then look to see how well it identified the actual respondents. Your model has problems if it doesn't predict well. It may also have problems if it predicts too well (there may be bias in the model).

Score. After the model is built, the prospects are then scored. This means flagging the prospects with a value to determine what their predicted behavior might be. Often, several scores are combined to make appropriate campaign selections such as special promotions to long-term customers who are most likely to renew their subscription, for example, and pay in full up-front.

Measurement And Feedback
How well did your campaign perform? In an e-mail campaign, you can evaluate the program as it is happening. To understand results, you look at response (or whatever the goal behavior is) against what you expected. In a simple response model, you expect that the groups with the highest predicted likelihood to respond actually do so at a significantly higher rate than the average. Are they? Worse than expected? Better? This information is then returned as a component of your promotional history table in your database and used as learning for subsequent campaigns and models.

Whatever your path, remember to set your goals first and then look for the tools to do the job. One of your first goals should be to know your customer -- regardless of whether they are or aren't who you thought they were. You must embrace them. Listening and paying attention to your customers is critical to any business relationship. Use these tools to listen to your customers and then reach out to them.

Thomas J. Siragusa is senior vice president of Internet Applications for internetQueryObject Corporation (www.queryobject.com).