Data Mining applications accross the industries

I am often asked the question about what are the most common applications of analytics in a specific industry. Even though each industry has some application of analytics and data mining that are specific to them, they also have cross-industry applications that are common to many industries. Example of industry-specific analytical application is “policy-lapse prediction” in the insurance industry. Examples of cross-industry applications could be customer segmentation or customer retention, since in any industry where there are customers there is also need to segment them and retain them. Following is a mix of analytical applications and can be done in a specific industry:    

Banking (retail): Analytics can help banks understand and drive decisions related to customer profitability, as well as to enable banking institutions to segment customers according to a multitude of variables: demographics, account history, etc. – in order to create more meaningful and targeted marketing programs. Furthermore, analytics can help banks improve retention rates by determining its causes and predicting future customer attrition. In addition, banks can apply analytics to historical data to find out which customers are good candidates for cross-selling and up-selling and as a result achieve increase in revenue and wallet share. For most banks analytics are used as the most powerful weapon in the fight against fraud.

Banking (investment): In investment banking analytics can be of tremendous value in supporting cross-asset trading and various other trading strategies. Also, analytical technologies are invaluable for enterprise-wide, market and credit risk management. Other applications of an analytics are segmenting and predicting the behavior of homogeneous groups of customers, uncovering hidden correlations between different indicators, create models to price futures, options, and stocks, and optimize portfolio performance.

Insurance (short term): Analytical applications in short term insurance are in rate-making by identifying risk factors that predict profits, claims and losses as well as in identifying potentially fraudulent claims. Common applications of analytics are in segmenting and profiling customers and then doing a rate and claim analysis of a single segment for different product, as well as performing market basket analysis and sequencing that answers the question of what insurance products are purchased together or in succession. Other common applications are in reinsurance, and in estimating outstanding claims provision (severity of the claim, exposure, frequency, time before settlement, etc.), as well as in using analytics to separate claims between digital and mobile assessors.

Insurance (life):  A common application of analytics in life insurance is around policy lapse predictions, modeling brokers’ performance, reactivating of dormant customers to estimating the buying potential, and realizing the untapped potential through using analytics for more effective cross-selling. In addition analytics are commonly used to model response in direct marketing of specific insurance products.

Telco's: Analytics in telecoms are used for churn management, network fault prediction, up-selling and cross-selling, capacity planning personalized advertising and subscriber profiling.

Retail: Analytics in retail are being used for supply chain and demand planning, customer segmentation and profiling, for improving response in direct marketing, for better cross-selling and up-selling, for product management, and for better understanding which products are purchased together or in sequence.

Industrials: Analytics among Industrials are being used for warranty analysis, quality control, process optimization, waste management, supplier segmentation, product and customer profitability, causal analysis, service parts optimization, and for supply chain optimization and demand planning.

Resources: The use of analytics in exploitation of natural resources is to better understand the operational risks associated with situations like equipment failures, human error and security breaches. Analytics can also be used to analyze usage patterns, weather, econometric data, changing demographics, etc. in order to accurately and confidently predict energy purchase/supply requirements.

Oil and Gas (upstream): Analytics in Oil and Gas are used for exploration and production optimization, facility integrity and reliability (predicting shut-downs, outages and downtime in production), reservoir modeling and oil-field production forecasting, estimating the shape of an oil field, fluid flood optimization and permeability prediction. It is also used for optimization of the reliability of equipment. Other applications of analytics include managing oil field assets by identifying trends in asset performance and potential, estimate the potential for infill drilling locations, screening and prioritizing workover candidates, and discover the characteristics of high potential producing assets and identify opportunities for acquisitions.

Oil and Gas (downstream): Common analytical applications are in demand forecasting, prediction of outages (planned, unplanned), grid overloads as well as predictive asset maintenance and fault prediction. Other applications are workforce optimization and consumer analytics.

Healthcare: Analytics in healthcare are being used for medical claims analysis (segmentation of claims (normal claims, claims for case managers, claims for investigative units), outcome analysis, both clinical and financial (mortality, length of stay, etc.), for disease management, for medical errors, as well as for the patient, supplier relationship management (increased patient satisfaction levels, segment suppliers and providers of cost, efficiency and quality of service).

Goods: Analytics among goods manufacturers are being used for quality control, process optimization, waste management, for inventory optimization and demand planning.

Public: Analytics in the public sector are used for improving of improving service delivery and performance of government agencies, improving safety, minimizing of tax evasion, detecting fraud, waste and abuse, analyzing scientific and research information, managing human resources, optimizing resources, and analyzing intelligence information.

Goran Dragosavac

Applications of Analytics and Data Mining in Telecommunications

The telecommunications industry was an early adopter of data mining technology and therefore many data mining applications exist.  Telco’s generate a tremendous amount of data, such as call detail data, which describes the calls across the telecommunication networks, network data, which describes the state of the hardware and software components in the network, and customer data, which describes the telecommunication customers. Such rich data is a fertile environment for many data mining applications built with the purpose of reducing some of the most pressing business problems in telecommunications.

In general , the telecommunication industry is interested in answering some strategic questions using data mining applications such as :

-       which customer group is highly profitable, which one is not?

-       to which customers should we advertise what kind of special offers?

-       which customers are most likely to churn?

-       how do customer profiles change over time?

-       fraud detection and prediction ( for example stolen mobile phones or phone cards )

-       how does one retain customers and keep them loyal as competitors offer special offers and reduced rates?

 -      how does one predict whether customers will buy additional products and services like cellular services,

 call waiting or basic services?

-       what characteristics differentiate our products from those toour competitors?

-       when is a high-risk investment, such as new fiber optic lines ,acceptable?

-       what kind of call rates would increase profit without losing good customers?

Overview of the most common app's of data mining in telco's in more detail: 

////Sources: web, GDDM library ////

Network Fault Prediction

Network shut-downs for prolonged periods of time and more often can mean two things – loss of revenue and loss of customers. Here, predictive modeling can be used to generate alert just before shut-down so that immediate preventative actions can be taken. Model is built on historical instances of previous shut-downs and state of the network prior to shut-down. Such model is then applied in future time periods being able to recognize times before network failures and generating alerts.

Capacity Planning

Capital expenses contribute significantly to the overall cost of running a network. Operators invest in network capacity to address scalability and future growth. Since this growth can be unpredictable, operators typically over-provision their networks—leading to significant amounts of unutilized capacity that cannot be immediately monetized. Data mining and correlation techniques applied successfully on network data help the operator identify heavily utilized parts of the network at different points in time. This helps the operator to make key decisions related to adding capacity at the right location at the appropriate time. This analytics-assisted capacity planning, combined effectively with dynamic traffic routing, helps operators to optimize network resources—leading to overall cost reductions.

Subscriber Data Analysis and Profiling

Operators have access to large amounts of data about a subscriber, based on their usage of the operators’ services. Analysis of calling patterns, billing data and support requests, when combined with subscriber’s personal information such as demographics, age, gender, home address and income, forms the basis for creating a profile of the subscriber. For mobile and wireless services, current location and changes to the location provide additional context for the subscriber’s profile. The subscriber profile becomes the basis for other innovative services.

Social Network Modeling and Analysis

By leveraging calling patterns and other data points from a subscriber’s profile, operators can build a social networking model for the subscriber that identifies connections and proximities between different subscribers. The social network model deduces these proximities through data analytical techniques and is periodically validated and reinforced through automated and manual actions.

Personalized Advertising

Given the lower ARPU and competitive environment, operators are exploring alternate sources of revenue. Advertisement-based revenue is one such popular source. Randomized advertisements, being intrusive and interruptive, can adversely affect the subscriber’s satisfaction with the operator. On the other hand, personalized advertising that caters to the likes and needs of the individual can enhance loyalty. These advertisements, when combined with context-specific information such as location, can significantly improve the “hit-rate.” Further, advertisers are amenable to paying premium rates for personalized advertising to the targeted audience, resulting in increased revenues for the operator.

Up-Selling and Innovative Tariffs

The 80-20 principle holds true for most operators—wherein 80% of the revenue comes from 20% of the high net-worth subscribers. The analysis of service usage and billing can help the operator identify the top 20% of subscribers and focus their attention on improving loyalty by ensuring high subscriber satisfaction. Specifically, tariffs can be personalized to provide the best value for the subscribers’ money without reducing operators’ ARPU—a win-win situation. Further, this analysis also provides an opportunity to up-sell additional services (preferably personalized) based on subscribers’ profiles.

Churn Management

Competition among operators (especially mobile operators) lends itself to increased subscriber churn because subscribers have multiple options to select from. This is further exacerbated by mobile number portability, reducing the barrier for churn. To retain their subscriber base, it is important for operators to proactively identify subscribers who are likely to churn and incentivize them to stay. Many techniques, including social network modeling, can be used to identify the subscribers who are most likely to switch out. The churn management solution is integrated with the CRM systems to ensure that appropriate actions such as personalization of tariff, discounts etc. are offered to retain the customers.

When to consider “Solutions on Demand”?

First, what is the “Solution on Demand”?  This is when you outsource specific business application to some external subject matter experts to manage it for you.

So this is what you do when you want to reduce both risks and the costs in a same time. While this may not be always long term answer – short term tactical benefits are the main appeal of going this route, especially now at a time of slowdown across IT sectors, the times of reduced budgets, and general uncertainty – while at a same time demands of new business applications continue to grow.

Another term for “Solution On Demand” is the term “Software-as-a-service” (SaaS), and a growing  number of organizations are investing in SaaS as the most mature form of cloud deployment. This deployment model is changing the way software services are consumed by the lines of business, spurring the use of business analytics and improving competitiveness. 

IT departments will discover that cloud deployments allow for a renewed focus on core competencies, reduced staffing and zero maintenance – all that while passing all the risks to expert hands.

Send me a comment or contact me directly on goran.dragosavac@zaf.sas.com

Analytics and Data Mining: Use, Abuse and Goldman Sachs

As a practitioner I always hope that the benefits of helping my clients to improve their use of analytical technologies will be somehow passed to their customers. Most of the applications of analytics are geared in that direction.  The whole idea of analytics is to enable you to serve your customers better than your competitor, so that you get in return their purchasing loyalty, and a larger share of their wallets.

However, not all users of analytics think that way. Some are only interesting about what they can get out of it, and customers are there to be milked for all its worth.

One such example is Wall Street’s biggest darling: Goldman Sachs.

It has been known that banks like that have been using analytics and data mining heavily. And not necessarily just as a value add to the customer but primarily to support various trading strategies. Analytics are being used to the abundance of market information to figure transaction flows, trends, whether the market will turn bullish or bearish, and to figure out excess returns.  In other words they are using powerful analytical capabilities coupled with super-computing power to game the system.

While it is illegal to trade on insider knowledge about company finances, these people are trading on insider knowledge about market order flow. That’s how Goldman Sachs and the other biggest houses make so much from trading. Economists have a term for that: “rent seeking” which extracts billions from the market without putting anything back. As one blogger points out: difference between usage of analytics between Google and Goldman Sachs is that Google wants to sell you a book you may be interested in, while Goldman uses analytics to take away all the books you ever bought.

So far, this wasn’t bad business for Goldman Sachs. Their profits soared, $2.3 billion in 2008 and $13 billion in 2009. Never mind the fact that financial markets were shaken and one million people lost their homes and other million lost their jobs.

But of late Goldman Sachs has been under increasing pressure from US lawmakers.  According to US senate sub-committee, which produced 639-page report on the financial crisis and Goldman Sachs role in it - one of the recommendations is that Goldman Sachs top executives are referred to the Department of Justice for possible prosecution.

This report provides the evidence that their sales people were selling securities to clients based on very shaky and volatile bonds, that they knew they will blow-up. So, they unloaded it as fast as they could on their clients, while at a same time there were betting on these bonds to fold. Scam!

Rolling Stone contributing editor Matt Taibbi says: Only reason this is controversial and we even asking if GS’s execs should be jailed, is because this is a financial service company, and things are not as obvious if this would be some other industry. If this was a car dealership – for instance – there would be no question”.

Taibbi continues: “Imagine if you are  Ford dealership and you get a full inventory of Ford Broncos that have brake defect and you decide not only to sell them, but also to give bonuses to your sales people who sell these defective products to encourage them to sell more. And then you go out and take life insurance policies on drivers of these cars you have sold to!”

That’s exactly what has happened here.  

And while there is a lot of evidence against Goldman Sachs, proving the criminal intent of its top executives will not be easy.  Add to that some powerful connections in Washington bought through political donations to the both parties and it becomes clear why not single Wall Street executive has been convicted after the recent financial meltdown.

And as for analytics here – well, you can use a screwdriver to fix a lot of things around the house, but you can also use it to take someone’s wallet.  It is just "a thing”, and how will it be used depends of the user's intent.

Goran Dragosavac

"Buzz" Analytics

The other day, at one of our regular coffee chats with the senior partner here, he dropped the word, “Buzz Analytics”.
“Heard about it”, I said, “but how important is it?”

“It’s a very effective tool”, he replied, and went on to quote Shane Atchison of Zaaz, who says “the explosion of online dialogue isn’t going away, and you can either sit on the sidelines or try to proactively influence and manage the situation.”

“Or else”, he went on, “as Neil Mason of Clickz Network puts it “blogs and forums are sources of unstructured consumer data on brands. In some ways, it’s no different than surveying consumers for brand opinions, other than that the opinions are unadulterated. This is a valuable source of intelligence for companies concerned about their brands’ reputation.”

“Buzz Analytics tools” the senior partner added, “put together all the information in a rational manner of what’s being said about brands and their competition online. So what what we have here is a software monitoring the various feeds set up and determines the sentiment and essence of what’s being said using natural-language text-processing algorithms.

After which, the software reports what’s being said about you and your competitors.”

“Hmmm”, was all I could manage”, I sipped my coffee. I think it was more because I was at a loss of words.

Later, as I sat down to catch up on more about web analytics, it reminded a lot of market research, where data is collected, analyzed and communications developed in accordance.

Yes, buzz analytics is here to stay, dear reader, as tools get even smarter by the day, and companies start competing more fiercely for eyeballs online.

--------------------

This article has been written by Wigbert Piedade, aka Wiggy, who is a Creative Director at Pigtail Pundits. A writer and film maker, Wiggy dreams of utopia; he hopes to one day reside in a village by the sea and head to the city on weekends.

How to Wake Up Dormant Customers

Maintaining the relationship with a customer is a costly exercise, and not all the customers provide the same value to an organization. There are some customers who buy little, the value of their purchase is low and this is unlikely to change, regardless of the type of stimuli. The organization loses money by investing in this type of relationship.

Then, there are other customers with whom a stronger relationship and better customer interaction would result in more profitable relationship and higher value to the organization. These two groups of customers differ in their buying potential.

Recently, I was tasked with providing assistance to one financial company with the goal to estimate purchasing potential of about a million customers who have rarely been contacted in the past.

Due to the prohibitive cost of the acquisition programs, and the fact that it is cheaper to sell to an existing customer than to a newly acquired customer, this South African financial organization has decided to re-acquire a segment of their dormant base that was rarely contacted in the last 5 years. The intention was to use advanced analytics to separate customers whose buying potential was spent, from the customers who would still respond given appropriate marketing stimuli.

So, this was two-step process. On a first step we needed to quantify customer buying potential, through building simple “look-a-like” model, which would assign probability to purchase. So for customers with potential above a specific threshold we needed to produce cross-selling model which would give us a probability of purchase of specific product. After that was done, we were not only able to isolate customers who still had buying potential, but moreover – we were able to tell which product should be offered to which customer so that this potential is realized.

After modelling was done, we have done some pre-implementation testing which was successful, and that opened the door for full-on implementation. About 12 months later,  collected numbers have indicated that 24% off dormant segment was re-activated.

Regards to all readers,
Goran

Advanced customer segmentation

Customer/market segmentation is one of these topics that are defined in a multitude of different ways. And while all these definitions are in a way correct - they tell more about the person ‘s understanding of the subject – rather than of the subject itself!

The biggest point of contention is whether segmentation is the business application, or it is an analytical method. So, to those users who think that segmentation is just synonym for clustering technique – I would say – it is a good starting point, but there is so much more into it.

And once different facets of segmentation are understood (natural segmentation - versus business driven - versus segmenting on specific business dimensions) - then you can start appreciating all the different directions where this application can take you. Add to that substantive knowledge of market, customers and data sources together with effective data preparation and your ingredients for success are coming together.

Why customer segmentation? Well, customers with similar attributes tend to behave in similar ways, more often than not. This fact is particularly evident in customer relationship management, marketing, and risk management. People within same life-stage segment tend to buy certain-types of products, so promoting products that go with that specific group can lead to successful marketing. In credit and insurance industry, good customer segmentation can lead to minimum exposure to risk. Similarly, in catalog sales, customers can be selectively targeted to reduce marketing cost.

What is the first step in segmentation? It is to know your segmentation objective. What is the goal that you want this application to take you to? Do you want to just to see your natural, data driven-segments among your customers?. Do you want to better explain your existing business-driven segments (who are my “gold-card” customers for example, what do they buy, and where do they come from)?  Or, do you want to segment your customers based on their buying potential, value, risk, propensity to attrite, or something else?

After you know your segmentation objective it is just a matter of translating it into data driven analytical process, supported by business knowledge.  Once you have a segment your population you can now act on these segments in and measure their movement and stability.

So, is that all to it? No -this is just beginning to more advanced segmentation.

If you imagine segment in a circular shape, you can imagine that there is inner and outer layer. Inner layer you can describe as your core segment members who are typical for that segment. Then you have an outer layer which is far less stable. Any segment migration strategies are done on outer layer. And that’s where the fun begins. So, depending on the direction of the portion of the outer layer you can now do two basic things. You can try to stop movement in a specific direction or you can try to encourage movement by running specific (marketing) stimuli on this sub-segment.

So not only that you are able to group your customers, and understand what makes them similar, but you can entice sub-segment movement, or even  stop it – if that is what would go in line with your initial objectives.

So, hopefully this article gave you some more insight into the power of segmentation and what can you do with it.  But how would you do it? Well, this may be in my next article, so – I hope to see you again on my blog!