The start of predictive analytics and machine learning
Predictive analytics started in the early 90s with pattern recognition algorithms—for example, finding similar objects. Over the years, things have evolved into machine learning. In the workflow of data analysis, you collect data, prepare data, and then perform the analysis. If you employ algorithms or functions to automate the data analysis, that’s machine learning.
Surprised and unsupervised learning
Both in predictive analytics and machine learning, you work with both supervised and unsupervised learning. In the case of machine learning, you add reinforcement learning, which is a form of unsupervised learning. For example, an adaptive controller adjusts parameters for an oil refinery in real time. The controller automatically adjusts the parameters based on yield, marginal costs, and other factors. This is a clear deviation from the original parameters set up by engineers. This example looks basic, but there are an active decision-making agent and environment, and the agent seeks to achieve its goal despite the uncertainty of the environment. The agent’s actions represent the future state of the environment, and the agent learns continuously and improves its accuracy over time. Predictive analytics and machine learning are interrelated in many ways, and automation is the essential bridge between these two.
Steps of building a predictive model
Before you start to build a predictive model, you need to have a problem statement defined, and the right analytical dataset prepared. After that, you’ll need to define the targets; then you start the iterative process of building a predictive model. In each iteration of the model, you adjust the variable relevant to your problem and targets until you are confident in the model’s performance.
Once the model is built, and confidence is highly based on validation and testing, your next step is to embed the model directly into the application. This allows you to extend the predictive models to many users in your organization. The embedded predictive model must incorporate two key assets—business rules and scoring equations. These business rules and scoring equations are part of the application, either in the application logic layer or application target database layer.
What is the goal of predictive analytics? How is it different from traditional data analytics?
Traditionally companies focused heavily on BI with history and answering mostly “what happened?” Various factors (such as competition, demand, volatility, and market conditions) push organizations to answer a different set of questions:
- What will happen?
- How and why did it happen?
- What is the risk if it does/doesn’t happen again?
- How do you prevent or ensure it will happen again and what is the best that could happen?
To answer these questions, you need predictive analytics.
How does predictive analytics work?
I was talking before about How can you use your data in the best way.
Before you start to build a predictive model, you need to have a problem statement defined, and the right analytical dataset prepared. After that, you’ll need to define the targets, and then you start the iterative process of building a predictive model. In each iteration of the model, you adjust the variable relevant to your problem and targets until you are confident in the model’s performance. Once the model is built, and confidence is high based on validation and testing, your next step is to embed the model directly into the application. This allows you to extend the predictive models to many users in your organization.
The embedded predictive model must incorporate two key assets—business rules and scoring equations. These business rules and scoring equations are part of the application, either in the application logic layer or application target database layer. When does predictive analytics become a limitation to your enterprise? When is it a competitive advantage? Building a data set of sufficient size and quality is the most significant driving factor that can limit predictive analytics. The second biggest limitation comes from the failure to define the problem and respective target that needs to be predicted.
The sky is the limit for applied predictive analytics opportunities in the enterprise, and some use cases have profound impacts on the business.
The maturity of an organization and how well it adopts predictive analytics drive the competitive advantage. A well-defined and articulated enterprise-level analytics strategy supports predictive initiatives where people, processes, and technology are aligned and, if used optimally, helps enterprises develop their competitive advantages. What are the data preparation challenges of predictive analytics?
Fundamental data preparation challenges are defining “what I want to predict” and “what granularity is needed to obtain data about this problem.”
Once you have right problem definition for the targets, the immediate challenges are:
- Determining the variables needed
- Choosing the proper level of granularity
- Clarifying the data quality problems that need to be addressed
- Deciding if you should analyze all the variables and all the data available
- Resolving how to handle dimensionality issues
What is exploratory predictive analytics?
In advanced analytics you let mathematical algorithms work on the data to build a predictive model.
Here is a good example of exploratory predictive analytics and how to solve it.
You then use this model to make operational decisions.
For example, build a model which answers a question such as “Is this prospect likely to buy my product?”
In the case of exploratory predictive analytics, you make use of the same algorithms of predictive analytics to obtain insights that help you answer questions such as “Why are customers buying my product?”
To summarize, it is the segment between the analytics (descriptive and diagnostic) and advanced analytics that focuses on such things as key influencers, points of interest, and segmentation, and a model based on “what-if” analysis.
Predictive analytics is a data mining solution that uses algorithms and techniques used on both structured and unstructured historical data to determine certain outcomes in order to answer business questions.
Interfaces rather than requiring the model builder to use a scripting language. Some software tools suggest models based on input data and the specification of targets of interest (such as “buy or don’t buy” or “remain a customer or drop service”). Some tools will even automate the analysis for the user once target variables are identified. With this ease-of-use movement, business analysts are rapidly becoming predictive model builders. What do organizations need to consider before jumping on the predictive analytics bandwagon?