Skip to content

Predictive Analytics

    Predictive analytics is a form of advanced analytics that uses both new and historical data to forecast activity, behavior and trends. It involves applying statistical analysis techniques, analytical queries and automated machine learning algorithms to data sets to create predictive models that place a numerical value or score on the likelihood of a particular event happening.

    Predictive analytics software applications use variables that can be measured and analyzed to predict the likely behavior of individuals, machinery or other entities. Predictive analytics can be used for a variety of use cases. For example, an insurance company is likely to take into account potential driving safety variables, such as age, gender, location, type of vehicle and driving record, when pricing and issuing auto insurance policies. 

    Multiple variables are combined into a predictive model capable of assessing future probabilities with an acceptable level of reliability. The software relies heavily on advanced algorithms and methodologies, such as logistic regression models, time series analysis and decision trees. 

    Predictive analytics has grown alongside the emergence of big data systems. As enterprises have amassed larger and broader pools of data in Hadoop clusters and other big data platforms, they have created increased data mining opportunities to gain predictive insights. Heightened development and commercialization of machine learning tools by IT vendors have also helped expand predictive analytics capabilities.

    Marketing, financial services and insurance companies have been notable adopters of predictive analytics, as have large search engine and online services providers. Predictive analytics is also commonly used in industries such as healthcare, retail and manufacturing. 

    Business applications for predictive analytics include targeting online advertisements, analyzing customer behavior to determine buying patterns, flagging potentially fraudulent financial transactions, identifying patients at risk of developing particular medical conditions and detecting impending parts failures in industrial equipment before they occur.

    The predictive analytics process and techniques

    Predictive analytics requires a high level of expertise with statistical methods and the ability to build predictive data models. As a result, it’s typically in the domain of data scientists, statisticians and other skilled data analysts. They’re supported by data engineers, who help to gather relevant data and prepare it for analysis, and by software developers and business analysts, who help with data visualization, dashboards and reports.

    Data scientists use predictive models to look for correlations between different data elements in website clickstream data, patient health records and other types of data sets. Once the data collection has occurred, a statistical model is formulated, trained and modified as needed to produce accurate results. The model is then run against the selected data to generate predictions. Full data sets are analyzed in some applications, but in others, analytics teams use data sampling to streamline the process. The data modeling is validated or revised as additional information becomes available.

    The predictive analytics process begins by understanding the business and preparing the data. A statistical model is then created, evaluated and deployed to handle the data and derive predictions.

    The predictive analytics process isn’t always linear, and correlations often present themselves where data scientists aren’t looking. For that reason, some enterprises are filling data scientist positions by hiring people who have academic backgrounds in physics and other hard science disciplines. In keeping with the scientific method, these workers are comfortable going where the data leads them. Even if companies follow the more conventional path of hiring data scientists trained in math, statistics and computer science, having an open mind about data exploration is a key attribute for effective predictive analytics.

    Once predictive modeling produces actionable results, the analytics team can share them with business executives, usually with the aid of dashboards and reports that present the information and highlight future business opportunities based on the findings. Functional models can also be built into operational applications and data products to provide real-time analytics capabilities, such as a recommendation engine on an online retail website that points customers to particular products based on their browsing activity and purchase choices.

    Beyond data modeling, other techniques used by data scientists and experts engaging in predictive analytics may include:

    • text analytics software to mine text-based content, such as Microsoft Word documents, email and social media posts;
    • classification models that organize data into preset categories to make it easier to find and retrieve; and
    • deep neural networking, which can emulate human learning and automate predictive analytics.

    Applications of predictive analytics

    Online marketing is one area in which predictive analytics has had a significant business impact. Retailers, marketing services providers and other organizations use predictive analytics tools to identify trends in the browsing history of a website visitor to personalize advertisements. Retailers also use customer analytics to drive more informed decisions about what types of products the retailer should stock.

    Predictive maintenance is also emerging as a valuable application for manufacturers looking to monitor a piece of equipment for signs that it may be about to break down. As the internet of things (IoT) develops, manufacturers are attaching sensors to machinery on the factory floor and to mechatronic products, such as automobiles. Data from the sensors is used to forecast when maintenance and repair work should be done in order to prevent problems.

    IoT also enables similar predictive analytics uses for monitoring oil and gas pipelines, drilling rigs, windmill farms and various other industrial IoT installations. Localized weather forecasts for farmers based partly on data collected from sensor-equipped weather data stations installed in farm fields is another IoT-driven predictive modeling application.

    Analytics tools

    A wide range of tools is used in predictive modeling and analytics. IBM, Microsoft, SAS Institute and many other software vendors offer predictive analytics tools and related technologies supporting machine learning and deep learning applications.

    In addition, open source software plays a big role in the predictive analytics market. The open source R analytics language is commonly used in predictive analytics applications, as are the Python and Scala programming languages. Several open source predictive analytics and machine learning platforms are also available, including a library of algorithms built into the Spark processing engine.

    Analytics teams can use the base open source editions of R and other analytics languages or pay for the commercial versions offered by vendors such as Microsoft. The commercial tools can be expensive, but they come with technical support from the vendor, while users of pure open source releases must troubleshoot on their own or seek help through open source community support sites.

    Predictive Analytics Primer

    No one has the ability to capture and analyze data from the future. However, there is a way to predict the future using data from the past. It’s called predictive analytics, and organizations do it every day.

    Has your company, for example, developed a customer lifetime value (CLTV) measure? That’s using predictive analytics to determine how much a customer will buy from the company over time. Do you have a “next best offer” or product recommendation capability? That’s an analytical prediction of the product or service that your customer is most likely to buy next. Have you made a forecast of next quarter’s sales? Used digital marketing models to determine what ad to place on what publisher’s site? All of these are forms of predictive analytics.

    Predictive analytics are gaining in popularity, but what do you—a manager, not an analyst—really need to know in order to interpret results and make better decisions?  How do your data scientists do what they do?  By understanding a few basics, you will feel more comfortable working with and communicating with others in your organization about the results and recommendations from predictive analytics.  The quantitative analysis isn’t magic—but it is normally done with a lot of past data, a little statistical wizardry, and some important assumptions. Let’s talk about each of these.

    The Data:  Lack of good data is the most common barrier to organizations seeking to employ predictive analytics. To make predictions about what customers will buy in the future, for example, you need to have good data on who they are buying (which may require a loyalty program, or at least a lot of analysis of their credit cards), what they have bought in the past, the attributes of those products (attribute-based predictions are often more accurate than the “people who buy this also buy this” type of model), and perhaps some demographic attributes of the customer (age, gender, residential location, socioeconomic status, etc.). If you have multiple channels or customer touchpoints, you need to make sure that they capture data on customer purchases in the same way your previous channels did.

    All in all, it’s a fairly tough job to create a single customer data warehouse with unique customer IDs on everyone, and all past purchases customers have made through all channels. If you’ve already done that, you’ve got an incredible asset for predictive customer analytics.

    The Statistics:  Regression analysis in its various forms is the primary tool that organizations use for predictive analytics. It works like this in general: An analyst hypothesizes that a set of independent variables (say, gender, income, visits to a website) are statistically correlated with the purchase of a product for a sample of customers. The analyst performs a regression analysis to see just how correlated each variable is; this usually requires some iteration to find the right combination of variables and the best model. Let’s say that the analyst succeeds and finds that each variable in the model is important in explaining the product purchase, and together the variables explain a lot of variation in the product’s sales. Using that regression equation, the analyst can then use the regression coefficients the degree to which each variable affects the purchase behavior to create a score predicting the likelihood of the purchase.

    You have created a predictive model for other customers who weren’t in the sample. All you have to do is compute their score, and offer the product to them if their score exceeds a certain level. It’s quite likely that the high scoring customers will want to buy the product assuming the analyst did the statistical work well and that the data were of good quality.

    The Assumptions:  That brings us to the other key factor in any predictive model—the assumptions that underlie it. Every model has them, and it’s important to know what they are and monitor whether they are still true. The big assumption in predictive analytics is that the future will continue to be like the past. As Charles Duhigg describes in his book The Power of Habit, people establish strong patterns of behavior that they usually keep up over time. Sometimes, however, they change those behaviors, and the models that were used to predict them may no longer be valid.

    What makes assumptions invalid? The most common reason is time. If your model was created several years ago, it may no longer accurately predict current behavior. The greater the elapsed time, the more likely customer behavior has changed. Some Netflix predictive models, for example, that were created on early Internet users had to be retired because later Internet users were substantially different. The pioneers were more technically-focused and relatively young; later users were essentially everyone.

    Another reason a predictive model’s assumptions may no longer be valid is if the analyst didn’t include a key variable in the model, and that variable has changed substantially over time. The great—and scary—example here is the financial crisis of 2008-9, caused largely by invalid models predicting how likely mortgage customers were to repay their loans. The models didn’t include the possibility that housing prices might stop rising, and even that they might fall. When they did start falling, it turned out that the models became poor predictors of mortgage repayment. In essence, the fact that housing prices would always rise was a hidden assumption in the models.

    Since faulty or obsolete assumptions can clearly bring down whole banks and even (nearly!) whole economies, it’s pretty important that they be carefully examined. Managers should always ask analysts what the key assumptions are, and what would have to happen for them to no longer be valid. And both managers and analysts should continually monitor the world to see if key factors involved in assumptions might have changed over time.

    With these fundamentals in mind, here are a few good questions to ask your analysts:

    • Can you tell me something about the source of data you used in your analysis?
    • Are you sure the sample data are representative of the population?
    • Are there any outliers in your data distribution? How did they affect the results?
    • What assumptions are behind your analysis?
    • Are there any conditions that would make your assumptions invalid?

    Even with those cautions, it’s still pretty amazing that we can use analytics to predict the future. All we have to do is gather the right data, do the right type of statistical model, and be careful of our assumptions. Analytical predictions may be harder to generate than those by the late-night television soothsayer Carnac the Magnificent, but they are usually considerably more accurate.

    Big data analytics projects raise stakes for predictive models

    One of the keys to success in big data analytics projects is building strong ties between data analysts and business units. But there are also technical and skills issues that can boost or waylay efforts to create effective analytical models for running predictive analytics and data mining applications against sets of big data.

    A fundamental question is how much data to incorporate into predictive models. The last few years have seen an explosion in the availability of big data technologies, such as Hadoop and NoSQL databases, offering relatively inexpensive data storage. Companies are now collecting information from more sources and hanging on to scraps of data that in the past they would have considered superfluous. The promise of being able to analyze all that data has increased its perceived value as a corporate asset. The more data, the better seemingly.

    But analytics teams need to weigh the benefits of using the full assortment of data at their disposal. That might be necessary for some applications — for example, fraud detection, which depends on identifying outliers in a data set that point toward fraudulent activity, or uplift modeling efforts that aim to segment potential customers so marketing programs can be targeted at people who might be positively influenced by them. In other cases, predictive modeling in big data environments can be done effectively and more quickly with smaller data sets through the use of data sampling techniques.

    Tess Nesbitt, director of analytics at DataSong, a marketing analytics services company in San Francisco, said statistical theorems show that, after a certain point, feeding more data into an analytical model doesn’t provide more accurate results. She also said sampling analyzing representative portions of the available information can help speed development time on models, enabling them to be deployed more quickly.

    Predictive models benefit from surplus data

    Still, there’s an argument to be made for retaining all the data an organization can collect. DataSong helps businesses optimize their online ad campaigns by doing predictive analytics on what sites would be best to advertise on and what types of ads to run on different sites; for sales attribution purposes, it also analyzes customer clickstream data to determine which ads induce people to buy products. To fuel its analytics applications, the company ingests massive amounts of Web data into a Hadoop cluster.

    Much of that data doesn’t necessarily get fed directly into model development, but it’s available for use if needed — and even if it isn’t, Nesbitt said having all the information can be useful. For example, a large data set gives modelers a greater number of records held out of the development process to use in testing a model and tweaking it for improved accuracy. “The more data you have for testing and validating your models, it’s only a good thing,” she said.

    Data quality is another issue that needs to be taken into account in building models for big data analytics applications, said Michael Berry, analytics director at travel website operator TripAdvisor LLC’s TripAdvisor for Business division in Newton, Mass. “There’s a hope that because data is big now, you don’t have to worry about it being accurate,” Berry said during a session at the 2013 Predictive Analytics World conference in Boston. “You just press the button, and you’ll learn something. But that may not stand up to reality.”

    Staffing also gets a spot on the list of predictive modeling and big data analytics challenges. Skilled data scientists are in short supply, particularly ones with a combination of big data and predictive analytics experience. That can make it difficult to find qualified data analysts and modelers to lead big data analytics projects.

    Analytics skills shortage requires hiring flexibility

    Mark Pitts, vice president of enterprise informatics, data and analytics at Highmark Inc., said it’s uncommon for data analysts to come out of college with all the skills that the Pittsburgh-based medical insurer and healthcare services provider wants them to have. Pitts looks for people who understand the technical aspects of managing data, have quantitative analysis skills and know how to use predictive analytics software; it also helps if they understand business concepts. But the full package is hard to find. “All of those things are very rare in combination,” he said. “You need that right personality and aptitude, and we can build the rest.”

    Along those lines, a computer engineer on Pitts’ staff had a master’s degree in business administration but didn’t really know anything about statistical analysis. Highmark paid for the engineer to go back to school to get a master’s degree in statistics as well. Pitts said he identified the worker for continuing education support not only because the engineer had some of the necessary qualifications but also because he had a personality trait that Pitts is particularly interested in: curiosity.

    At DataSong, Nesbitt typically looks for someone with a Ph.D. in statistics and experience using the R programming language, which the company uses to build its predictive models with R-based software from Revolution Analytics. “To work on our team, where we’re building models all the time and we’re knee-deep in data, you have to have technical skills,” she said.

    Ultimately, though, those skills must be put to use to pull business value out of an organization’s big data vaults. “The key to remain focused on is that this isn’t really a technical problem — it’s a business problem,” said Tony Rathburn, a senior consultant and training director at The Modeling Agency, an analytics consultancy in Pittsburgh. “That’s the real issue for the analyst: setting up the problem in a way that actually provides value to a business unit. That point hasn’t changed, regardless of the amount of data.”

    Faster modeling techniques in predictive analytics pay off

    At Enova International Inc., a Chicago-based online financial services firm, the company has been investing heavily in Ph.D. level data scientists. But that approach to building an analytics team raises a question: How you do you adapt academic predictive modeling techniques to business processes?

    Joe DeCosmo, Enova’s chief analytics officer, said a member of his team recently told him that when the analyst first started working at the company, he had to get over his academic instincts to detail every theory-based aspect of the predictive models he builds in order to focus more on the business impact the models can have.

    “They have to realize they don’t have to build the perfect model,” DeCosmo said. “It’s about building something that’s better than what we’re doing currently.”

    This issue is heating up as more businesses look for workers with data science skills. Often, the people who have the skills organizations need, which include statistical analysis, machine learning, and R and Python programming, come from academic backgrounds. But businesses don’t have the kind of time that Ph.D. programs give students to build analytical models. In the real world, models need to be built and deployed quickly to help drive timely business strategies and decisions.

    Focus on perfection doesn’t pay

    About 20% of the people on the analytics team at Enova have doctorates. DeCosmo said most of the analysts come around to a more business-focused way of doing things once they see how the end-product of their work can improve a specific business process. For example, Enova recently applied predictive modeling techniques to identify suitable recipients for a direct mail marketing campaign, to better target the mailing. That helped improved response rates by about 25%, according to DeCosmo. The model may not have been perfect, he added, but the kind of rapid improvement it led to helps data scientists understand and appreciate the value of their work.

    “At our scale, if we can get a model into production that’s 10% better, that adds material impact to our business,” DeCosmo said.

    There’s always a tradeoff between time and predictive power when developing analytical models. Spending more time on development to make a model better could allow a data scientist to discover new correlations that boost the strength of its predictions. But DeCosmo said he sees more business value in speedy development.

    “We’re very focused on driving down the time [it takes to develop models],” he said. “There’s no such thing as a perfect model, so don’t waste your time trying to build one. We’d rather get that model out into production.”

    Simplicity drives predictive modeling speed

    For Tom Sturgeon, director of business analytics at Schneider Electric’s U.S. operations in Andover, Mass., the top priority is empowering business analysts to do some straightforward reporting themselves and free up his team of data scientists to focus on more strategic analysis work.

    Schneider Electric is an energy management company that sells products and services aimed at making energy distribution and usage by corporate clients more efficient. In the past, for every new report or analysis a business unit wanted, Sturgeon and his team would have to pull data out of a complex architecture of ERP, CRM and business intelligence systems, all of which were themselves pulling data from back-end data stores. Sturgeon described these systems as middlemen because they hold a lot of useful data, but on their own don’t make data easily accessible. His team had to manually pull data out, an action which itself has less value than the actual analysis.

    But since 2013, they’ve been using a “data blending” tool from Alteryx Inc. to bring all the data into an analytics sandbox that business analysts can access with Tableau’s data discovery and visualization software. Sturgeon said that allows the business analysts to skip the “middleman” reporting systems and build their own reports, while his team does deeper analyses.

    “We take the data and bring it together,” he said. “Then we say, ‘Here’s the sandbox, here are some tools, what questions do you want to ask?'”

    Even when doing more data science work, though, the focus is on simplicity. The analytics team is still working to develop its predictive capabilities, so for now it’s starting small. For example, it recently looked to see if there was a correlation between macroeconomic data published by the Federal Reserve and Schneider Electric’s sales. The goal was to improve sales forecasting and set more reasonable goals for the company’s salespeople. The analysts could have brought in additional economic data from outside sources to try to strengthen the correlation, but they instead prioritized a basic approach.

    “We aren’t looking to build the best predictive model,” Sturgeon said. “We’re starting simple and trying to gain traction.”

    Predictive modeling isn’t BI as usual

    In looking to unleash effective and speedy predictive modeling techniques in an organization, bringing a standard business intelligence mindset to the process won’t cut it, said Mike Lampa, managing partner at consultancy Archipelago Information Strategies.

    Speaking at the TDWI Executive Summit in Las Vegas, Lampa said workers involved in predictive analytics projects need to have much more freedom than traditional BI teams, which typically spend a lot of time initially gathering project requirements. That would be a waste of time in a predictive project, he added. Meaningful correlations are often found in unexpected data sets and may lead to recommendations that business managers weren’t necessarily looking for.

    Setting project requirements at the outset could slow down the analytics process and limit the insights that get generated, Lampa cautioned, adding that data scientists have to be able to go where the data takes them. “You can’t create effective models when you’re always tied down to predetermined specifications,” he said.

    Business focus is key when applying predictive analytics models

    At the oil and gas drilling company Halliburton, traditional BI is still important, but there is a growing emphasis on predictive analytics models. One company official said this trend is going to be the key to differentiating the Houston-based firm from its competitors and making it more successful.

    “You can do as much business intelligence as you want but it’s not going to help you win against your competitors in the long run,” said Satyam Priyadarshy, chief data scientist at Halliburton, in a presentation at the Predictive Analytics World conference in Boston. He added that predictive modeling is going to be a “game changer.”

    But simply doing predictive analytics modeling isn’t enough. For Priyadarshy and other conference presenters, predictive initiatives are only successful when they are business-oriented and narrowly tailored to address specific problems.

    Predictive modeling is a stat-heavy, technically intensive exercise. But when implementing a predictive modeling program within a company, it’s important to not get bogged down in these areas in order to push projects to deliver true business value.

    For Priyadarshy, this approach means breaking down some of the data silos that inevitably spring up. During the process of exploring and drilling a new gas or oil well, tremendous volumes of data are generated. But they come from several different departments. For example, data from seismic surveys of sites have traditionally not been shared with the drilling operations teams, Priyadarshy said. But there’s an obvious need for the crews manning the drills to know what kind of material they’re likely to hit at certain depths.

    Priyadarshy said he and his team are working on a homegrown data platform that would make this data more accessible. The platform is a combination of Hadoop, SQL, and in-memory database tools. It also includes a data virtualization tool that allows different teams to access data wherever it is stored. Doing so allows drilling teams to build predictive analytics models based on data coming off of drilling sensors and from seismic surveys. These models allow the drilling teams to predict in real time how fast they should run the drill bit and how much pressure to apply.

    Having such knowledge separates predictive modeling from traditional BI, Priyadarshy said. Rather than producing a static BI report that retrospectively explains certain events during the drilling process, the predictive models allow teams to make adjustments in real time and address specific problems.

    “With predictive models, you want to build actionable things rather than just dashboards,” Priyadarshy said.

    Keep predictive modeling projects business-focused  

    Predictive modeling is most effective when it’s used to tackle known business problems, rather than looking to predict correlations that don’t necessarily have specific business value. 

    “You want to be clear about what types of problems you’re trying to solve,” said Alfred Essa, vice president of analytics at McGraw-Hill Education in Columbus, Ohio, during a presentation at Predictive Analytics World. “This helps you ask deeper questions.”

    McGraw-Hill works with clients — primarily local school districts and colleges — to look at their data to predict student performance. McGraw-Hill and the schools have been able to reliably predict how students are likely to perform in classes, including which students could fail or drop out, Essa said. But simply giving this information to schools isn’t necessarily helpful. He talks to clients to make sure they have a plan for how they intend to use the information. Just telling students they’re likely to fail and they need to work harder might actually backfire, causing them to give up. Schools need to develop curriculums to help failing students before they do anything with the predictions, he said.

    For Essa, the answer to this kind of question often comes during exploratory data analysis. This early stage of modeling typically involves just looking at the data, graphing various elements and trying to get a feel for what’s in the data. This stage can help modelers see variables that may point to trends, Essa said. In the case of predicting student failure, they may be able to see factors that lead students to fail, enabling schools to address these worries. This action goes beyond just a predictive model.

    “Before you start to do modeling, it’s really helpful to pose questions and interactively get answers back,” Essa said.

    Simplify outputs of predictive analytics models

    There’s always statistics underpinning any predictive model, which are useful to the modelers. But for the lines of business that interact with the results of predictive models, these stats are nothing but distraction.

    Instead, predictions need to be clear and concise, said Patrick Surry, chief data scientist at airfare prediction mobile app Hopper, based in Cambridge, Mass. He talked about how one of Hopper’s competitors gives customers purchasing recommendations as a confidence interval. The problem is that few people understand what the site means when it says, for example, it’s 70% confident a given price is the lowest that can be expected. Similarly, when Hopper was testing its service it used the word “forecast” to talk about changes customers should expect in prices. Surry said this just made people think Hopper was talking about the weather.

    “When you watch people try to interact with predictions, there are things you don’t even think about,” he said. “As soon as you put the word ‘confidence’ in there you’ve lost 90% of the audience.”

    Today, the Hopper app simply tells users to buy now because prices are as low as they’re likely to get or to wait because a better deal is likely to pop up. There are some complicated predictive models running behind the scenes analyzing things like historic price data, prices for given days of the week and month, destinations and past sales. But Surry said customers don’t need to know all these calculations; they just need to know if they should buy an airline ticket or wait.

    Predictive analytics tools point to better business actions

    From recommending additional purchases based on the items that customers place in online shopping carts to pinpointing hospital patients who have a greater risk of readmission, the use of predictive analytics tools and techniques is enabling organizations to tap their collections of data to predict future business outcomes — if the process is managed properly.

    Predictive analytics has become an increasingly hot topic in analytics circles as more people realize that predictive modeling of customer behavior and business scenarios is “the big way to get big value out of data,” said Mike Gualtieri, an analyst at Forrester Research Inc. As a result, predictive analytics deployments are gaining momentum, according to Gualtieri, who said that he has seen an increase in adoption levels from about 20% in 2012 to “the mid- to high-30% range” now.

    That’s still relatively low — which creates even bigger potential business benefits for organizations that have invested in predictive analytics software. If a company’s competitors aren’t doing predictive analytics, it has “a great opportunity to get ahead,” Gualtieri said.

    Predictive analytics projects can also provide those benefits across various industries, said Eric King, president and founder of The Modeling Agency LLC, an analytics consulting and training services firm based in Pittsburgh. “Everyone is overwhelmed with data and starving for information,” King noted.

    But that doesn’t mean it’s just a matter of rolling out the technology and letting analytics teams play around with data. When predictive analytics is done well, the business benefits can be substantial — but there are “some mainly strategic pitfalls” to watch out for, King said. “Many companies are doing analytics to do analytics, and they aren’t pursuing analytics that are measurable, purposeful, accountable and understandable by leadership.”

    Data scientists don’t know it all

    One common mistake is putting too much emphasis on the role of data scientists. “Businesses think the data scientists have to understand the business,” Gualtieri said. With that in mind, they end up looking for experienced data analysts who have all the required technical skills and also understand their business practices, a combination that he warned can be nearly impossible to find. “That’s why they say, ‘A data scientist is a unicorn.’ But it doesn’t have to work that way.”

    Instead, he recommended, business managers should be the ones who walk through customer experience management operations or other business processes and identify the kinds of behaviors and trends they’d like to predict, “then go to the data scientists and ask if they can predict them.”

    King agreed that organizations often give data scientists too much responsibility and leeway in analytics applications.

    “They’re really not analytics leaders in a lot of cases,” he said, adding that data scientists often aren’t very effective at interviewing people from the business side about their needs or defining analytics project plans. Echoing Gualtieri, King said a variety of other people, from the business and IT, should also play roles in predictive analytics initiatives. “When you have the right balance with your team, you’ll end up with a purposeful and thriving analytics process that will produce results.”

    Plan ahead on predictive analytics

    Companies looking to take advantage of predictive analytics tools also shouldn’t just jump into projects without a plan.

    “You can’t approach predictive analytics like you do a lot of other IT projects,” King said. It’s important, he advised, to think strategically about an implementation upfront, plotting out a formal process that starts with a comprehensive assessment of analytics needs and internal resources and skills. “That’s where we’re seeing not only a greater adoption of predictive analytics, but far greater results,” he said.

    In addition, companies need to understand the data they have at their disposal and make it easily accessible for analysis, which is “no small task,” according to Gualtieri. Without an effective data management strategy, analytics efforts can grind to a halt: “Data scientists consistently report that a large percentage of their time is spent in the data preparation stage,” he said. “If they can’t effectively get that data together or it takes too much time, opportunity is wasted.”

    Another mistake that some companies make is turning to inexperienced workers to get the job done, said Karl Rexer, president of consultancy Rexer Analytics in Winchester, Mass.

    “Predictive analytics requires knowledge of statistics, sample sizes, regression and other sorts of analytics tools and techniques that isn’t commonly found inside the current staffs that businesses have,” he said. If hiring experienced workers isn’t an option, he suggested outsourcing initial pilot programs to external experts who can help produce some early successes while also working to transfer the needed skills to existing staffers.

    Once those skills are in place and projects are under way, Rexer said a key to getting good results from predictive analytics techniques is focusing on one business initiative at a time — for example, customer retention or getting online shoppers to add more items to their carts. In some cases, companies think “they can take all the data, throw it in [predictive models] and magically insights are going to come out,” he said. “Predictive analytics can be very helpful, but it’s not magic. You need to be tightly focused.”