When it comes to personalization there is a growing trend with using machine learning to predict and optimize for the user experience at a 1:1 level.
We call the collective use of these machine learning techniques as predictive personalization campaigns.
Predictive personalization refers to a type of campaign where a machine learning model is used to predict what is the best experience to serve a visitor based on current/historical performance and the user’s individual data profile (contextual bandit). Decisions are made in real-time and at a 1:1 level and the model makes use of all the data available about that user and also takes in the context about the location, content, and other factors that go into that experience.
The predictive campaign will send the majority of traffic to the experiences that the model predicts will perform best, exploiting those insights in real-time, but continue to hold out some traffic to continue learning and exploring performance trends for the other experiences.
Just like with an A/B test, you can test predicted campaigns vs. a control to determine if a statistically significant uplift is achieved. But unlike an A/B test, the expectation with predictive campaigns is that it is “always on” and that it’s constantly adjusting traffic to the right experience at a 1:1 level.
The adoption of predictive personalization campaigns is still in its early days for our industry. For many programs, experience and maturity with these techniques are still low but there is a growing interest, especially as the solutions on the market continue to grow.
In this article we discuss best practices we have acquired from years of running predictive campaigns across a range of mid-market and enterprise clients using the FunnelEnvy platform.
What Makes For More Effective Predictive Personalization Campaigns
Below is a list of our observations after running hundreds of personalization campaigns over the years on the optimal conditions that contribute to successful predictive personalization campaigns.
- A variety of user intent. With a predictive campaign the more varied in intent and behavior that exists with users the more valuable predictive models will be in detecting and predicting better user experiences. The more varied the signal the more value predictive decisions can play a role in deciding and predicting the best outcome for a range of user intentions.
A good example of this is with predictive campaigns on the homepage, where user intent typically varies across the board.
- A variety of goals to predict for. A variety of possible user outcomes to predict for is another valuable ingredient for a successful predictive campaign. Like with user intent, we want to capitalize on the strengths of predictive models, and the more varied outcomes to predict for usually results in better business outcomes.
Good example of this is with a campaign that rotates a variety of goals or user journeys as the primary offer and call to action (CTA). Like in B2B, where you may have 4-5 different offers like free trial, request demo, request pricing, book a Drift meeting, download whitepaper, or register for a webinar. Having multiple offers to present or predict usually results in better performance of the predictive models.
- A variety of goal values to predict for. Multiple goals are a good thing, and having multiple goals with different values is even more helpful to your predictive campaigns. Once again we are seeking out scenarios where we have multiple goal options, all with a range of perceived value. This becomes important because we want to give the model more detailed feedback on what is working and what is more valuable to the business. If like in the previous example, you had 5-6 B2B goals but all of them were valued the same, (say $100 per goal completion), all goals would be perceived as equal, providing less signal back to the model. However, if your goals vary widely in range from $50 to $1000 per lead, then the predictive model has far more data and data points to work with.
An example of goal variety we typically see with SaaS clients is that goal values will fall into 2 tiers, lower value content engagement goals (content download, webinars, video views), and higher value sales intent goals (request demo, request pricing, contact sales, free trial).
- Goals are aligned to business revenue. While goals can vary widely, it is important that your goals are either revenue goals or events that correlate closely with revenue. Ultimately, predictive campaigns do best when they predict which experience is the most valuable.Therefore it’s important that you set up your campaigns to predict high value outcomes like purchases, trial to paid conversions, MQL/SQL, and closed/won deals. Where possible avoid micro-conversion and vanity metrics as your primary KPIs whenever possible. Examples of vanity metrics would be clicks, page views, video plays, etc. Whenever your KPIs are not tied to revenue/business value the harder it will be to have predictive campaigns be effective for you.
- There is some version of a revenue journey that you can track and optimize for. The more diverse and complex your buyer journey is the more you will benefit from using a predictive campaign that can take in all the data and outcomes and predict for better outcomes for your users.
If anyone has the same exact journey, there is less to predict for. However, if your a typical SaaS business and you have a variety of SMB to enterprise offerings, and you offer self service and enterprise sales scenarios, or you have a longer sales funnel that includes MQLs/SQLs/Opportunity Stages, then identifying the valuable trends across multiple goals in that revenue journey is a specific challenge that machine learning models do better at predicting for.
- Testing of high impact placements on the page. This recommendation is not unique to predictive campaigns and holds true for all types of campaigns, including A/B tests. If you are going to run a predictive campaign, you need to run it in a high impact location for it to be most effective.So examples of good placements would be primary offer/CTA on page sections above the fold. We have seen predictive campaigns run on small content strips or on content sections below the fold, and while they may contribute value, it’s just not going to generate a significant impact compared to continually optimizing for the high impact locations.
- Higher volume of traffic and conversions. Another no brainer but worth calling out. You want to run campaigns on highly trafficked pages that correlate well to conversions and revenue (think product and pricing pages vs. community or blog pages).
You should be prioritizing opportunities where you can move the needle and truly impact revenue for your site.
- More available data attributes for predictions. Predictive models need data to provide relevant signals. In many of the above recommendations I call out that the more variety of intent and goal outcomes, the more data points the model has to work with for its predictions. The objective here is to generate more contextually relevant data attributes that can feed into the model. The strategy here is to instrument more meaningful data events and attributes to feed the predictive model, often referred to as feature engineering.
Examples here would be setting up more targeted audience segments (flagging prospects and customers for example), or integrating a new data source like a firmographics API from Demandbase or Clearbit, or integrating with your CDP, CRM or Marketing automation platform, or defining content and product affinities based on user behavior and pages visited. In each of these examples we are feeding additional relevant data around user intent, behavior, or data we have about our users. All of which can help a model identify additional experience/user matches that correlate better to revenue.
- Stakeholders understand and appreciate the differences between A/B testing and predictive campaigns. This is an organizational consideration, but when you run predictive campaigns, there are certain differences that you and the larger organization needs to be aware of and comfortable with. Unlike an A/B test, your experiences may not be sticky to a user over the life of the campaign (optional). Secondly, predictive campaigns are designed to maximize revenue, you may not have a clear winner or a straightforward outcome or winner after running the campaign. In predictive campaigns there is often no one winner. You find that different experiences perform better for different user segments. Some organizations are very comfortable with this reality, others struggle and prefer the simpler situations and outcomes that come with traditional A/B tests where you have a clear learning and you can full scale a single winner for the entire population.This is where you need to educate your organization on the pros and cons of running predictive campaigns.
- Stakeholders value and are responsible for revenue. While everyone in the company will say they value revenue, not everyone is responsible for generating revenue on behalf of your business. And that is ok. But, when it comes to predictive campaigns the best outcomes are usually achieved when the primary stakeholder is aligned to revenue targets and takes the responsibility to grow those revenues as best they can with proactive initiatives like site personalization. This is where we often see the most success with our clients. Being motivated by revenue vs. just learnings, will result in a more focused approach to revenue optimization, and in many cases a predictive campaign is more effective than an A/B test in maximizing revenue for your site and user experience.
Please keep in mind that while these are 10 preferred conditions we believe improve the likelihood of your predictive campaigns being successful, this full list should not serve a prerequisite or requirements before you launch your first or next predictive campaign. Typically, if you can align on 2-3 of these conditions you are often in good shape to see success.
The goal of this list was to share with you lessons learned and what to look out for as you continue to grow and expand your personalization programs and are ideally incorporating predictive campaigns more often into your portfolio of marketing and personalization initiatives
Now that you know what to look out for when designing and prioritizing your predictive campaigns, let us know how we can help you maximize your personalization initiatives.
We’d love to hear from you! You can contact us here: https://www.funnelenvy.com/contact/
