How to Build a Health Score That Predicts Renewals with 95%+ Accuracy

Health models allow revenue leaders to communicate easily with other teams internally and confidently articulate their revenue forecast. If the health score doesn’t assist these leaders in accurately predicting revenue, it can cast doubt on the leader, the CS team, and the strategy in place. 

But there are two challenges teams seeking to maintain a consistent and accurate measure of their health have: The difficulty in selecting health factors for a health score combined with the temptation to incorporate constant changes."

As post-sales teams discover new information, they might also be tempted to constantly change their health scores to keep up with the evolving world around them. This constant change presents a challenge down the road when looking to attribute true impact to specific factors in their score. Moreover, the individuals that are asked to use the health score as their guide can get lost in the dust if factors within the score are constantly changing. They are being asked to learn and adopt a new approach with each change – sometimes without supporting data on why the change happened. 

Heap is an analytics platform that automatically captures customers' actions on a company’s website or app so companies can see how and why users interact with their digital products. Heap organizes its post-sales teams under The “Customer Success organization” comprising of Professional Services, Solutions Consultants, Pre-Sales Engineers, Customer Success Managers, Education, Scale, Adoption, and Support. The health score is a north star to the Customer Success organization at Heap. 

Lane Hart, the Senior Director of Customer Strategy and Operations at Heap, has been responsible for the accuracy of the health score at Heap for years and has built multiple iterations of the model. He knows the importance of having a reliable health score to inform the team, drive the value of Heap across their customer base, and influence revenue forecasting. 

“The goal with the health score isn’t to tell the CSMs how to do their jobs because that would be unfulfilling (and annoying), but it’s a good way to highlight their book of business in a way that shows them areas where they can drive the most impact.”

Previous to Lane’s ownership of the health score, anyone working accounts could update the health score by selecting either red (at risk), yellow (neutral), or green (healthy). To inform their decision, the post-sales teams usually review the usage and adoption data within their product, Heap, as it houses behavioral data. But this process was ad-hoc, inconsistent, and ultimately reliant on human capital. 

With so many available metrics, Lane and his team quickly learned that the teams weren’t all looking at the same data when rating health. In addition, an individual’s sentiment was too subjective – some were pessimists while others were overly optimistic, which influenced the health score they associated with an account.  

The results? The account health was not reliable.

Lane knew that if they wanted their health score to be reliable and impactful, they needed to build a model that surfaced the right data at the right time so the account owners could take the right action. In this Play, Lane details the steps Heap took to operationalize their predictive health score that’s powered by data.

When deciding where to start when building a predictive health score model, Lane, and his team wanted to leverage the data they had at hand and ensure their score used leading indicators of success as the foundation. Heap wanted CSMs to have timely information to mitigate risks and boost account expansion.

To build the workflow, Heap used its own platform, Salesforce, and Catalyst. Here’s the step-by-step breakdown of the Play: 

The Heap team learned a few years ago that not every customer needed the same level of service. 

Customers with larger digital footprints had a higher amount of sessions, larger teams, and more data being requested by team members. To provide an appropriate experience, the Heap team provided a strategic resource to these teams. On the other hand, customers that had smaller digital footprints had fewer sessions and smaller teams. These teams didn’t require a lot of high-touch support. 

Clear customer segmentation is an important step in building a predictive health score as what your customer needs, outcomes, and the playbooks you’ll run won’t always be a one-size-fits-all approach. 

Thanks to these segments, Heap is then able to create multiple health profiles that meet its customer's needs and allocate the team’s resources accordingly. 

Given the nature of being a product analytics company, the post-sales team at Heap is fortunate to have access to a lot of customer data! However, having access to so much data doesn’t always mean the right data is being used in a health score. 

In 2020, two quarters after implementing a new team, the Adoption team, in partnership with their Data Science team, identified 18 metrics to test their renewal data. After analyzing renewal data against these 18 metrics, the team was able to see that the likelihood of renewal was strongly correlated to the number of queries a customer was running.

They learned that the more Product Managers run queries, the more likely they were to answer their own questions, ultimately associating an increased use of Heap with delivering more value. In contrast, Product Managers that were not running queries often were likely getting those answers elsewhere and not driving much value out of Heap directly. They further dug into this metric to look at frequency and discovered that the monthly frequency of running queries resulted in the retention of users and the growth of accounts.

With data-backed metrics, the Customer Success team at Heap built a health score combining three tools, Heap, Salesforce, and Catalyst.

In this health score, the team organized their factors into two buckets: adoption and relationship. 

Though it’s incredibly tempting to make changes on the fly, Lane can’t stress enough how important it is for CS teams to allow a set amount of time to pass before making any changes to a health score. 

At Heap, the CS and Data Science teams committed to a six-month period of unaltered data collection for their health score. In this period, they may review any trends as the data accumulates, but no changes can be made to the health score. 

After a six-month period had lapsed, Lane, along with Michelle Mazzotta, the Customer Success Operations Manager at Heap, partnered with their Data Science team to analyze the findings.

The objective of their analysis was to identify additional leading indicators that can help the Customer Success team better mitigate churn and drive account expansion all at once. They also reviewed the impact of the existing indicators to ensure they were still strongly correlated to renewal predictability.

The Data Science team was brought in to help with the analysis of the data, looking for patterns and trends, and hypothesizing on new vectors that could be added to the health score. The team did some statistical testing themselves using R, the Statistical Computing Programming Language, but the Data Science team helped to validate it.

As mentioned in step 2, the first iteration of the health score included two buckets: adoption and relationship. 

The adoption bucket included the product usage data (i.e., queries) and how much of the platform the user consumed. However, after reviewing the performance of this bucket, they learned that they needed to add a new bucket called “consumption”.

Ultimately, customers who had purchased a larger plan than needed had lower-than-expected adoption scores, which skewed their results. To account for this, the consumption bucket measures how well the user is doing against what they purchased. Since Catalyst allows teams to weigh each bucket of the health score accordingly, for Heap, the consumption bucket accounts for a small percentage of the overall health score.

The relationship bucket was initially based on manual inputs from the Customer Success Manager (CSM), specifically around the last time they talked to the economic buyer or the champion of the account. through their analysis, Lane and the team found that this dependency on the individual led to outdated data and, therefore, a less reliable health score. With their next iteration, they committed to automating that information. Their team used Snowflake to aggregate data from activities in Salesforce. They would look at whether a contact was tagged as an economic buyer or champion and when the last inbound communication was received from them.

In addition to the data analysis conducted, Lane and Michelle also identified some learnings related to the health score itself, which they used to update their new version of the score.

Their learnings were:

1. Create more trust in the score by showing logical groupings and weights that align with the team’s POV (not everything is weighted equally!)
2. Preference metrics that are tighter in a timeframe to ensure it’s showing the most up-to-date data (i.e., monthly instead quarterly)
3. Each part of the health score has a specific action a CSM can take to improve it (and each action aligns with customer value so Heap doesn’t unintentionally create annoying interventions like 'touch bases')
4. Validate that the metrics actually are leading indicators of renewal and/or growth and lean on your team for feedback
5. Maintain separate metrics for "CSM Sentiment" & present them together
6. Reduce the onus on the customer-facing team, as this is the one reason data can become out of date (i.e., relying too much on manual inputs)

‍

With their findings in hand, it was finally time to put pen to paper and revise the health score.

‍

Heap runs its health score model in Catalyst, its Customer Growth Platform. Using Heap’s Salesforce Data Connector, they pushed data directly to Catalyst (without the help of engineering). 

Once the data was in Catalyst, they updated their health profile to include the new bucket they defined, “consumption,” and update the existing buckets “adoption” and “relationship” with the new data points (e.g., “Last Inbound Executive Sponsor Email”).

With the right data points added to Catalyst, updating their health score model was a quick plug-and-play in a single settings page to do the following:

• Create a new indicator bucket
• Update data points across adoption, relationships, and consumption

Once saved, their teams were able to start seeing updates to the health score within their accounts so that they could quickly begin actioning with little delay.

Heap combined its findings and learnings to develop the latest version of the health score: 

“As with anything you do in a system, it’s only as good as the people and the process around that tool. You can’t just create a health score and then expect health to improve,” says Lane.

Lane and his team have refined the operating cadences that rely on Heap’s health score several times. Their goal is to drive efficiency and action. Here are the two iterations of their operating cadence: 

Version 1: Initially, CSMs and managers reviewed accounts every week during 1:1s, using the health score. However, this was time-consuming, especially for CSMs managing 50-60 accounts.

Version 2: A cross-functional team meeting was established to review different parts of the customer journey. The audience includes heads from CS, Professional Services, Solutions Consulting, Pre-sales Engineering, Education, Scaled Adoption, and Support. The bi-weekly call was divided in four sections: 

• Part 1: Actioned on accounts that need to be resourced (these are new accounts)
• Part 2: Reviewed accounts that are being implemented and how they’re tracking, as well as surfaced any needs from the post-sales team they might have 
• Part 3: Highlighted success stories for implemented accounts on track to achieve their first win with the platform, and 
• Part 4: Solutioned how to secure renewal and expansions for mature customers as well as review accounts that have been with Heap for a while and are either on/off track to renew or expand.

The team leverages the health scores not only to help determine the scope of customers discussed during the meeting but also to advise on next steps. They use the health scores to understand why accounts are off track, and, if the account owner’s sentiment or health score is not green, they assign specific tasks to each account based on the performance against the included indicators to make progress toward improving the health.

This latest version ensures all post-sale teams are aligned on the plan of action to drive renewals and expansions, and the account owners leave with clear next steps that lead to results. When accounts are off track, they create “Get Well” tasks during that meeting for those accounts and assign actions + due dates to specific stakeholders (i.e., CSMs, Executives, Professional Services Consultants).

‍

Lane and his team incorporate the health score into their medium and long-term planning to forecast renewals. 

Every 6 months, there is a forecasting exercise done using a point-in-time look at the health score to help their finance and leadership teams negotiate targets for the team and business. Lane uses the numeric health score (a number between 1-10 to 1 decimal) in a model to predict renewal rates for the next 6 months. If the health score is anything but green with a renewal in the next 6 months, it is a red flag that requires their focus.

‍

The impact of this Play has been threefold for Heap: 

1. Accuracy: The predictive health score helped Heap predict renewals with 95%+ accuracy. This means that Heap's revenue leaders can confidently articulate their revenue forecast, which is critical for the success of the business, define future strategies and communicate the overall impact of the post-sales team. ‍
2. Time-saving: The post-sales team no longer needs to rely on heavy manual work around their health score. Updating one-off fields and performing their own data analysis was eliminated. As a result of the predictive health score, CSMs reallocated 5+ hours per week back to customer-facing work by no longer needing to analyze data manually. ‍
3. Confidence: The post-sales team now feels confident in suggestions offered by prescriptive playbooks. This is an important impact, as it means that they can be more proactive in providing guidance to customers and can save time by relying on the prescriptive playbooks.

Now it’s your turn!

Here’s how to start building a predictive health score model broken down into three sections: 

‍

1. Define the objectives of the health score model and how it will be used to guide the actions of the CS team.
2. Determine which factors are important to consider when building a health score based on your business model and what you deem valuable.
3. Decide on the data sources to be used for building the health score. This may include usage data, account owner sentiment, and other relevant data points.
4. Identify potential challenges that may arise and how they can be addressed.

1. Segment your customers to provide tailored support and playbooks for each group.
2. Use data-backed metrics to inform the health score and select the right data to use for each segment of customers.
3. Combine the data from different sources using appropriate tools and build the health score.
4. Train your customer-facing teams on the leading indicators used and how to best leverage the health score.
5. Collect data without making any changes for a set period of time to establish a baseline for the health score.

1. Analyze the data collected and determine which factors in the health score truly lead to impact.
2. Continuously evaluate and update the health score as new information becomes available.
3. Leverage the health score to deliver forecasts to the customer-facing team and revenue leaders with confidence.
4. Measure the impact of the health score on renewals, revenue forecasts, and efficiency gains for your customer-facing teams.

The health score used in this Play was powered by Catalyst. In addition to having a data-rich health score, Catalyst can also trigger actions to increase your revenue team’s proactivity, reduce churn and seize expansion opportunities at the right time. If you’re interested in learning more about how Catalyst can help you predict renewals accurately, book a demo with us!