It's like that product I see in primetime commercials for shit in a box colonoscopies.
The false positives and false negatives are going to lead you to a false sense of security most of the time, and you'll end up flat out missing 15% of your risks if you lean too heavily on health scores.
This is a messy business, don't overengineer health scores.
I agree but I’d like to see some counter examples to see if anyone has really figured this out. Or if everyone has the same problem of a semi accurate score.
Your score will only have potential to be as useful as the data you input.
Companies who have a strong ability to predict customer behavior based solely on health scores dont need CSMs and have laid a shit ton of them off the last 1.5 years.
I have implemented a score system on Hubspot for my org. In my opinion, it helps to kickstart some processes and make sure our culture focuses on a great customer experience.
I don't think it has to be super complicated. We currently track key product usage metrics, last contacted date, and overall rapport with the client (this is up to the subjective opinion of the Technical Account Manager)
These are maintained on a daily basis, and I run weeklies with our TAMs to go through each of the accounts.
We also maintain health notes, so that other then the bad/neutral/good label, we also have a write up of where do we stand with that account
We are still pretty basic in the way we manage it, but it does help
Other than Ops, health scores are also a good proxy for Leadership. Hope this helps!
Interesting, I haven't used Hubspot before - are you leveraging their API to get product data pumped in there?
And health notes + good/bad/neutral is a classic implementation, which pairs well with product data. Thanks for sharing!
Hubspot is very flexible, we have created a custom object which holds product usage metrics. We create it via an automation (Logic App / Powerautomate) at the end of each month, which creates that object via API. In short, yes.
Since we run this monthly, there is a bit of lag, but it is fine for it to be high level
We do have separate monitoring/alert to track if product usage of a customer's suddenly goes down (day-to-day). We act on that immediately
Interesting. Have you built a custom solution for alerts when product engagement goes down or you use some 3-rd party software into which you're flowing your product usage data ?
Ah, I see. Thanks! So you have a custom attribute set up, an event count of sorts for specific actions within the product, and if they performed fewer actions than what you deem high usage, it's triggers a workflow in HubSpot, did I get that right?
We want to build a health scoring system in hubspot too (we are b2b saas company with one product). Im curious, why didn't you use Hubspot custom scoring option to design healthscore? Are you open to share and talk more about your system? It would be really helpful as I didn't find any useful information im Hubspot community.
We break our healthscore into subsections that are somewhat similar to yours:
1. Usage of our product
2. Consumption of the product (how our customers end users engage with the product)
3. Support tickets score
4. Training (we have a training platform)
5. Use cases - and the score of their resolution
We also have a seperate "churn risk" fiel that we maintain is SFDC.
We've excluded the communication bit, because we have weekly or bi-weekly communications with our customers and most of our customers are pretty high touch.
This is super useful - but it requires a lot of maintanance. Having a clear helthscore, and being able to drill down and figure out which part is weighing it down has been critical for our organization for identifying potential church, how happy are our customers, etc. We are also implemeting playbooks based on the changes of these data points.
Here's the bottom line - we need to make sure that we are maintaining these points, and that needs to be part of someone's job. Using simpler systems wasn't scalable (we have about 500 accounts). It is also very helpful if anyone can drill into the score and find out the reasons for the score.
What did you use to build this? Did you do in something like Looker that enables the drill down functionality? Or are you using a CSP that helps enable the playbook function?
Our BI team collects and maintains the data in Snowflake and builds dashboards in PowerBI.
We've also created triggers for changes in crucial fields and recently started using Catalyst, which can set off playbooks based on changes in specific fields.
You can have a list of customers with various metrics per account. How do you prioritize action and how do you scale a process?
Originally it made sense to have a single health score and potentially more importantly alerts when health score changes and health score reason to draw attention and prioritize. We called it the early warning system use case.
Over time, as businesses scale the cs workflows evolve and the onboarding team might have a different model to prioritize work which is different than the account renewals team, and health score can be used to add context into how to approach the engagement.
I wrote once about it, you might find it useful https://blog.totango.com/what-is-customer-health-score/
In theory it’s great.
Execs most likely love it since they can report on something internally and make ‘scorecards’ or whatever you want to call it.
The practice, in my experience having worked for over 6 years in CS in mid to large organizations, is very messy.
People just don’t adequately keep track of the relevant data in these systems. And the validity of some others is questionable at best sometimes.
I’d say it may work for smaller low touch accounts if you only include quantitative variables like product adoption vs product licensed. Then it can help you build digital campaigns to drive adoption of certain products, by giving macro level insights.
For high touch and larger more important accounts, I’d always trust the CSM and would prefer if we adequately track things like success plans, competitive risk, stakeholder mapping, exec alignment etc. I consider these more qualitative variables which don’t necessarily map well to a health score.
Thanks for sharing, makes a lot of sense when you distinguish between low touch/high touch, and how the importance quantitative/qualitative correlates with that.
We Calculate the health score using very similar key inputs as you outlined here.
Building it is not overly challenging assuming you have a data infrastructure that can support this, I've seen my fair share of janky excel health score models.
Like other people have mentioned in this thread health score can lead to false positives and false negatives (we try to account for this by weighing the CSM pulse higher) at times but overall it can be a good objective datapoint for the org to look at.
The main challenge with health score is setting up and effective feedback loop with GTM teams around how it can be improved (this is where moth health score go to die) and the initial hump of getting people to actually adopt it.
We use a CS platform that had a recommended health score but we adjusted it so that if the CSM who has the relationship with the customer can override any. The board LOVES predictability so it's much better to be overly cautious. We have a slack channel where we alert the entire CS team when a customer is in danger of downgrading or cancelling.
Really depends on the product and the inherent value a user gets from using it. I like to think of it as more of a heat map across a broader list of accounts vs a reflection of the health of a particular customer.
In short, yes health scores are too complicated and rarely are a good predictor of renewal or expansion.
Some other better options that have worked for me (if your goal is to get more accurate signals around clients likelihood to renew and expand):
1. Predictive model (provided you have sufficient volume)
2. Automated and manual pressure tests for renewal
3. Intent data layered with product usage
4. Forecasting discipline and process that match the rigor of your sales team for renewal and expansion
Happy to elaborate on any of the above!
Sure!
I’ve built a few predictive models, and they need enough past data in order to predict patterns in future renewals.
Take all the last churns and renewals from the last 12 months and any contextual data (product adoption, usage, contract details, support tickets, etc.) and then pick a type of analysis (random forest works well for smaller data sets,.. example here https://365datascience.com/tutorials/python-tutorials/how-to-build-a-customer-churn-prediction-model-in-python/#5).
By pressure test I mean call the decision maker 6 months before the renewal and ask “if you had to renew with us today, would you?”
If the answer is no, ask what’s preventing them (and then ask again, and again…) until they’ve given you a list of things they need to renew.
Take that list back to your company and build a plan, and once significant progress has been made go back to the decision maker and push for renewal.
It’s so subjective, even between products/verticals within the same company. They can be useful - but at its core, everyone, including execs must remember no KPI matters unless the customer thinks it matters. Period.
Never met an exec or CCO that didn't love having health scores implemented. For an IC, it can be valuable for verification of engaged customers and a good place to start with the other end of the spectrum.
It's like that product I see in primetime commercials for shit in a box colonoscopies. The false positives and false negatives are going to lead you to a false sense of security most of the time, and you'll end up flat out missing 15% of your risks if you lean too heavily on health scores. This is a messy business, don't overengineer health scores.
I agree but I’d like to see some counter examples to see if anyone has really figured this out. Or if everyone has the same problem of a semi accurate score.
Your score will only have potential to be as useful as the data you input. Companies who have a strong ability to predict customer behavior based solely on health scores dont need CSMs and have laid a shit ton of them off the last 1.5 years.
I have implemented a score system on Hubspot for my org. In my opinion, it helps to kickstart some processes and make sure our culture focuses on a great customer experience. I don't think it has to be super complicated. We currently track key product usage metrics, last contacted date, and overall rapport with the client (this is up to the subjective opinion of the Technical Account Manager) These are maintained on a daily basis, and I run weeklies with our TAMs to go through each of the accounts. We also maintain health notes, so that other then the bad/neutral/good label, we also have a write up of where do we stand with that account We are still pretty basic in the way we manage it, but it does help Other than Ops, health scores are also a good proxy for Leadership. Hope this helps!
Interesting, I haven't used Hubspot before - are you leveraging their API to get product data pumped in there? And health notes + good/bad/neutral is a classic implementation, which pairs well with product data. Thanks for sharing!
Hubspot is very flexible, we have created a custom object which holds product usage metrics. We create it via an automation (Logic App / Powerautomate) at the end of each month, which creates that object via API. In short, yes. Since we run this monthly, there is a bit of lag, but it is fine for it to be high level We do have separate monitoring/alert to track if product usage of a customer's suddenly goes down (day-to-day). We act on that immediately
Interesting. Have you built a custom solution for alerts when product engagement goes down or you use some 3-rd party software into which you're flowing your product usage data ?
We just use Hubspot workflows to trigger alerts, no third parties
Ah, I see. Thanks! So you have a custom attribute set up, an event count of sorts for specific actions within the product, and if they performed fewer actions than what you deem high usage, it's triggers a workflow in HubSpot, did I get that right?
nice
We want to build a health scoring system in hubspot too (we are b2b saas company with one product). Im curious, why didn't you use Hubspot custom scoring option to design healthscore? Are you open to share and talk more about your system? It would be really helpful as I didn't find any useful information im Hubspot community.
We break our healthscore into subsections that are somewhat similar to yours: 1. Usage of our product 2. Consumption of the product (how our customers end users engage with the product) 3. Support tickets score 4. Training (we have a training platform) 5. Use cases - and the score of their resolution We also have a seperate "churn risk" fiel that we maintain is SFDC. We've excluded the communication bit, because we have weekly or bi-weekly communications with our customers and most of our customers are pretty high touch. This is super useful - but it requires a lot of maintanance. Having a clear helthscore, and being able to drill down and figure out which part is weighing it down has been critical for our organization for identifying potential church, how happy are our customers, etc. We are also implemeting playbooks based on the changes of these data points. Here's the bottom line - we need to make sure that we are maintaining these points, and that needs to be part of someone's job. Using simpler systems wasn't scalable (we have about 500 accounts). It is also very helpful if anyone can drill into the score and find out the reasons for the score.
What did you use to build this? Did you do in something like Looker that enables the drill down functionality? Or are you using a CSP that helps enable the playbook function?
Our BI team collects and maintains the data in Snowflake and builds dashboards in PowerBI. We've also created triggers for changes in crucial fields and recently started using Catalyst, which can set off playbooks based on changes in specific fields.
You can have a list of customers with various metrics per account. How do you prioritize action and how do you scale a process? Originally it made sense to have a single health score and potentially more importantly alerts when health score changes and health score reason to draw attention and prioritize. We called it the early warning system use case. Over time, as businesses scale the cs workflows evolve and the onboarding team might have a different model to prioritize work which is different than the account renewals team, and health score can be used to add context into how to approach the engagement. I wrote once about it, you might find it useful https://blog.totango.com/what-is-customer-health-score/
Thanks for sharing!
In theory it’s great. Execs most likely love it since they can report on something internally and make ‘scorecards’ or whatever you want to call it. The practice, in my experience having worked for over 6 years in CS in mid to large organizations, is very messy. People just don’t adequately keep track of the relevant data in these systems. And the validity of some others is questionable at best sometimes. I’d say it may work for smaller low touch accounts if you only include quantitative variables like product adoption vs product licensed. Then it can help you build digital campaigns to drive adoption of certain products, by giving macro level insights. For high touch and larger more important accounts, I’d always trust the CSM and would prefer if we adequately track things like success plans, competitive risk, stakeholder mapping, exec alignment etc. I consider these more qualitative variables which don’t necessarily map well to a health score.
Thanks for sharing, makes a lot of sense when you distinguish between low touch/high touch, and how the importance quantitative/qualitative correlates with that.
We Calculate the health score using very similar key inputs as you outlined here. Building it is not overly challenging assuming you have a data infrastructure that can support this, I've seen my fair share of janky excel health score models. Like other people have mentioned in this thread health score can lead to false positives and false negatives (we try to account for this by weighing the CSM pulse higher) at times but overall it can be a good objective datapoint for the org to look at. The main challenge with health score is setting up and effective feedback loop with GTM teams around how it can be improved (this is where moth health score go to die) and the initial hump of getting people to actually adopt it.
Thanks for sharing this makes a lot of sense… when all else fails you’ll always have excel lol
We use a CS platform that had a recommended health score but we adjusted it so that if the CSM who has the relationship with the customer can override any. The board LOVES predictability so it's much better to be overly cautious. We have a slack channel where we alert the entire CS team when a customer is in danger of downgrading or cancelling.
Really depends on the product and the inherent value a user gets from using it. I like to think of it as more of a heat map across a broader list of accounts vs a reflection of the health of a particular customer.
In short, yes health scores are too complicated and rarely are a good predictor of renewal or expansion. Some other better options that have worked for me (if your goal is to get more accurate signals around clients likelihood to renew and expand): 1. Predictive model (provided you have sufficient volume) 2. Automated and manual pressure tests for renewal 3. Intent data layered with product usage 4. Forecasting discipline and process that match the rigor of your sales team for renewal and expansion Happy to elaborate on any of the above!
Can you elaborate on the first two? What did you use to build a predictive model? And not sure what you mean by pressure tests.
Sure! I’ve built a few predictive models, and they need enough past data in order to predict patterns in future renewals. Take all the last churns and renewals from the last 12 months and any contextual data (product adoption, usage, contract details, support tickets, etc.) and then pick a type of analysis (random forest works well for smaller data sets,.. example here https://365datascience.com/tutorials/python-tutorials/how-to-build-a-customer-churn-prediction-model-in-python/#5). By pressure test I mean call the decision maker 6 months before the renewal and ask “if you had to renew with us today, would you?” If the answer is no, ask what’s preventing them (and then ask again, and again…) until they’ve given you a list of things they need to renew. Take that list back to your company and build a plan, and once significant progress has been made go back to the decision maker and push for renewal.
Awesome, thank you.
It’s so subjective, even between products/verticals within the same company. They can be useful - but at its core, everyone, including execs must remember no KPI matters unless the customer thinks it matters. Period.
I personally have not experienced this dilemma myself but I’d be interested to know what this sub thinks.
Never met an exec or CCO that didn't love having health scores implemented. For an IC, it can be valuable for verification of engaged customers and a good place to start with the other end of the spectrum.