Diving deep into B2B SaaS metrics with Todd Gardner, Managing Director at SaaS Advisors

Episode Description

In this episode of Diving Deep, Subscript's CEO, Sidharth, has an engaging conversation with Todd Gardner, Managing Director at SaaS Advisors.

Sidharth and Todd go deep into B2B SaaS metrics as they discuss:

The pros and cons of seat-based vs usage-based pricing
Which companies should consider usage-based pricing (and which absolutely should NOT)
Why you need to look at retention and CAC by product line
The top mistake for measuring churn (and how to avoid it)
And more!

Show Notes

Follow Sidharth: https://www.linkedin.com/in/sidharthkakkar/

Follow Todd: https://www.linkedin.com/in/toddgardner1/

Follow Subscript: https://www.linkedin.com/company/subscript/

About Diving Deep with Subscript

Diving Deep with Subscript is a video series where we dive deep and explore SaaS metrics with leading investors, CEOs, and finance leaders.

Watch the entire series of Diving Deep with Subscript

Get caught up on the entire series right here: https://www.subscript.com/diving-deep

Episode Transcript

Sidharth Kakkar
So the first question which won't surprise you at all, that I wanted to ask you is if you could pick just like, one main metric that you obsessed over when it comes to SaaS companies. Like when you see someone's pitch deck and like the page you flipped to see what is the thing you're looking for to understand the business?

Todd Gardner
Yeah. So my background was venture and then lending. And when you think about lending, you really care about the stability of the revenue stream. So retention is always where we focus. Even 15 years ago, we were sort of trying to figure out the best way to do retention so that dollar retention tells you both the stability of the revenue stream, but also what we think of as sort of the capacity to grow. So obviously, when that's over 100%, it shows you that, hey, this business is really set up to grow nicely because all new bookings are additive. And if you struggle with retention, it's a less stable business and it's harder to grow. Right, because you're having to sort of refill that bucket. So, I know NRR/NDR has become super popular lately, but it's kind of been the drum we've been banging on for a long time and I've written a fair amount about this. It's a cumulative number. So over time it has this amazing impact on your valuation because it builds your ARR, which is one of the components of valuation, and it also supports higher growth, which improves your multiple. So you get this amazing compounding effect when you have high net dollar retention. So that's my one.

Sidharth Kakkar
Related question to high NDR. A lot of companies that we work with, they struggle with this idea of LTV when they have pretty high NDR, especially when like, let's say you have a customer retention that's pretty good but not amazing, like 80% or something like that, then you have NDR that's like 130%. LTV becomes a very hard thing or like a slippery concept. What do you recommend people think about when they're in that type of situation?

Todd Gardner
You're right. It's hard. I think, in my opinion, sort of lifetime value is something you look at every once in a while. It's not a metric that you track necessarily or react to monthly. You got to make sure it's sufficient to support your spend or your investment in a customer. But as it approaches 100%, it becomes a really weird number. It's like, oh, because if you have retention over 100%, your lifetime value is infinite and it just keeps growing. And that's not really helpful and not really true. It is interesting. We've looked at lots of companies where there was a pretty big gap between net dollar retention and maybe gross retention. One, I would look at it on a gross basis and just get sort of a decent baseline and then you might segment that customer base. Right. I have another company I'm working with that has hundreds of little customers that churn on a pretty regular basis and then they have enterprises that grow like 300%. If you add those together, you get just a shit show and the median is not helpful at all.

Sidharth Kakkar
That's a good point. Also in the NDR front, you wrote up a really awesome piece on Snowflake in their metrics and I would love if you could share a little bit more context.

Todd Gardner
Just to be clear, in that piece I didn't slam Snowflake at all. It's just they reported this really curious set of metrics. They reported Net Dollar Retention of like 170 ish percent, maybe 175, 172, but they forecasted growth in ARR at 65%. You're like, wait, NDR is supposed to be, hey, if I don't sell any new customers, this is not going to be my growth rate. And you're like, how can this be? And two things are going on. One is they do calculate NDR differently than anybody else on the planet. You sort through all the public disclosures and a customer has to be a customer for more than a year to get included in the beginning NDR, and then they need to be a customer another year before there's actually a calculable number. So it's looking two years in arrears and that's pretty unusual. Most companies doing cohorts static pool are looking at year interviews and there's a reason they did that, which is the second point, I think they were in good faith trying to solve for a problem, but it really creates this incredible lag in terms of your reporting. The other thing that's interesting about the business is even when they sell into the enterprise, they sell, what do they call it, sort of starter kit, proof of concept.

Todd Gardner
So even if they're selling into Proctor and Gamble, it's going to start small. And so they have this one year ramp, or more than one year ramp in all of their businesses to just get them on boarded, right? And when you're looking at NDR, you're like, oh, ramp up phase versus static phase, it might be a 300%, 500% difference. And that's what's driving their net dollar retention. And the weird thing about that is it's a function of new bookings, not a function of actual underlying gross retention and cross sell and upsell. So they've got a different business. I don't fault them for the way they're trying to calculate NDR because I think in a way they're trying to mitigate that because the customer hasn't been around for a whole year. But they do need a different metric and it surprised the street. Their stock got hammered. Hey, once a company reaches X, it's a sustained, implemented customer and then we'll do a bunch of metrics on that cohort. But when they're in the wrap up phase, that's a different group. So that's my recommendation to Snowflake and they have not called and asked for it shockingly.

Sidharth Kakkar
It's interesting because I think it feels to me I don't know if you agree with the summary, but it feels to me like the thing you're pointing out is that a cohort chart is a two dimensional thing. And they're basically turning it into like, you have to turn it into a zero dimensional thing, like one number. And they're trying their best, but their cohort chart has these differing sort of rates of NDR depending on the lifecycle of the customer. And trying to summarize that in one number is just hard.

Todd Gardner
Yeah, and I think they needed to do a better job sort of signaling that, because they were just I mean, they went public at 133 times ARR. We're kind of living on that incredible NDR number. And I don't think the analysts realize a lot of that was driven by these new customers ramping up, which is very different than your installed base growing at 70% every year, which is virtually impossible.

Sidharth Kakkar
I have another sort of related question, which is, I don't know if you've ever looked at the Costco annual report they published a cohort chart in there like a full on cohort chart, and you can see by store opening date what revenues look like on average. It's pretty cool. One thing I wanted to ask you is, who else does this who else actually explain cohort behavior in a setting like that?

Todd Gardner
Yeah, I don't know. Despite what I wrote about Snowflake, like, I don't really cover the public stocks that much, although I am writing one additional piece. So I have not dug into a lot of annual reports. Yeah, companies, I see them doing it around fundraising. And the other thing I've seen, especially in the later stage fundraising, where there's more data to analyze, there's more dollars at risk, and the private equity firms are dipping down into as, you know, sort of doing Series B or whatever, a lot of them don't look at the company's metrics at all. That's interesting. Give us MRR by customer by month, and we'll do our own analysis. And so this one company I was talking about that had these various different groups, they got the cohort analysis back from the VCs, and that shouldn't happen. You should be on top of that ahead of time. And once you sort of and cohort analysis can be super like, oh my God, it's so much data. You need to be proactive to put it in the best light that you can. And we ended up doing that, but it was sort of after the fact.

Sidharth Kakkar
Other things that you feel like our common sort of mistakes companies make before they go out for a fundraise, like things that you think people should have put together but frequently don't.

Todd Gardner
Yeah, I think that's one of them. You can do, like we said, cohorts a variety of different ways by the month closed, by the kind of customer, by ACV. You can do stuff by product. I don't know that there's any one specific area, it tends to be there's CEOs who are very on top of their metrics, and then there are CEOs that I think feel like my job is a level above that or something like that, or maybe they're just not interested, or they're very much a product person or whatever. I will say, having funded 70 of these companies and talked to thousands, there is a correlation between the CEOs who know their business well and success. You do get nervous. Not that you can't be successful as a CEO without getting in the weeds, but there's a correlation for sure, and you get a little nervous when the guy says "I don't know what our MRR is." What would you guess it might be like? Come on. Or retention. Don't know the difference between gross and net, et cetera, or calculating it in a weird way. So no one specific area, but there is a delta in terms of how much CEOs engage with it.

Todd Gardner
We're doing the fundraising. So if you're not a metrics person and you're going out to do fundraising, become a metrics person. You need to know the answers to that. That will give confidence to the investors, for sure.

Sidharth Kakkar
Yeah, totally.

Todd Gardner
I'm sure you knew the metrics left, right, and upside down.

Sidharth Kakkar
I'm definitely wired that way. But there's like other parts of the CEO where I'm less good. So I think you probably noticed and then done a lot of thinking around the fact that there's so much more usage based pricing nowadays than not very long ago. I'd love for us to start with, maybe just like how do you think about or how do you define seat-based versus usage based? Like, what are the boundaries that you think about there since it's almost like a continuum.

Todd Gardner
Yeah, usage-based is what it sounds like. Right. There's some metric, maybe several metrics that get combined and then the customer's billed based on the amount they use. They could be billed in advance. They could be billed in arrears. It could be part of a larger contract. It could have tiers where as long as you're inside the tier, it's the same thing. So there's lots of different varieties. Also, most usage based pricing now, just over 50%, is now done in combination with some other form of pricing. So there might be a core level, just enterprise amount. There might be seat base pricing and then overages. So you're getting a wide variety of different approaches, which makes sense if you're seat based. What we're seeing a lot of now is companies want to do more product led growth. They want to get higher NDR. They're launching products that are usage based in conjunction with their maybe enterprise, seat-based approach. It's crazy and growing so quickly, and there's even more companies looking at it.

Sidharth Kakkar
What's your perspective on what kind of products and companies lend themselves to usage based pricing?

Todd Gardner
Yes, I wrote a pretty lengthy article on this, and it was really just looking at the public companies that are leveraging usage based pricing and grouping them. So not surprising a ton of what we call lower in the stack companies do usage-based pricing. So I wouldn't say low value added companies, but they're just lower in the stack. It's Snowflake, right? It's some utility computing. AWS isn't a SaaS company but they do tremendous job of usage based pricing and those are relatively high cost of goods sold businesses and literally that's why they adopted usage based pricing. And the guy at Snowflake said we had to because we had these direct costs on the other side. It wasn't like this brilliant strategy to drive NDR, but it's working and it is a good alignment done right? The other really interesting group is like the other end of the stack. You've got lower end of the stack and then you've got any SaaS applications that touch revenue. There's an aggregation of usage based pricing there. So it's ecommerce companies, it's payment companies and they do have some direct costs. But I think the bigger driver there is it aligns so well with value, right?

Todd Gardner
Like oh my ecommerce store sold X. I'm happy to pay a percentage of that because I won't be paying more for my store unless it's generating more. So if you're a company and you can find a metric that aligns itself to revenue, I think that's a fantastic place to try to start with some usage based pricing. So we're seeing marketing companies do that. HubSpot is a great example because HubSpot uses seat based pricing for sales force management because your value is being delivered to a person, a salesperson. On the marketing side, they don't, they base it on contacts and contacts are a proxy for well, your marketing universe is bigger so hopefully you're generating more revenue. So they use different metrics based on sort of where the value was created and there's still a lot of companies that will and should continue to do seat based pricing. Like a learning company, you're training people. The person is the person who's been a human being is benefited, it should be on the seat basis. And then there was one other group was other high cost of goods sold companies falling into this would be any companies that have communications feature Twilio or others who have direct sort of I had to send an email, even email companies, there's some direct cost there.

Todd Gardner
And then AI companies because we're using a tremendous amount of compute will have also gone to usage pricing because they have a decent cost. The other thing which isn't really a type of company but something to think about if you're serving a customer base target that isn't high growth on its own. If you pick a metric that's relatively stable inside that business, you will get no advantage from usage based pricing. So think of like appointment setting in doctor's offices. They're only going to set so many appointments for sure. So you go to usage based pricing, and then it'll just be static and be like, where's my NDR and all the other things that are supposed to happen? So you need to look for metrics that have sort of asymmetrical growth capabilities, or if you're greenfield, you can target high growth industries where at least those metrics will likely grow at the same pace the company does.

Sidharth Kakkar
When you're a SaaS business selling to non high growth businesses like doctor's offices or other similar businesses that aren't intended to be growing super fast, how do you think about NDR? Is that just not a metric you focus on?

Todd Gardner
Well, you would get there a different way. And I've got another piece coming out which is called how do Sales Led companies leverage PLG tactics. One of them is simply disaggregating your products into more easily consumable modules. So one company we're working with, they literally did analysis to find out it was Human Capital Management, and they already had a fair number of different modules, but they did analysis to figure out, okay, what's my best CAC, easiest to sell product. And they put all their energy behind that product in a very intentional land and expand sort of strategy. And then they would go back and sell three, four, or five different modules to that customer over time, but they didn't ask for a big check up front when the knowledge between the two organizations is at its lowest. Let's just get started with this. And so if you're selling into a doctor's office, in theory, you would really want to think about your product strategy and drive NDR more through cross sell and upsell than rely on usage based pricing. And you might have a product that has some usage based pricing, but not necessarily.

Sidharth Kakkar
This is maybe getting way too in the weeds. But one of the things that I feel gets like a little bit harder in this sort of cross sell, heavy cross sell businesses is how do you think about your overall sort of check? And do you allocate sort of the additional cross selling all to COGS? Do you allocate it to sales? Do you allocate it to, I don't know, something else? Do you have thoughts?

Todd Gardner
Not really. I mean, it's a company by company thing. I've definitely seen companies say, okay, our new logo CAC is this. Our expansion CAC is this. It seems a reasonable way to try to do it, but CAC is full of lots of nuances anyway. Allocations.

Sidharth Kakkar
I think we hit on this question from two different angles, but I think it's still worth sort of putting them together, which is when you have usage based pricing, how do you think about net revenue retention? We talked about Snowflake a bit, but still worth, I think, exploring that a bit deeper. Love to hear and I know you wrote a really good piece about this.

Todd Gardner
So I would just say yes. A couple of things. One is if you can do it right. If your product is set up to benefit from usage based pricing and you can pull it off, it's amazing. It's literally frictionless growth from quarter to quarter from your installed base with none of the costs associated with that cross selling efforts that we just talked about. Cross selling is still selling, and it still takes a lot of energy and effort. And usage based pricing, while there may be some people in there to encourage more usage, etc. it's still a much more efficient way to do it. I think as usage based pricing relates to net dollar retention, it's the best way to drive net dollar retention. And there's evidence that the market will give you a premium for that. But we haven't seen and my friend David Spitz, he's like, what about when usage goes down? Right? We've been in this economy that's generally been up and to the right. Like, say this for most of the companies I've talked to that are utilizing usage based pricing, they do have some things in the contract that make it look more like a ratchet as opposed to a yoyo.

Todd Gardner
Floors get established if you can, I guess, get away with it, right? It's a great way to do it. So I don't think if you saw a slowdown in the economy or usage go down for any particular reason, you would see the exact same percentage decrease in revenue from these businesses because they do have sort of some contractual.

Sidharth Kakkar
In your ideal world, what's something you wish people could go a level deeper on when it comes to metrics? Like, I don't know, maybe being able to separate them out by product line or going deeper on some portion of cohorts or anything like that. Is there a thing that you wish people could go farther?

Todd Gardner
Yes. And you just mentioned them both. Okay, so products really interesting, and not many people are doing it. And I think there's two things on product. One is if you're pursuing a product led growth strategy that has multiple products in it, you definitely need to sort of do some analysis there. And you have to have the GL set up in the right way and tagged properly. And there's some real data analysis that goes with all of this, which on the usage based pricing front is super painful, by the way. Really? Research shows the number one reason why people don't do usage based pricing is they can't find the right metrics. Once they find the right metrics, the second thing is they don't have the systems to do it. So there's some new companies out there that are helping sort of solve for that, and you might be one of them, but these are more like in the weeds data folks. But I got off track. What was the question? Yeah, so product, so you have to tag it. Right? But then, yeah, retention by product, CAC by product which is the one company leverage very useful.

Todd Gardner
Why are we spending all of our sales resources against all these products when this one is so easily and readily adopted? So let's go do that. And then ultimately I think when you get to some level of scale, you should be looking at profitability by product. And that's relatively hard to do because it's hard to disaggregate the cloud data bill or just the cloud bill. The AWS bill is like this big monolithic thing that comes in. But again, some products help you with that. But that's important, especially when you get to usage based pricing, et cetera. I wouldn't do it early on, like a smaller company. They're like, okay, our gross margin is this our product gross margin, is that right? But if you have a number of products and product profitability at scale, if you can tweak a point or two out of gross margin, that's a pretty big deal. And then customer level profitability, right? Some people are like just banging your servers all day long and you're not getting rewarded for that. And so in theory, you could be upside down on some of these customers. But again, at scale, I would think no company below 5 million should be sort of messing around with this kind of analysis because any customer is good, revenue is good, sort of you're optimizing for margin sort of later.

Todd Gardner
And then I would say cohort for sure, we talked about this, but just MRR by customer cohort as to when they were assigned is just this amazing thing. And don't even try to like my big thing lately is like don't try to pretty up the data. Evidently our brains are wired. We can assimilate lots of different data points. So just seeing it like you can see some helpful, useful trends like oh, are the newer cohorts getting better? Are they bigger? Are they maintaining their revenue longer? I think just looking at the raw data can be super helpful sometimes.

Sidharth Kakkar
I couldn't agree with all of that more. Very cool. Is there a metric or set of metrics that you feel are regularly miscalculated?

Todd Gardner
I'm so frustrated that like retention, like Snowflake just chose to do it a different way. Two things. One, and I've written a whole thing on this too, is like doing cohort retention metrics versus the formula approach which is churn over beginning MRR. If you have churn that is not associated with the customers who are in beginning MRR, you can get some absolutely wrong numbers, right? So there's some real drawbacks to using the formula approach. I also see people measuring too short a time frame and extrapolating that and just creating really noisy metrics. It's 75, it's 95 months and months a month. It really shouldn't be bouncing around that much. And so when you take short periods and you annualize them, you get to that. And then my final thing that drives me nuts is when they're annualizing numbers. If they're annualizing a quarterly number, then that's divided by four. They're annualizing a monthly number, then multiply it by twelve. That's not the way you do it. You take it to the Exponent, so you take it to the fourth or the 12th. People like, oh my God, churn times twelve will be completely out of customers in twelve months.

Todd Gardner
I'm like, well no it doesn't work that way because you had churn off the lower base. Just that tiny little nuance of when annualizing use the Exponents.

Sidharth Kakkar
My thing is when you annualize numbers but you're a seasonal business so the thing doesn't make any sense whatsoever. Some months or any month, like some months. It's like ridiculously low.

Todd Gardner
You were in the education business, right? So they're highly seasonal, those companies drove me nuts, where you get inside a little bit, but when you do rev rec in those businesses you get the perfectly smooth MRR line, which is great about SaaS, but underneath is this huge swing and bookings and billings and cash. But you look at the P and L, perfectly smooth. Seasonality can be challenging in a SaaS business. From the metrics and our cash flow perspective.

Sidharth Kakkar
You probably relate to this but venture debt is really nice for education businesses because it can help with that like super low point cash. But the business is so stable and the counterparty is very good credit risk wise. So it ends up being actually a really good sort of business.