About Harmful Metrics by Alexander Gornyi


The post was originally published in Russian on Startup of the Day. Alexander kindly agreed to republish what we think is of great value to our readers.

I published an important post on Facebook, but for some reason, it didn’t get into the channel. Apologies, I’m correcting the error.

Everyone knows that a business should be built on metrics because they are meaningful. Investors ask startups for metrics, startups brag about metrics at conferences, and hired products at interviews. Perfect.

The problem is that most of the generally accepted metrics have no physical meaning and don’t mean anything at all; the conversations about them are pointless. Let’s take CAC as an example. Imagine an online store. A marketer can run campaigns on Facebook with refrigerators priced at RUB 50,000 or with cutting boards for RUB 500 each. It’s clear that the CAC in the first case will be dramatically larger than in the second. It’s clear that the total cost of attracting users to the store is directly regulated by the ratio between several ads. More refrigerators – more CAC, more boards – less CAC; however, the business efficiency doesn’t change even a bit. At the same time, an investment fund analyst looks at the charts and draws profound conclusions from them.

Now, let’s look at an online school. The course costs 10 thousand, 10 thousand were invested in advertising, 10 students were attracted. CAC is 1000 rubles, 10% LTV, excellent result. But for some reason, the school administration decided to spoil it. On the last day, the manager called those who hesitated until dying seconds and made another sale with a 50% discount. Let’s add 5,000 to marketing costs, and the CAC has grown by almost 1.5 times. The business has become more efficient, everything has become better, and the metric has dramatically deteriorated.

This example can be rescued by saying that discounts are deducted from revenue and not added to marketing. Well, let’s remember this approach and apply it in a different situation. In the advertising campaign for the full price of the course, the CAC was 1000. We added a promo code for 500 rubles, and the bid on Facebook was lowered. Now, the traffic costs 800 per person. The cost of attracting “fell”, the share of marketing in the revenue too. It was 1000/10000 = 10%, now, it’s 800/9500 = 8.4%. All metrics have improved, the business has deteriorated.

I can give more examples, but these two seem enough for illustration. Numbers are crucial, but they are very often fake; their dynamics say nothing at all, except the phase of the moon. And, I guess, the most dangerous of them are those that are measured in money apiece (CAC, LTV, AOV, and others) – there might be a pod in the numerators and denominators. Have an eye on them in detail. 


The rest of the text  is in my telegram channel: https://t.me/startupoftheday/1851

Translation : Valeria Stupnikova

Notify of
Inline Feedbacks
View all comments