🛠 The Product Person #47 — 5 Mistakes To Avoid When Measuring Product Data
hi, Anthony here! In this newsletter, I curate insights on how to build great products. You’ll improve your product skills with every issue.
Let’s get to it.
(Btw, my PM Discord group has 50+ members now. Come join!)
5 Mistakes To Avoid When Measuring Product Data
(Insight from Pendo’s ProductCraft Blog)
When measuring data for your product, you’ll probably make these 5 mistakes:
📈 Not measuring real engagement
🙉 Ignoring qualitative feedback
🗣 Relying too much on your loudest customers
📊 Focusing too much on downstream metrics
🤔 Measuring too many things
Let me explain a bit further.
📈 Not measuring real engagement
Why you might do this:
There are tons of easy (read: lazy) metrics to measure (total users, raw page views, downloads).
Why you shouldn’t do this:
Just because it’s easy to track doesn’t mean it’s actionable or tells you whether users are getting value from your product.
What you should do instead:
Look less at “total metrics” and more at “frequency metrics”. (For example: how often do they use the product after signing up? How often do they use features X, Y, and Z?)
🙉 Ignoring qualitative feedback
Why you might do this:
Staring at dashboards of product usage data is a lot easier and cheaper than talking to customers.
Why you shouldn’t do this:
That data often lacks important context — it lacks the “why” of certain behavior you’ll see on graphs and charts.
What you should do instead:
Talk to users!
🗣 Relying too much on your loudest customers
Why you might do this:
It’s tempting to always reach out to them since they’ll happily provide feedback.
Why you shouldn’t do this:
They often don’t represent the majority of users — For ex: they might be so vocal because they’re using your product for unintended edge cases (so they’ll need more support).
What you should do instead:
Actively reach out to your less vocal users to get holistic data so you can make more informed decisions.
📊 Focusing too much on downstream metrics
Why you’ll do this:
Downstream metrics, like revenue, are easy to track.
Why you shouldn’t do this:
When a customer churns, just seeing “-$X” isn’t an actionable metric for understanding why they left or how to fix the problem.
What you should do instead:
Along with revenue, track more upstream metrics around user engagement within shorter time periods — For ex: Login frequency. If you see that a user logs in less and less over a 30 day period, you can reach out early to see why.
🤔 Measuring too many things
Why you’ll do this:
With so much potential data to track, why not track them all?
Why you shouldn’t do this:
“Analysis paralysis” kicks in.
There’s no point in collecting data for data’s sake.
What you should do instead:
Collect the most essential metrics first then collect more as you realize which metrics provide actionable insight and which don’t.
Once a metric stops being helpful — stop collecting it.
End Note
📌 Announcement: If you’re a young PM looking for community with other young PMs, join my Discord group :)
Thanks for enjoying The Product Person. I’d love it if you shared it with a friend or two. If this was shared with you, you can subscribe here.
❤️ Hit the heart button as well, it helps a lot.
Have a great day,
Anthony
(P.S. Anything I can help you with? Just reply to this email, I’ll respond)
One last thing, how did you like this post? Vote below!