🛠 The Product Person #47 — 5 Mistakes To Avoid When Measuring Product Data

hi, Anthony here! In this newsletter, I curate insights on how to build great products. You’ll improve your product skills with every issue.

Let’s get to it.

(Btw, my PM Discord group has 50+ members now. Come join!)


5 Mistakes To Avoid When Measuring Product Data

(Insight from Pendo’s ProductCraft Blog)

When measuring data for your product, you’ll probably make these 5 mistakes:

  • 📈 Not measuring real engagement

  • 🙉 Ignoring qualitative feedback

  • 🗣 Relying too much on your loudest customers

  • 📊 Focusing too much on downstream metrics

  • 🤔 Measuring too many things

Let me explain a bit further.

📈 Not measuring real engagement

Why you might do this:

  • There are tons of easy (read: lazy) metrics to measure (total users, raw page views, downloads).

Why you shouldn’t do this:

  • Just because it’s easy to track doesn’t mean it’s actionable or tells you whether users are getting value from your product.

What you should do instead:

  • Look less at “total metrics” and more at “frequency metrics”. (For example: how often do they use the product after signing up? How often do they use features X, Y, and Z?)

🙉 Ignoring qualitative feedback

Why you might do this:

  • Staring at dashboards of product usage data is a lot easier and cheaper than talking to customers.

Why you shouldn’t do this:

  • That data often lacks important context — it lacks the “why” of certain behavior you’ll see on graphs and charts.

What you should do instead:

  • Talk to users!

🗣 Relying too much on your loudest customers

Why you might do this:

  • It’s tempting to always reach out to them since they’ll happily provide feedback.

Why you shouldn’t do this:

  • They often don’t represent the majority of users — For ex: they might be so vocal because they’re using your product for unintended edge cases (so they’ll need more support).

What you should do instead:

  • Actively reach out to your less vocal users to get holistic data so you can make more informed decisions.

📊 Focusing too much on downstream metrics

Why you’ll do this:

  • Downstream metrics, like revenue, are easy to track.

Why you shouldn’t do this:

  • When a customer churns, just seeing “-$X” isn’t an actionable metric for understanding why they left or how to fix the problem.

What you should do instead:

  • Along with revenue, track more upstream metrics around user engagement within shorter time periods — For ex: Login frequency. If you see that a user logs in less and less over a 30 day period, you can reach out early to see why.

🤔 Measuring too many things

Why you’ll do this:

  • With so much potential data to track, why not track them all?

Why you shouldn’t do this:

  • Analysis paralysis” kicks in.

  • There’s no point in collecting data for data’s sake.

What you should do instead:

  • Collect the most essential metrics first then collect more as you realize which metrics provide actionable insight and which don’t.

  • Once a metric stops being helpful — stop collecting it.


End Note

📌 Announcement: If you’re a young PM looking for community with other young PMs, join my Discord group :)

Thanks for enjoying The Product Person. I’d love it if you shared it with a friend or two. If this was shared with you, you can subscribe here.

❤️ Hit the heart button as well, it helps a lot.

Have a great day,
Anthony

(P.S. Anything I can help you with? Just reply to this email, I’ll respond)


One last thing, how did you like this post? Vote below!

😀 🙂 😐 😞