Understanding Metrics and Measurement

Welcome to Tackling Execution and Analytics Questions, a course designed to help you succeed in product management interviews that test your ability to think critically and use data to guide product decisions.

Execution and analytics questions commonly ask you to define success metrics for a feature, investigate why a product metric is changing, or evaluate the impact of different product strategies. These questions test more than your knowledge of analytics — they reveal how you think under pressure, how well you align with business goals, and how you balance user needs with long-term outcomes.

In this lesson, you'll get a high-level overview of how to think about metrics as a product manager before diving into deeper case-based exercises in upcoming units.

Product managers must define and track the right metrics to understand how their product is performing across the user journey. A structured way to approach this is by using the AARM framework:

• Acquisition: How are users discovering and signing up for the product?
  (e.g., sign-up rate, cost per acquisition, new installs)

• Activation: Are new users reaching a point where they experience value?
  (e.g., onboarding completion rate, time to first key action)

• Retention: Are users coming back and using the product over time?
  (e.g., DAU/MAU, churn rate, weekly active users)

• Monetization: Is the product generating revenue effectively?
  (e.g., ARPU, conversion to paid, LTV)

These four categories help ensure you're measuring success at every stage of the funnel. In interview settings, referencing AARM gives you a structured foundation to think through which metrics are relevant for a given feature or product area.

Product managers don’t just track metrics — they use them to make decisions. Whether you’re evaluating a feature, diagnosing a problem, or identifying opportunities for growth, your ability to connect metrics to business goals is what sets you apart in interviews and on the job.

The key is to understand what the company or product is trying to achieve — and then select metrics that directly inform whether you're making progress toward that outcome. Without this connection, you risk focusing on numbers that look impressive but offer little insight (often called vanity metrics).

For example, a spike in app downloads might initially seem like a win. But if daily active users (DAU) and retention are falling, that growth isn’t translating into sustained value. Instead of reacting to metrics in isolation, a strong PM asks: “What are we trying to accomplish, and does this data help us make a decision?”

Here’s how this might sound in a team conversation:

• Ryan: App downloads are up 40% this month, but daily active users have dropped by 20%.
• Natalie: That’s interesting. What’s our goal right now—just acquisition, or are we prioritizing engagement and retention?
• Ryan: Retention is the main focus. We’re trying to keep new users coming back.
• Natalie: Then we should look at metrics like day-1 and day-7 retention, and maybe time to first meaningful action. That’ll tell us if the new users are getting value.
• Ryan: Good call. I’ll also segment by acquisition channel. Maybe our campaign is attracting users who bounce quickly.
• Natalie: Perfect. If the issue is user quality, that’s a marketing discussion. If it’s activation or onboarding, we’ll need to look at the product experience.

This kind of thinking shows you're not just monitoring data—you’re using it to navigate trade-offs, clarify priorities, and drive outcomes.

When making product decisions — especially those involving new features or experiments — it's easy to focus only on the primary success metric. But strong product managers also anticipate unintended consequences, and that’s where guardrail metrics come in.

Guardrail metrics help ensure that while you're optimizing one part of the product, you're not inadvertently harming another. They act as a safety check.

For example, when optimizing for revenue, you might use retention as a guardrail to ensure monetization efforts don’t drive users away. Similarly, when increasing the number of ads shown per session, session duration or user satisfaction scores can act as guardrails.

In interviews, mentioning guardrail metrics shows that you're thinking holistically. It signals that you understand product trade-offs and are proactive about maintaining user experience and long-term value.

Congratulations—you nailed the Product Sense interview and have been invited to NovaTech's Product Execution interview! With a wider variety of potential question types, you're feeling a bit anxious and decide to prep thoroughly. Before tackling specific question formats, you'll focus on the fundamentals of metrics and analytics: defining key metrics, connecting them to business goals, and exploring guardrail metrics.