Introduction 🎉

Welcome to Judge Claims Under Uncertainty! This lesson brings everything you learned across the previous three courses together into a single workflow. Instead of introducing a brand-new tool, you will learn how to apply a repeatable sequence of questions, and how to combine all your skills to evaluate any probability or risk claim you encounter. By the end of this lesson, you will have learned to:

  • Pinpoint the claim and identify the group by translating vague statements into specific, measurable sentences, ensuring you know exactly who and what was studied.
  • Check the base rate and framing by finding the background frequency and restating numbers in alternative formats to see if the original presentation is distorting your perception.
  • Evaluate any risk claim with a structured five-step checklist that names what is missing and draws on every skill from this path, so you never have to accept a statistic purely at face value.

Consider how airline pilots handle emergencies. They do not rely on memory or instinct alone; they follow a checklist. Not because they lack expertise, but because stress, complexity, and time pressure make it easy to overlook something important. The same logic applies when you encounter a statistic in a headline, a sales pitch, or a medical report.

Throughout this path, you have seen many ways a claim can mislead: small samples, reversed conditionals, relative risk without a baseline, survivorship bias, and more. A structured routine ensures that each of those vulnerabilities gets checked, every time, rather than only when you happen to think of it. Once you internalize the routine, evaluating a new claim becomes a matter of habit rather than inspiration.

The Five-Step Evaluation Routine 📋

The routine consists of five questions, asked in order. Each one targets a different way a claim can go wrong.

Flowchart of the five-step evaluation routine
StepQuestionWhat It Catches
1. Pinpoint the claimWhat exactly is being stated?Vague or exaggerated language that sounds precise but isn't
2. Identify the groupWho does this claim apply to?Hidden selection bias or non-comparable groups
3. Find the base rateWhat is the background frequency?Missing denominators and inflated alarm
4. Check the framingIs the number presented in a way that could distort perception?Relative vs. absolute risk confusion, percentage-point tricks
5. Name what's missingWhat information is absent?Cherry-picked time windows, omitted side effects, survivorship bias

Now let's see how to apply this five-step evaluation routine with a real example.

Evaluating a Real Headline 📰

Imagine you see this headline:

"New app reduces anxiety by 40%."

  1. Pinpoint the claim. Anxiety is reduced by 40%. But 40% of what? A score on a questionnaire? Number of anxious episodes? The claim is still fuzzy.
  2. Identify the group. Who was studied? College students during finals week? Adults with diagnosed anxiety disorders? You need to know before you can generalize.
  3. Find the base rate. What was the starting level of anxiety before the app was used? A 40% reduction sounds impressive, but it means something very different if anxiety dropped from 50 to 30 on a scale than if it dropped from 5 to 3.
  4. Check the framing. The "40%" is a relative reduction. Restating it in absolute terms might make the effect look far smaller.
  5. Name what's missing. Was there a control group? How long did the effect last? Who funded the study? Were dropouts counted?

After these five questions, you haven't rejected the claim outright, but you know exactly what information you need before trusting it. That is the goal of the routine: informed judgment, not automatic belief or automatic doubt.

You do not need to apply every step with equal depth every time you encounter a claim. Some headlines will immediately raise questions about missing context, while others hinge on a misleading comparison or an unusually small sample. The goal is not to perform a lengthy analysis every time—it is to develop the habit of pausing and asking the right questions before accepting a claim at face value.

Conclusion and Next Steps

You now have a five-step evaluation routine that ties together everything from the previous courses into one practical workflow. The steps are simple to remember: pinpoint the claim, identify the group, find the base rate, check the framing, and name what's missing. With practice, running through them becomes second nature — much like a pilot scanning instruments before takeoff.

Up next, you will put this routine to work in a set of hands-on exercises. You will match each step to the error it catches, apply the checklist to real-sounding claims, and guide a conversation toward a well-supported conclusion.

Sign up
Join the 1M+ learners on CodeSignal
Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal