Estimating Probability from Data

Over the past three lessons, we built up a complete picture: computing relative frequency as a data-driven estimate, understanding why small samples produce unstable results, and watching how those estimates gradually settle near the true probability as more data accumulates. Now it is time to bring all of those ideas together into one clear, practical skill: estimating probability from real data and evaluating how much that estimate can be trusted.

As you may recall from Lesson 3, the relative frequency of an event tends to stabilize near its true probability as the number of trials grows — a principle known as the Law of Large Numbers. That stabilization is precisely what makes experimental estimation useful: once enough data has been collected, the relative frequency becomes a reliable stand-in for a true probability we might not be able to calculate directly.

This matters especially in real-world situations where the true probability cannot be determined by logic alone. Suppose you want to know the probability that a customer will leave a positive review, or that a particular flight will arrive on time. These events have no clean, theoretical answers — but with enough observed data, you can build a solid estimate from what actually happened. The recorded outcomes become your evidence.

This final lesson focuses on two connected questions. First, how do we produce a probability estimate from a real data set? Second, how do we decide whether we have collected enough data to trust that estimate? By the end, you will have a clear, repeatable process for answering both.

The estimation procedure uses the same relative frequency formula introduced in Lesson 1. Given a data set, we apply:

The symbol $\approx$ means "approximately equal to," which is appropriate because any estimate derived from experimental data is, by nature, an approximation.

Let's walk through a concrete example. Suppose a city meteorologist records the weather over 180 days and notes that rain fell on 54 of those days. We want to estimate the probability that a randomly selected day in this city is a rainy day. Applying the formula:

We can also express this estimate as 30%. Based on the available data, there is approximately a 30% chance of rain on any given day. Notice that we are not claiming this is the exact true probability — we are offering a data-supported approximation.

Producing a probability estimate always follows the same five steps:

1. Identify the event of interest (for example, "the day is rainy").
2. Count the occurrences of that event in your data.
3. Count the total number of trials (observations or records).
4. Divide occurrences by total trials.
5. Interpret the result as your estimated probability.

A probability estimate is only as good as the data behind it. As we saw in Lesson 2, small samples can produce estimates that shift dramatically with just one or two additional observations. Before reporting an estimate, it is worth asking: do we have enough trials for this result to be reasonably stable?

There is no single universal cutoff, so it is better to think of sample size as a continuum rather than a set of sharp categories. A rough guide is:

These are only rough guidelines, not hard rules: a sample of 99 is not fundamentally different from a sample of 100. The underlying principle is the one from Lesson 3: as the number of trials $n$ grows, the maximum possible shift caused by any single new observation, $\frac{1}{n+1}$, shrinks. A larger sample means the estimate has had more chances to settle near the true probability, and each additional result carries less power to disturb it.

Note: This explanation offers a simplified approach suitable for everyday estimates. Determining the exact sample size needed is actually quite complex. In scientific practice, this is handled through mathematical methods involving confidence intervals, margin of error calculations, and other statistical techniques that are beyond the scope of this course. For everyday purposes, the general guidelines we've outlined here will serve you well.

In real-world situations, you may come across two different probability estimates for the same event, each based on data sets that vary greatly in size. Generally, the estimate from the larger sample tends to be more reliable.

That said, sample size alone isn't everything — the way a sample is constructed matters just as much.

For example, imagine you want to estimate how many people in a city prefer public transport over driving. Sample A surveys 10,000 people, but all of them were interviewed at a subway station during rush hour. Sample B surveys only 500 people, but they were randomly selected from across the entire city. Despite being far smaller, Sample B is likely to produce a more accurate estimate, because Sample A is heavily skewed toward people who already use public transport. A large but biased sample can be more misleading than a small but representative one.

However, if two samples have a similar structure and are equally representative, you should trust the bigger one. Consider this scenario: a software company tests a new feature with two groups of randomly selected users. Group A has 15 testers; 9 of them found the feature useful, giving an estimate of $\frac{9}{15} = 0.60$. Group B has 400 testers; 212 found the feature useful, giving an estimate of $\frac{212}{400} = 0.53$. Both suggest that a majority of users find the feature useful, but which estimate is more trustworthy?

The answer is 0.53 from Group B. With only 15 testers, the Group A estimate is highly sensitive to individual responses: if just 2 of those 9 users had felt differently, the estimate would drop to $\frac{7}{15} \approx 0.47$ — a notable shift from a single change of opinion. The Group B estimate, by contrast, has stabilized over 400 trials and would require many simultaneous changes to move significantly. When presenting probability estimates, it is also good practice to always report the sample size alongside the estimate, since a number without that context tells only part of the story.

In this final lesson of Probability from Experiments, we brought together the skills from all three previous lessons into a complete estimation workflow. We applied the relative frequency formula to a real data set, evaluated whether a sample is large enough to support a trustworthy estimate, and compared two estimates to identify the one backed by stronger evidence.

The practice section ahead will take you through realistic data sets — from customer feedback to travel records — where you will compute estimates, judge their reliability, and express your reasoning in your own words. That hands-on variety is exactly what transforms a concept from something understood into something truly mastered.