Judging the Strength of Generalizations
Introduction
Welcome back to Choosing Samples You Can Trust! This is lesson six of six, the final lesson of the course. Congratulations on making it here. Over the past five lessons, we have built a complete set of tools: identifying representative samples, spotting convenience and voluntary-response bias, catching coverage gaps in sampling frames, defining true random selection, and understanding why randomness reduces bias.
Now it is time to put all of those tools to work at once. In this lesson, we will learn how to look at a generalization someone makes about a population and decide whether that generalization is actually justified by the sample behind it. This is the skill that ties the entire course together.
It Is Not About the Claim, It Is About the Method
Before we dig in, let's set up an important mindset shift. When we hear a statement like "Most residents support the new library," our natural instinct is to ask, Does that sound right? But as we have learned throughout this course, gut feeling is not a reliable judge. A claim can sound perfectly reasonable and still rest on a deeply flawed sample.
The real question to ask is: How was the sample chosen? A generalization is only as trustworthy as the sampling method behind it. This single habit, looking past the claim to examine the method, is what separates careful reasoning from guesswork.
What "Justified" Means Here
Let's be precise about the word justified. In this course, a generalization from a sample to a population is justified when two conditions are met:
- The sample was selected using a method that gives members of the population a fair or known chance of being included (i.e., random selection).
- There is no obvious source of systematic bias that would consistently push the results in one direction.
When both conditions hold, we have good reason to believe the sample reflects the population, and the generalization deserves our confidence. When either condition fails, the generalization is not justified, regardless of how large the sample is or how believable the claim sounds.
Notice that "justified" does not mean "guaranteed to be correct." As we saw in the previous lesson, even random samples carry some random variation. A justified generalization simply means the method supports extending the finding to the population. An unjustified one means the method does not.


