Welcome back to Evaluating and Improving Data Quality! We are now on Lesson 2 of this course, and you are making great progress. In our first lesson, we learned to recognize six common data quality issues and practiced spotting them in sample records. Two of those issues, invalid data and inconsistent data, deserve a closer look because catching them requires a specific set of checks. That is exactly what this lesson is about.

By the end of this lesson, you will be able to determine whether values in a dataset follow expected rules, formats, and ranges, and whether data stays consistent across different systems. These skills will sharpen the way you evaluate any dataset before it is used for analysis or decision-making.

In Lesson 1 we focused on spotting problems — scanning records and naming what looked wrong. Now we need to move from noticing a problem to systematically checking for it. The difference is like the difference between glancing at a crowd and checking every person's ID at the door.

That shift in mindset is the key idea here: think like a gatekeeper. A gatekeeper does not rely on gut feeling; they compare each visitor against a clear set of rules. In the same way, checking validity and consistency means defining those rules first and then comparing every value against them. Once the rules are written down, the process becomes repeatable and much less likely to miss problems.

While validity checks whether a single value follows the rules, consistency checks whether related values agree with each other. Consistency issues can surface in two places: within a single dataset and across multiple systems.

Within a dataset, inconsistency often shows up as different representations of the same thing. Imagine a product catalog where some rows list a department as Home & Garden, others as Home and Garden, and still others as home_garden. Each entry might look valid on its own, but when you try to group or filter by department, you get three separate groups instead of one.

Across systems, inconsistency arises when two databases describe the same entity differently. For example, a CRM system might list a customer's address as "100 Main St" while the billing system records "100 Main Street, Suite 5." Neither value is necessarily wrong, but the mismatch can cause errors when the systems need to share or reconcile data.

Validity and consistency serve different but complementary purposes. A value can be valid yet inconsistent: the date 03/15/2024 follows a legitimate date format, but if the rest of the column uses YYYY-MM-DD, it creates a consistency problem. Likewise, a value can be consistent yet invalid: if every row in a column stores ages as negative numbers, the data is perfectly consistent but entirely invalid.

Running both types of checks gives you a much fuller picture of data quality. Validity catches values that break the rules, while consistency catches values that break the pattern. Together, they help you trust that your data is not only correct in isolation but also reliable when combined, compared, or moved across systems.

In this lesson, we explored two essential types of data quality checks. Validity verifies that each value follows defined format, range, and domain rules, while consistency ensures that related values agree within a dataset and across systems. We also learned how to measure validity with a simple rate formula and how to spot consistency gaps by counting unexpected variations.

Up next, you will put these ideas into action with hands-on exercises. You will judge whether specific values are valid or invalid, correct inconsistent formats, and match cross-system conflicts to the consistency issues they represent — let's see how sharp your gatekeeper instincts have become!