Parsing Raw Text Data

Welcome back to Practical Data Processing with Python Collections! Our toolkit is filling up nicely: we can tally and group with dictionaries, and we can transform and filter lists in a single readable line.

So far, though, our data has always arrived prepackaged as Python lists and dictionaries. Real data rarely shows up that way. It comes as raw text: a line exported from a spreadsheet, a block pasted from a form, or the contents of a simple .csv file. Before we can count, group, or summarize anything, we have to convert that text into structured Python objects.

In this lesson, we will build a converter for simple, well-behaved delimited text, using only tools we already have: split, strip, list comprehensions, and dictionaries. This hand-built approach is great for learning exactly how parsing works, and it is genuinely useful for tidy data we generate ourselves or receive in a predictable format. Real-world .csv files can be messier, though, and near the end of this lesson we will briefly meet Python's standard csv module, the tool built specifically to handle those messier cases. By the end, we will be able to:

• Split a block of raw text into individual lines
• Extract and clean the column names from the header line
• Pair each row's fields with those headers to form a record dictionary
• Convert numeric text like "36" into a real integer
• Recognize when hand-written splitting is not enough, and know which standard tool to reach for instead

Let us start with our data exactly as it might arrive: one long string containing several lines. Python's triple-quoted strings let us write that literally, including the newlines.

raw = """name,age,city
Ada, 36 , London
Max,29,Paris
Sam, 41 ,Berlin"""

lines = raw.split("\n")

Notice the deliberate messiness: some values carry extra spaces around them, such as " 36 " and " London", while others are tidy. Simple exports like this are common, so our parser must handle both.

The newline character \n marks the end of each line inside the string, so raw.split("\n") uses it as the delimiter and hands us a list of four strings: one header line, "name,age,city", followed by three data lines. Each item is still a single string with commas inside it; we have separated the rows, but not yet the fields.

By convention, the first line of a delimited file names the columns rather than holding data. That makes lines[0] special: it tells us what each field in every following row means.

# The first line holds the column headers
headers = [h.strip() for h in lines[0].split(",")]

Reading this from the inside out: lines[0].split(",") breaks "name,age,city" into three raw field names, and the comprehension applies strip() to each one. The result is the clean list ["name", "age", "city"].

Stripping headers may look unnecessary here, but it is the cheapest insurance we can buy. These strings will become dictionary keys, so a single stray space would give us a key of " age" instead of "age" in every record, and later lookups like record["age"] would fail.

Before looping over everything, let us handle just one row so the idea stays small. Suppose line holds "Ada, 36 , London".

values = [v.strip() for v in line.split(",")]
record = dict(zip(headers, values))

We reuse the exact same split-and-strip comprehension, which turns the line into ["Ada", "36", "London"]. Then zip(headers, values) walks both lists in parallel and pairs them up by position, and dict(...) converts those pairs into keys and values:

As the table shows, position is everything: zip assumes the two lists line up. When the lists have different lengths, zip() stops at the shorter list, so a row with only two values would produce a record with no 'city' entry and no error to tell us. Our record is now the dictionary {'name': 'Ada', 'age': '36', 'city': 'London'}, where every value is still a string.

One record is nice; we want all of them. We collect them in a list, processing one line at a time.

records = []
for line in lines[1:]:
    values = [v.strip() for v in line.split(",")]
    record = dict(zip(headers, values))
    records.append(record)

The slice lines[1:] is the key detail: it starts at index 1, which skips the header line we already consumed. Without it, "name,age,city" would be parsed as if it were a person named "name".

We could ask why this is not a comprehension, given our emphasis on concise list building. The answer is that the body already does three separate things, and it is about to do a fourth. A regular for loop stays readable when several statements belong together. Right now, records is a list of three dictionaries whose values are all strings.

String values hide a real problem: "36" cannot be added, averaged, or compared numerically, since "9" > "36" is True for text. So we convert the numeric field right after the record exists.

records = []
for line in lines[1:]:
    values = [v.strip() for v in line.split(",")]
    record = dict(zip(headers, values))
    # Convert the numeric field from string to int
    record["age"] = int(record["age"])
    records.append(record)

print("Records:", records)
print("First person's age:", records[0]["age"])

The assignment reads the string under "age", passes it to int(), and writes the integer back over it. Running the full program prints:

Records: [{'name': 'Ada', 'age': 36, 'city': 'London'}, {'name': 'Max', 'age': 29, 'city': 'Paris'}, {'name': 'Sam', 'age': 41, 'city': 'Berlin'}]
First person's age: 36

The ages appear without quotes, confirming that they are integers, while the names and cities keep theirs. And records[0]["age"] reaches into the first record for a value we can now do math with.

Parsing code fails in a few predictable ways, and knowing them in advance saves a lot of debugging time.

• Forgetting lines[1:]: iterating over all of lines turns the header into a bogus record, and int("age") then raises an error.
• Skipping strip() before int(): int() tolerates surrounding whitespace, so int(" 36 ") works, but a stray character such as "36 years" does not.
• Non-numeric fields: int() raises a ValueError on anything it cannot read as a whole number, so int("N/A") or int("29.5") will stop the program.
• Trusting split(",") on real CSV exports: quoted fields that contain commas or embedded newlines will be split incorrectly, scrambling the resulting record; the csv module described just below is built to handle those cases.
• Mismatched lengths: if a row has fewer fields than there are headers, zip silently stops at the shorter list, and the record quietly loses a key.

The last one is the sneakiest because nothing crashes: we simply end up with an incomplete record that surfaces as a problem much later.

Everything we just built works because our sample data is deliberately tidy: every field is separated by a plain comma, and no field itself contains a comma, a quotation mark, or a line break. That description fits plenty of small, self-generated datasets and classroom exercises, but it does not fit every file with a .csv extension.

Real spreadsheet exports often quote a field that contains the delimiter, so a perfectly valid line can look like this:

name,age,city
"Smith, Ada",36,London

The person's name contains a comma, so the exporting program wrapped the whole field in quotation marks to signal "this comma is part of the value, not a separator." If we ran '"Smith, Ada",36,London'.split(",") on that line, we would get four fields instead of three — ['"Smith', ' Ada"', '36', 'London'] — and zip would then pair every value with the wrong header for the rest of the row. Quoted fields can even contain a newline character, which would confuse our raw.split("\n") step before we ever reached the commas.

The technique from this lesson, then, is best scoped to what it actually handles well: understanding how delimited parsing works, and processing simple, well-behaved data that we generate or fully control ourselves, such as the small exports in these practices. For real .csv files coming from spreadsheets, databases, or other people's programs, Python's standard library already provides a module built exactly for this job.

import csv

reader = csv.DictReader(raw.split("\n"))
records = list(reader)

csv.DictReader correctly understands quoting, embedded delimiters, and embedded newlines, and it hands back a dictionary per row keyed by the header, much like the records list we built by hand, but reliably, even on messy real-world files. We will not explore its full set of options in this course, but knowing it exists — and reaching for it whenever we parse an actual .csv file outside of a controlled exercise — will save us from subtle bugs that manual splitting cannot handle.

We just built a complete parsing pipeline out of four small steps: split the raw text into lines, clean the header line into a list of keys, zip each remaining row against those headers to form a record dictionary, and convert the fields that should be numbers into real numbers. Within its scope, this pattern is flexible: swap the delimiter passed to split and adjust which keys get converted, and the same steps handle other simple, unquoted formats such as tab-separated exports or a table with twenty columns, as long as no field contains the delimiter itself. The moment quoting or embedded delimiters enter the picture, reach for the csv module instead, as we just saw.

The practices ahead follow that same progression: pulling clean headers out of a raw block of text, mapping a single row into a dictionary, looping to build the full list of records, and converting a numeric field so a typed value comes back. Those typed records are ready to be turned into totals, averages, and neatly formatted reports. Let us go parse some text and get our data into shape.