Parsing and Managing CSV Data in Scala

In this lesson, we will explore working with CSV files — a universally employed format for data storage and interchange. By the end of this lesson, you will gain skills in reading data from CSV files, recognizing rows and columns separated by commas, and effectively manipulating string data using Scala. This lesson builds on your prior experience with file parsing using tabs and introduces you to working specifically with CSV data.

CSV stands for Comma-Separated Values, a format that stores tabular data in plain text. Each line represents a data row, with columns separated by commas, facilitating easy storage and interpretation.

Consider a CSV file named data.csv:

Name,Age,Occupation
John,28,Engineer
Alice,34,Doctor
Bob,23,Artist

In this file:

• The first line contains the headers: Name, Age, and Occupation.
• Each subsequent line holds data for an individual, with values separated by commas.

Understanding this structure is crucial as it guides us on parsing the data effectively within our programming environment.

To read all lines from the file and skip the header line, we can utilize the capabilities you are already familiar with:

// Define the file path
val filePath = os.pwd / "data.csv"

// Read all lines from the CSV file
val lines = os.read.lines(filePath)

// Use drop to skip the header and access the data lines
val dataLines = lines.drop(1)

To parse each line of the CSV file, we employ the string split method to extract individual data fields. Each line read from the file is divided into an array of strings, representing the row's columns:

// Parse each data line into columns
val data = dataLines.map { line =>
  // Split each line by commas
  line.split(",")
}

The line.split(",") operation breaks down each line at the commas, storing the result in an array that can be directly manipulated or accessed using indices.

To ensure the CSV data is correctly parsed, we display the structured data. Here's how you can format and verify the output:

// Iterate over each row of parsed data
data.foreach { row =>
  // Format and print each row
  println(row.mkString(", "))
}

The expected output should resemble the following, confirming accurate parsing:

John, 28, Engineer
Alice, 34, Doctor
Bob, 23, Artist

This output verifies that each line from the CSV has been successfully parsed into structured arrays of data.

Now, let's map the parsed data into a list of classes, leveraging your prior knowledge. We'll use a Person case class to organize the data:

case class Person(name: String, age: Int, occupation: String)

In this process:

• We use the Person case class to encapsulate data, mapping each column to its corresponding field.
• Each parsed line is transformed into a Person object, holding structured data.

// Map each line to a Person object
val people = data.map { columns =>
  val name = columns(0)
  val age = columns(1).toInt
  val occupation = columns(2)
  Person(name, age, occupation)
}

// Print each Person object
people.foreach { person =>
  println(s"Name: ${person.name}, Age: ${person.age}, Occupation: ${person.occupation}")
}

The expected output should resemble the following, confirming accurate mapping and printing:

Name: John, Age: 28, Occupation: Engineer
Name: Alice, Age: 34, Occupation: Doctor
Name: Bob, Age: 23, Occupation: Artist

Here is how the complete code looks like, integrating all the steps from reading the CSV file to mapping it into a list of classes and printing the results:

import os._

case class Person(name: String, age: Int, occupation: String)

@main def main() = 
  // Define the file path
  val filePath = os.pwd / "data.csv"
  
  // Read all lines from the CSV file
  val lines = os.read.lines(filePath)
  
  // Use drop to skip the header and access the data lines
  val dataLines = lines.drop(1)
  
  // Parse each data line into columns
  val data = dataLines.map { line =>
    // Split each line by commas
    line.split(",")
  }
  
  // Map each line to a Person object
  val people = data.map { columns =>
    val name = columns(0)
    val age = columns(1).toInt
    val occupation = columns(2)
    Person(name, age, occupation)
  }
  
  // Print each Person object
  people.foreach { person =>
    println(s"Name: ${person.name}, Age: ${person.age}, Occupation: ${person.occupation}")
  }

In this lesson, we covered parsing CSV files using Scala, focusing on reading data with commas as delimiters and managing structured data using arrays and case classes. You've employed techniques you've already mastered, such as os-lib for file operations and string manipulation, to efficiently handle CSV content.

As you move forward to the practice exercises, verify the correctness of your parsed data and explore potential applications using Scala's functional programming paradigms for advanced data-handling techniques. Continue your excellent work, and keep exploring Scala's rich capabilities for data manipulation and processing.