Parsing and Managing CSV Data in Scala

Introduction and Context Setting

In this lesson, we will explore working with CSV files — a universally employed format for data storage and interchange. By the end of this lesson, you will gain skills in reading data from CSV files, recognizing rows and columns separated by commas, and effectively manipulating string data using Scala . This lesson builds on your prior experience with file parsing using tabs and introduces you to working specifically with CSV data.

Understanding CSV Structure and Delimiter

CSV stands for Comma-Separated Values, a format that stores tabular data in plain text. Each line represents a data row, with columns separated by commas, facilitating easy storage and interpretation. Consider a CSV file named data.csv : Name,Age,Occupation John,28,Engineer Alice,34,Doctor Bob,23,Artist Name,Age,Occupation John,28,Engineer Alice,34,Doctor Bob,23,Artist In this file: The first line contains the headers: Name, Age, and Occupation . Each subsequent line holds data for an individual, with values separated by commas. Understanding this structure is crucial as it guides us on parsing the data effectively within our programming environment.

Reading CSV Data

To read all lines from the file and skip the header line, we can utilize the capabilities you are already familiar with: Scala // Define the file path val filePath = os.pwd / "data.csv" // Read all lines from the CSV file val lines = os.read.lines(filePath) // Use drop to skip the header and access the data lines val dataLines = lines.drop(1) // Define the file path val filePath = os.pwd / "data.csv" // Read all lines from the CSV file val lines = os.read.lines(filePath) // Use drop to skip the header and access the data lines val dataLines = lines.drop(1)

Parsing Each Line

To parse each line of the CSV file, we employ the string split method to extract individual data fields. Each line read from the file is divided into an array of strings, representing the row's columns: Scala // Parse each data line into columns val data = dataLines.map { line => // Split each line by commas line.split(",")} // Parse each data line into columns val data = dataLines.map { line => // Split each line by commas line.split(",")} The line.split(",") operation breaks down each line at the commas, storing the result in an array that can be directly manipulated or accessed using indices.

Verifying Parsed Output

To ensure the CSV data is correctly parsed, we display the structured data. Here's how you can format and verify the output: Scala // Iterate over each row of parsed data data.foreach { row => // Format and print each row println(row.mkString(", "))} // Iterate over each row of parsed data data.foreach { row => // Format and print each row println(row.mkString(", "))} The expected output should resemble the following, confirming accurate parsing: text John, 28, Engineer Alice, 34, Doctor Bob, 23, Artist John, 28, Engineer Alice, 34, Doctor Bob, 23, Artist This output verifies that each line from the CSV has been successfully parsed into structured arrays of data.

Mapping to a List of Classes

Now, let's map the parsed data into a list of classes, leveraging your prior knowledge. We'll use a Person case class to organize the data: Scala case class Person(name: String, age: Int, occupation: String) In this process: We use the Person case class to encapsulate data, mapping each column to its corresponding field. Each parsed line is transformed into a Person object, holding structured data. Scala // Map each line to a Person object val people = data.map { columns => val name = columns(0) val age = columns(1).toInt val occupation = columns(2) Person(name, age, occupation)} // Print each Person object people.foreach { person => println(s"Name: ${person.name}, Age: ${person.age}, Occupation: ${person.occupation}")} The expected output should resemble the following, confirming accurate mapping and printing: text Name: John, Age: 28, Occupation: Engineer Name: Alice, Age: 34, Occupation: Doctor Name: Bob, Age: 23, Occupation: Artist

Complete Code Example

Here is how the complete code looks like, integrating all the steps from reading the CSV file to mapping it into a list of classes and printing the results: Scalaimport os._ case class Person(name: String, age: Int, occupation: String) @main def main() = // Define the file path val filePath = os.pwd / "data.csv" // Read all lines from the CSV file val lines = os.read.lines(filePath) // Use drop to skip the header and access the data lines val dataLines = lines.drop(1) // Parse each data line into columns val data = dataLines.map { line => // Split each line by commas line.split(",") } // Map each line to a Person object val people = data.map { columns => val name = columns(0) val age = columns(1).toInt val occupation = columns(2) Person(name, age, occupation) } // Print each Person object people.foreach { person => println(s"Name: ${person.name}, Age: ${person.age}, Occupation: ${person.occupation}") }

Summary and Preparing for Practice

In this lesson, we covered parsing CSV files using Scala, focusing on reading data with commas as delimiters and managing structured data using arrays and case classes. You've employed techniques you've already mastered, such as os-lib for file operations and string manipulation, to efficiently handle CSV content. As you move forward to the practice exercises, verify the correctness of your parsed data and explore potential applications using Scala's functional programming paradigms for advanced data-handling techniques. Continue your excellent work, and keep exploring Scala's rich capabilities for data manipulation and processing.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal