Welcome to another lesson, where we'll explore a fundamental technique in text data manipulation: reading files line-by-line using Go. In many real-world applications, processing data one line at a time is crucial for effective data management, particularly when dealing with large files like logs or data streams. By the end of this lesson, you'll understand how to efficiently read and process file data line-by-line using Go.

File handling can be accomplished in Go using the bufio package. Specifically, bufio.NewScanner provides a convenient method for reading files one line at a time. This scanner reads the file line by line, which is ideal for handling large files efficiently because it does not require the entire file to be loaded into memory:

Go
Copy to clipboard
1import "bufio"
2
3scanner := bufio.NewScanner(file)
4for scanner.Scan() {
5    line := scanner.Text()
6    // Process each line
7}

• file: Refers to the opened file object.
• scanner.Scan(): Advances the scanner to the next line.
• scanner.Text(): Retrieves the current line from the scanner.

This approach is perfect for processing large text files line by line efficiently.

To read a file line by line in Go, use bufio.NewScanner and iterate through the lines using a for loop. Let's see this in action:

Go
Copy to clipboard
1package main
2
3import (
4    "bufio"
5    "fmt"
6    "log"
7    "os"
8)
9
10func main() {
11    filePath := "data.txt"
12    file, err := os.Open(filePath)
13    if err != nil {
14        log.Fatal(err)
15    }
16    defer file.Close()
17
18    fmt.Println("Reading file line by line:")
19
20    scanner := bufio.NewScanner(file)
21    for scanner.Scan() {
22        line := scanner.Text()
23        fmt.Println(line) // Output each line to the console
24    }
25
26    if err := scanner.Err(); err != nil {
27        log.Fatal(err)
28    }
29}

Here, bufio.NewScanner(file) reads lines from data.txt one by one. Each line is stored in the line variable and then printed to the console.

Imagine needing to process a set of numbers stored in a file — reading each integer and calculating their sum. This is a practical task you can accomplish using a combination of bufio.NewScanner and strconv.Atoi for integer parsing.

Assume numbers.txt contains:

Plain text
Copy to clipboard
110
220
330
440

The following code reads integers from this file and calculates their sum:

Go
Copy to clipboard
1package main
2
3import (
4    "bufio"
5    "fmt"
6    "log"
7    "os"
8    "strconv"
9)
10
11func main() {
12    filePath := "numbers.txt"
13    totalSum := 0
14
15    file, err := os.Open(filePath)
16    if err != nil {
17        log.Fatal(err)
18    }
19    defer file.Close()
20
21    scanner := bufio.NewScanner(file)
22    for scanner.Scan() {
23        line := scanner.Text()
24        number, err := strconv.Atoi(line)
25        if err == nil {
26            totalSum += number // Add the integer to totalSum
27        }
28    }
29
30    if err := scanner.Err(); err != nil {
31        log.Fatal(err)
32    }
33
34    fmt.Println("The sum of the numbers is:", totalSum)
35}

• Reading Numbers: Each line is parsed to an integer using strconv.Atoi(), ensuring safe conversion.
• Calculating Sum: The integers are added together in the totalSum variable.

After executing this code, the output displays the total sum of the numbers:

Plain text
Copy to clipboard
1The sum of the numbers is: 100

In this lesson, you gained the skills to read a text file line by line using Go — a fundamental technique for processing large datasets efficiently. You've learned to manage file I/O operations using the bufio.NewScanner method and how to handle each line effectively.

Now, you're ready to dive into practice exercises where you can apply these concepts and strengthen your understanding. Continue to build on these skills as we explore further parsing techniques in future lessons. Keep up the great work!