Welcome to this lesson, where we'll explore an essential technique in text data manipulation: reading files line-by-line. In many real-world applications, processing data one line at a time is crucial for effective data management, particularly when dealing with large files like logs or data streams. By the end of this lesson, you'll understand how to efficiently read and process file data line-by-line, leveraging JavaScript's built-in functionality with Node.js.

In JavaScript, particularly when using Node.js, we use the fs module to handle file operations. To open a file and read its contents, we'll make use of the fs.readFileSync() function. This allows us to read the entire file into a string. Here's an example:

JavaScript
Copy to clipboard
1const fs = require('fs');
2const filePath = 'input.txt';
3const content = fs.readFileSync(filePath, 'utf-8');

In this example, filePath is a string indicating the location of your file, and content is a string containing the entire file's content, as read synchronously.

To read a file line-by-line in JavaScript, we can use the split() method to divide the file content string into an array of lines. Let's break down how to achieve this:

JavaScript
Copy to clipboard
1const fs = require('fs');
2const filePath = 'input.txt';
3const content = fs.readFileSync(filePath, 'utf-8');
4const lines = content.split(/\r?\n/);

Here, lines is an array containing all lines from input.txt, each as a separate string. For example, if the file's content is:

Plain text
Copy to clipboard
1Hello,
2world
3!

Our lines array will be: ["Hello,", "world", "!"], following the split operation.

The regular expression /\r?\n/ used in the .split method accounts for both Unix (\n) and Windows (\r\n) newline conventions, ensuring compatibility across operating systems. A simple split('\n') would miss splitting lines on \r\n (Windows) or \r (older Mac files).

Once you have your file lines in an array, you can iterate over them using a for...of loop. During this process, it's often necessary to clean up the output. The trim() method is a handy tool to remove unwanted newline characters and extra spaces. Let's see this in action:

JavaScript
Copy to clipboard
1const fs = require('fs');
2const filePath = 'input.txt';
3const content = fs.readFileSync(filePath, 'utf-8');
4const lines = content.split(/\r?\n/);
5
6for (const line of lines) {
7  console.log(line.trim());
8}

• Looping Over Lines: The for...of loop goes through each line in the lines array.
• Using trim(): This method is applied to each line to remove any leading and trailing whitespace, including newline characters.

The output of this code will neatly display each line from input.txt without extra newlines:

Plain text
Copy to clipboard
1Hello,
2world
3!

To extend our file line-by-line reading skills, let's look at an example where we read integers from a file and calculate their sum. Here's how you can do it:

Assume the numbers.txt file contains:

Plain text
Copy to clipboard
110
220
330
440

The following code reads integers from this file and calculates their sum:

JavaScript
Copy to clipboard
1const fs = require('fs');
2const filePath = 'numbers.txt';
3const content = fs.readFileSync(filePath, 'utf-8');
4const lines = content.split(/\r?\n/);
5
6let totalSum = 0;
7
8for (const line of lines) {
9  const number = parseInt(line.trim(), 10); // Convert each line to an integer
10  if (!isNaN(number)) { // Ensure the line is a valid number
11    totalSum += number; // Add the integer to totalSum
12  }
13}
14
15console.log("The sum of the numbers is:", totalSum);

• Reading Lines: The file's lines containing numbers are read into the lines array.
• Converting to Integers: Each line is stripped of whitespace and converted to an integer using parseInt().
• Calculating Sum: The converted integers are summed up in the totalSum variable.

Once you execute the code, it will output the total sum of the numbers:

Plain text
Copy to clipboard
1The sum of the numbers is: 100

In this lesson, you gained the skills to read a text file line-by-line using JavaScript — a fundamental technique for processing large datasets efficiently. You've learned to manage file I/O operations effectively with the fs module, explore splitting content with split(), and clean data with trim().

Now, you're ready to dive into practice exercises where you can apply these concepts and strengthen your understanding. Continue to build on these skills as we explore further parsing techniques in future lessons. Keep up the great work!