Text Processing with awk

Hello! Welcome to your next step in mastering Bash scripting. In this lesson, we will immerse ourselves in the world of text processing with the versatile command-line tool awk. awk is a powerful tool that allows you to manipulate and analyze text files with ease. By the end of this lesson, you’ll be equipped to efficiently handle and process text files, extracting meaningful data and performing relevant computations directly from your Bash scripts.

Let's get started by diving into how we can leverage awk for various text processing tasks.

First, let's create a sample data file to work with. This file will help us learn and practice various awk commands effectively.

The heredoc (short for "here document") is a special syntax in Unix shell scripting that allows you to create a multi-line string. It is particularly useful for creating files or including large blocks of text within your script. The syntax is <<EOF ... EOF, where EOF (End of File) is a marker indicating the beginning and end of the block of text. You can actually use any marker, but EOF is conventionally used.

Let's create a file called data.txt that includes data about computers in inventory.

#!/bin/bash

# Create a sample data file
cat << EOF > data.txt
Brand   Model     RAM
Apple   MacBook    32
Apple   iPad       16
Dell    XPS        32
Dell    Inspiron  128
Lenovo  ThinkPad  128
Lenovo  Yoga      256
Apple   MacBook    64
EOF

Let's break this code down:

• cat << EOF: This starts the heredoc and tells the cat command to begin reading the subsequent lines as a string until it encounters the ending EOF marker.
• > data.txt: This redirects the output of the cat command to a file named data.txt.
• The lines between << EOF and EOF are the content that will be written to data.txt.

The basic syntax of the awk command in Unix-like systems is:

awk options 'selection_criteria {action}' input-file > output-file

Here's a detailed breakdown of each component:

• awk: The command itself.
• options: These are optional flags you can pass to awk to modify its behavior (e.g., -F to specify the field separator).
• selection_criteria: This is an optional condition or pattern that specifies which lines of the input file to process. It can be a regular expression or a logical condition based on field values.
• {action}: This is the block of code to execute for each line that matches the selection criteria. Actions are enclosed in curly braces {}.
• input-file: The file that awk processes.
• > output-file: This optional part redirects the output to a file. If omitted, awk prints the output to the terminal.

With this understanding of awk syntax, let's dive into some examples.