Extracting Text with Cheerio

Introducing Cheerio and jQuery-like Selectors

Hello and welcome! In this lesson, we're learning about extracting text from paragraphs using Cheerio. As you might recall from our previous lesson, Cheerio provides a server-side implementation of core jQuery functionality, allowing us to traverse and manipulate HTML documents using familiar CSS selectors.

In this lesson, we will often use Cheerio's $() function — a powerful tool that uses CSS selectors to find elements in a document and returns a Cheerio object, allowing us to extract text from specific HTML elements. Cheerio objects are jQuery-like collections that can be iterated over to access the content of the HTML elements. This approach is versatile, allowing us to filter HTML elements by their tag name, attributes, classes, IDs, or even by their position within the document using CSS selector syntax.

Extracting Paragraphs from HTML Content

Now, let's define a simple HTML content and extract text from paragraph tags using Cheerio. Here's a quick example:

JavaScript
const cheerio = require('cheerio');

const html_content = `
<html><body>
<p>Hello, World!</p>
<p>Welcome to web scraping with Cheerio.</p>
</body></html>
`;

const $ = cheerio.load(html_content);

Remember our $() function? You can use it with CSS selectors to locate all p tags in our loaded HTML:

JavaScript
const paragraphs = $('p');
console.log(paragraphs.length); // Shows how many paragraphs were found

To see the actual HTML elements, you can iterate through them:

JavaScript
paragraphs.each((index, element) => {
    console.log($(element).toString());
});

The output of the above code will be:

HTML
<p>Hello, World!</p>
<p>Welcome to web scraping with Cheerio.</p>

This output demonstrates how Cheerio can easily locate all p tags within our HTML content using CSS selectors, returning them as a Cheerio object that we can work with. It's a foundational step for extracting data from specific HTML elements.

Want just the raw text, with no HTML tags? You can access the text of each element using the .text() method:

JavaScript
paragraphs.each((index, element) => {
    console.log($(element).text());
});

The output of the above code will be:

plaintext
Hello, World!
Welcome to web scraping with Cheerio.

This illustrates how easily you can extract and directly work with the text content of HTML elements, stripping away the HTML markup to get to the raw information you're after.

And just like that, you've extracted text from the paragraph tags in your HTML!

Extracting Paragraphs with Specific Classes Using CSS Selectors

In addition to extracting all paragraph tags, Cheerio's CSS selector approach allows us to narrow down our search to elements with specific attributes, such as class names. This is particularly useful when working with HTML documents that use CSS classes to style or categorize similar elements in different ways.

By using CSS class selectors in the $() function, we can filter elements based on their class attribute. We can use the standard CSS syntax p.classname to select paragraphs with specific classes.

Let's dive into an example to see how this works:

JavaScript
const cheerio = require('cheerio');

const html_content = `
<html><body>
<div id="main">
    <h1>Welcome</h1>
    <p>Learn web scraping.</p>
    <p class="special">Special paragraph about Cheerio</p>
    <p class="special">More exciting special paragraph about Cheerio</p>
</div>
</body></html>
`;

const $ = cheerio.load(html_content);

// Access paragraphs with 'special' class using CSS selector
const special_paragraphs = $('p.special');
console.log("Special paragraphs:");

const texts = [];
special_paragraphs.each((index, element) => {
    texts.push($(element).text());
});
console.log(texts);

In this code snippet, we are interested in extracting paragraphs that have been assigned the class special. By using the CSS selector p.special in the $() function, we successfully filter out only those <p> tags that have the class special.

The output of the code will be:

plaintext
Special paragraphs:
['Special paragraph about Cheerio', 'More exciting special paragraph about Cheerio']

This output demonstrates the effectiveness of Cheerio's CSS selectors in not only finding all instances of a tag but also in filtering tags based on their attributes. Here, only paragraphs with the class special are accessed and their texts extracted, leaving behind any other paragraph tags without the said class.

Incorporating CSS selector-based filtering adds an extra layer of precision to our web scraping tasks, enabling us to target and extract specific data sections within vast and complex HTML documents.

Lesson Summary and Practice

Congratulations! You've just taken another step in mastering web scraping with Cheerio. Today, we learned to utilize Cheerio's selector methods for locating and extracting elements within an HTML document using familiar CSS selector syntax. We then went a step further, exploring how to extract only the raw text from these elements.

In our upcoming practice exercises, you'll get a chance to flex your new Cheerio skills and solidify your understanding of these concepts. We will focus on hands-on experience, guiding you to write your own code for extracting text from different HTML tag types, such as headers or links.

Remember, practice is the best way to grasp and reinforce new concepts. Happy coding!

Previous LessonNext Lesson: Navigating HTML Tree Structures
Sign up

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal