Extracting Images from Websites

Welcome to the lesson on extracting and saving images from web pages. In this lesson, you will learn how to use JavaScript with the Cheerio and Axios libraries to scrape images from web pages and save them locally. By the end of this lesson, you will have a solid understanding of the entire process, from making web requests to locating image elements and saving the images.

We start by fetching the HTML content of the website we want to scrape. In this case, we'll use https://books.toscrape.com/.

First, import the necessary libraries, make an HTTP GET request to the website, and parse the HTML content using Cheerio.

const axios = require('axios');
const cheerio = require('cheerio');

(async () => {
    const url = 'https://books.toscrape.com/';
    const response = await axios.get(url);
    const $ = cheerio.load(response.data);
})();

In this example, we fetch and parse the HTML content of the Books website.

With the parsed HTML content, use Cheerio to locate image elements and extract their URLs from the src attribute:

const images = $('img');
const imageUrls = images.map((index, element) => $(element).attr('src')).get();

We now have a list of image URLs extracted from the web page.

Finally, we will download and save the extracted images to the local file system.

Let's first ensure the images directory exists and create it if it doesn't, using the fs module:

const fs = require('fs');
const path = require('path');

if (!fs.existsSync('images')) {
    fs.mkdirSync('images');
}

Next, we can iterate over the image URLs, send requests to each URL, and save the images:

(async () => {
    for (const src of imageUrls) {
        const fullSrc = src.startsWith('http') ? src : `https://books.toscrape.com/${src}`;
        
        try {
            const imgResponse = await axios.get(fullSrc, { responseType: 'stream' });
            
            if (imgResponse.status === 200) {
                const imgName = path.basename(src); // Extract the image name from the URL
                const filePath = path.join('images', imgName);
                
                const writer = fs.createWriteStream(filePath);
                imgResponse.data.pipe(writer);
                
                await new Promise((resolve, reject) => {
                    writer.on('finish', resolve);
                    writer.on('error', reject);
                });
                
                console.log(`Saved ${imgName}`);
            }
        } catch (error) {
            console.error(`Failed to download image from ${fullSrc}:`, error.message);
        }
    }
})();

After running the code, all the images will be saved in the images directory. Let's understand the code step by step:

1. We iterate over the image URLs extracted from the web page. Note that we construct the full URL by prepending the base URL if the image URL is relative.
2. For each URL, we send an HTTP GET request to download the image using Axios with responseType: 'stream' for efficient handling of binary data.
3. If the request is successful (status code 200), we extract the image name from the URL and create a write stream to save the image to the images directory. We use the pipe() method to efficiently transfer data from the response stream to the file.
4. We wait for the write operation to complete using a Promise that resolves when the stream finishes writing.
5. Finally, we print a message indicating that the image was saved or log any errors that occurred during the download process.

In this lesson, you learned how to extract and save images from web pages using JavaScript with the Cheerio and Axios libraries. You learned how to make web requests, parse HTML content, locate image elements, extract image URLs, and save images to the local file system.

Now it's time to practice what you've learned in the exercises. Good luck!