Exploring Data Retrieval with Python's 'requests' Library
Lesson Overview
Welcome! In this lesson, we will take our first steps into the world of gathering data from the Web in Python using the requests library. You will understand how to retrieve web pages and display their content. Let's get our hands dirty with requests!
Understanding Web Requests and the `requests` Library
In modern web development, data exchange between the client (your web browser or application) and the server (where the data is stored) is handled through HTTP requests. We generally use four types of requests, namely GET, POST, PUT, and DELETE, for fetching, sending, updating, and deleting data respectively. But for now, let's focus on the GET request, which we use to fetch data, such as the HTML code of a web page.
Python provides us with a wonderful library, requests, to handle these HTTP requests with ease in our Python programs. The requests library abstracts the complexities of making HTTP requests behind a simple API, allowing you to send HTTP requests with just a few lines of code.
Fetching Content from a Website Using `requests.get()`
Understanding the concept of HTTP requests, let's move on to how we can fetch a web page's content using Python requests.
Here, we have imported the requests library and then used the get function to send a GET request to the URL http://quotes.toscrape.com. The response from the server is stored in the variable response.
Validating the Successfulness of the Fetch Operation
How do we know if our fetch operation was successful? It's quite simple - we check the HTTP response status code. A status code of 200 means the request was successful. Anything in the range of 400-499 indicates a client-side error, and anything between 500-599 indicates a server-side error.
Our response object has an attribute ok which returns True if our request was successful (status code less than 400). Let's write some code to validate this:
The output of the above code will be:
This output confirms that the content was successfully fetched from the provided URL.
