Web scraping using PyQuery
This section is loaded with examples that explain how to code a web scraping script. We will be using the PyQuery and requests libraries and dealing with HTML and XML content from the web. We will write the data collected in these examples to CSV or JSON files.
Example 1 – scraping book details
In this example, we will be scraping books listed on http://books.toscrape.com in the Childrens category (http://books.toscrape.com/catalogue/category/books/childrens_11/index.html). This example is similar to another provided in Chapter 3, where the lxml library was used. The code for this example is available on GitHub: https://github.com/PacktPublishing/Hands-On-Web-Scraping-with-Python-Second-Edition/blob/main/Chapter04/example_1.ipynb.
As you can see in Figure 4.3, the Childrens category contains 29 results (a single page shows only 20 results):
Figure 4.3: Childrens category page with 29 results
Important note
...
