Scraping data
Scraping data is a common activity used for a wide range of purposes—especially when structured data isn't readily available through an API. Some common uses include:
- Market and competitive research, such as tracking new products, pricing changes, promotions, and customer reviews
- Gathering data for machine learning
- Tracking breaking news
- Monitoring brand reputation
- Compare data against multiple platforms and sources
- Build aggregation tools or sites like Zillow, Kayak, or Apartments.com
In this section, we'll do some basic scraping with the Edge browser and the recorder.
Scraping a single page
To begin scraping, we'll need some sample data. For this exercise, we're going to pull data from one of the sample sites listed earlier in this chapter. The page we're going to focus on has a list of items in an ecommerce store formatted in...