Reader small image

You're reading from  Hands-On Web Scraping with Python - Second Edition

Product typeBook
Published inOct 2023
PublisherPackt
ISBN-139781837636211
Edition2nd Edition
Right arrow
Author (1)
Anish Chapagain
Anish Chapagain
author image
Anish Chapagain

Anish Chapagain is a software engineer with a passion for data science, its processes, and Python programming, which began around 2007. He has been working with web scraping and analysis-related tasks for more than 5 years, and is currently pursuing freelance projects in the web scraping domain. Anish previously worked as a trainer, web/software developer, and as a banker, where he was exposed to data and gained further insights into topics including data analysis, visualization, data mining, information processing, and knowledge discovery. He has an MSc in computer systems from Bangor University (University of Wales), United Kingdom, and an Executive MBA from Himalayan Whitehouse International College, Kathmandu, Nepal.
Read more about Anish Chapagain

Right arrow

Web requests

Throughout the chapters of this book, the requests Python library has been used to establish communication between the code and the web. Plenty of Python libraries can be found at https://pypi.org/ if we search for ones similar to requests.

The following subsections list some Python libraries and technologies and provide brief introductions to them.

pycurl

The pycurl Python library (http://pycurl.io/) is a wrapper on top of the popular libcurl library. libcurl is one of the earliest Python libraries that was used to communicate with websites on the internet, based on the curl tool (also known as cURL).

curl (https://curl.se/) is a command-line tool that is used to connect and transfer data over the web. curl is the basis of network communication; it’s a core implementation that is used with the help of a wrapper across different operating systems (OSs), browsers, and machines that communicate with the internet. The curl command is machine-independent...

lock icon
The rest of the page is locked
Previous PageNext Page
You have been reading a chapter from
Hands-On Web Scraping with Python - Second Edition
Published in: Oct 2023Publisher: PacktISBN-13: 9781837636211

Author (1)

author image
Anish Chapagain

Anish Chapagain is a software engineer with a passion for data science, its processes, and Python programming, which began around 2007. He has been working with web scraping and analysis-related tasks for more than 5 years, and is currently pursuing freelance projects in the web scraping domain. Anish previously worked as a trainer, web/software developer, and as a banker, where he was exposed to data and gained further insights into topics including data analysis, visualization, data mining, information processing, and knowledge discovery. He has an MSc in computer systems from Bangor University (University of Wales), United Kingdom, and an Executive MBA from Himalayan Whitehouse International College, Kathmandu, Nepal.
Read more about Anish Chapagain