Home Cloud & Networking Python Network Programming Cookbook - Second Edition

Python Network Programming Cookbook - Second Edition

By Pradeeban Kathiravelu , Gary Berger , Dr. M. O. Faruque Sarker
books-svg-icon Book
eBook $43.99
Print $54.99
Subscription $15.99
$10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
BUY NOW $10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
eBook $43.99
Print $54.99
Subscription $15.99
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
  1. Free Chapter
    Sockets, IPv4, and Simple Client/Server Programming
About this book
Python Network Programming Cookbook - Second Edition highlights the major aspects of network programming in Python, starting from writing simple networking clients to developing and deploying complex Software-Defined Networking (SDN) and Network Functions Virtualization (NFV) systems. It creates the building blocks for many practical web and networking applications that rely on various networking protocols. It presents the power and beauty of Python to solve numerous real-world tasks in the area of network programming, network and system administration, network monitoring, and web-application development. In this edition, you will also be introduced to network modelling to build your own cloud network. You will learn about the concepts and fundamentals of SDN and then extend your network with Mininet. Next, you’ll find recipes on Authentication, Authorization, and Accounting (AAA) and open and proprietary SDN approaches and frameworks. You will also learn to configure the Linux Foundation networking ecosystem and deploy and automate your networks with Python in the cloud and the Internet scale. By the end of this book, you will be able to analyze your network security vulnerabilities using advanced network packet capture and analysis techniques.
Publication date:
August 2017
Publisher
Packt
Pages
450
ISBN
9781786463999

 

Sockets, IPv4, and Simple Client/Server Programming

In this chapter, we will cover the following recipes:

  • Printing your machine's name and IPv4 address
  • Retrieving a remote machine's IP address
  • Converting an IPv4 address to different formats
  • Finding a service name, given the port and protocol
  • Converting integers to and from host to network byte order
  • Setting and getting the default socket timeout
  • Handling socket errors gracefully
  • Modifying a socket's send/receive buffer size
  • Changing a socket to the blocking/non-blocking mode
  • Reusing socket addresses
  • Printing the current time from the internet time server
  • Writing an SNTP client
  • Writing a simple TCP echo client/server application
  • Writing a simple UDP echo client/server application
 

Introduction

This chapter introduces Python's core networking library through some simple recipes. Python's socket module has both class-based and instances-based utilities. The difference between a class-based and instance-based method is that the former doesn't need an instance of a socket object. This is a very intuitive approach. For example, in order to print your machine's IP address, you don't need a socket object. Instead, you can just call the socket's class-based methods. On the other hand, if you need to send some data to a server application, it is more intuitive that you create a socket object to perform that explicit operation. The recipes presented in this chapter can be categorized into three groups as follows:

  • In the first few recipes, the class-based utilities have been used in order to extract some useful information about host, network, and any target service.
  • After that, some more recipes have been presented using the instance-based utilities. Some common socket tasks, including manipulating the socket timeout, buffer size, and blocking mode has been demonstrated.
  • Finally, both class-based and instance-based utilities have been used to construct some clients, which perform some practical tasks, for example, synchronizing the machine time with an internet server or writing a generic client/server script.

You can use these demonstrated approaches to write your own client/server application.

 

Printing your machine's name and IPv4 address

Sometimes, you need to quickly discover some information about your machine, for example, the hostname, IP address, number of network interfaces, and so on. This is very easy to achieve using Python scripts.

Getting ready

You need to install Python on your machine before you start coding. Python comes preinstalled in most of the Linux distributions. For Microsoft Windows operating systems, you can download binaries from the Python website: http://www.python.org/download/.

Currently, Python 3.x is released in addition to Python 2.x. Many of the current Linux distributions and macOS versions are still shipping Python 2 by default. However, some ship both of them.

Download the relevant installer for your operating system and the relevant version based on whether your operating system is 32 bit or 64 bit.

You may consult the documentation of your operating system to check and review your Python setup. After installing Python on your machine, you can try opening the Python interpreter from the command line by typing python. This will show the interpreter prompt, >>>, which should be similar to the following output:

~$ python
Python 2.7.12 (default, Nov 19 2016, 06:48:10) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 
  

How to do it...

In the latter versions of Ubuntu since Ubuntu 14.04, Python 3 can be executed by typing python3:

~$ python3
Python 3.5.2 (default, Nov 17 2016, 17:05:23) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or 
"license" for more information.
>>>

Similarly, to be specific about which version you prefer to use, you may type python2 to execute Python 2 as well:

~$ python2
Python 2.7.12 (default, Nov 19 2016, 06:48:10) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

There are a few changes in Python 3 that made some code written for Python 2 incompatible with Python 3. When you write network applications, try to follow the Python 3 best practices as these changes and improvements are back ported to the latter versions of Python 2. Thus, you may be fine by running the latest versions of Python 2 such as Python 2.7. However, some code developed focusing on Python 2 may not run on Python 3.

The following recipes in this chapter are written in Python 3. However, please keep in mind that a few network projects and modules may have been developed for Python 2. In that case, you will either have to port the application to Python 3 or use Python 2 depending on your requirements.

As this recipe is very short, you can try this in the Python interpreter interactively.

First, we need to import the Python socket library with the following command:

>>> import socket  

Then, we call the gethostname() method from the socket library and store the result in a variable as follows:

>>> host_name = socket.gethostname()
>>> print "Host name: %s" %host_name
Host name: llovizna
>>> print "IP address: %s" 
%socket.gethostbyname(host_name)
IP address: 127.0.1.1

The entire activity can be wrapped in a free-standing function, print_machine_info(), which uses the built-in socket class methods.

We call our function from the usual Python __main__ block. During runtime, Python assigns values to some internal variables such as __name__. In this case, __name__ refers to the name of the calling process. When running this script from the command line, as shown in the following command, the name will be __main__. But it will be different if the module is imported from another script. This means that, when the module is called from the command line, it will automatically run our print_machine_info function; however, when imported separately, the user will need to explicitly call the function.

Listing 1.1 shows how to get our machine info, as follows:

#!/usr/bin/env python
# Python Network Programming Cookbook,
Second Edition -- Chapter - 1 # This program is optimized for Python 2.7.12
and Python 3.5.2. # It may run on any other version with/without
modifications. import socket def print_machine_info(): host_name = socket.gethostname() ip_address = socket.gethostbyname(host_name) print ("Host name: %s" %host_name) print ("IP address: %s" %ip_address) if __name__ == '__main__': print_machine_info()

In order to run this recipe, you can use the provided source file from the command line as follows:

$ python 1_1_local_machine_info.py  

On my machine, the following output is shown:

Host name: llovizna
IP address: 127.0.1.1
  

The hostname is what you assigned to your computer when you configured your operating system. This output will be different on your machine depending on the system's host configuration. Here hostname indicates where the Python interpreter is currently executing.

Please note that the programs in this book are run with both versions 2 and 3. We avoid mentioning python3 and python2 in commands, as they are too specific to some distributions and assumes that a specific version is installed. You may run any of the programs in either version by using python2 or python3 accordingly.

How it works...

The import socket statement imports one of Python's core networking libraries. Then, we use the two utility functions, gethostname() and gethostbyname(host_name). You can type help(socket.gethostname) to see the online help information from within the command line. Alternatively, you can type the following address in your web browser at http://docs.python.org/3/library/socket.html. You can refer to the following code:

    gethostname(...)
        gethostname() -> string 
        Return the current host name. 
    
    gethostbyname(...) 
        gethostbyname(host) -> address 
        Return the IP address (a string of the form 
'255.255.255.255') for a host.

The first function takes no parameter and returns the current or localhost name. The second function takes a single hostname parameter and returns its IP address.

 

Retrieving a remote machine's IP address

Sometimes, you need to translate a machine's hostname into its corresponding IP address, for example, a quick domain name lookup. This recipe introduces a simple function to do that.

How to do it...

If you need to know the IP address of a remote machine, you can use a built-in library function, gethostbyname(). In this case, you need to pass the remote hostname as its parameter.

In this case, we need to call the gethostbyname() class function. Let's have a look at this short code snippet.

Listing 1.2 shows how to get a remote machine's IP address as follows:

    #!/usr/bin/env python
    # Python Network Programming Cookbook, Second Edition
-- Chapter - 1 # This program is optimized for Python 2.7.12 and
Python 3.5.2. # It may run on any other version with/without
modifications. import socket def get_remote_machine_info(): remote_host = 'www.python.org' try: print ("IP address of %s: %s" %(remote_host,
socket.gethostbyname(remote_host))) except socket.error as err_msg: print ("%s: %s" %(remote_host, err_msg)) if __name__ == '__main__': get_remote_machine_info()

If you run the preceding code it gives the following output:

$ python 1_2_remote_machine_info.py 
IP address of www.python.org: 151.101.36.223
  

How it works...

This recipe wraps the gethostbyname() method inside a user-defined function called get_remote_machine_info(). In this recipe, we introduced the notion of exception handling. As you can see, we wrapped the main function call inside a try-except block. This means that, if some error occurs during the execution of this function, this error will be dealt with by this try-except block.

For example, let's change the remote_host value and replace https://www.python.org/ with something non-existent, for example, www.pytgo.org:

#!/usr/bin/env python
# Python Network Programming Cookbook,
Second Edition -- Chapter - 1 # This program is optimized for Python 2.7.12 and
Python 3.5.2. # It may run on any other version with/without
modifications. import socket def get_remote_machine_info(): remote_host = 'www.pytgo.org' try: print ("IP address of %s: %s" %
(remote_host,
socket.gethostbyname(remote_host))) except socket.error as err_msg: print ("%s: %s" %(remote_host, err_msg)) if __name__ == '__main__': get_remote_machine_info()

Now run the following command:

$ python 1_2_remote_machine_info.py 
www.pytgo.org: [Errno -2] Name or service not known
  

The try-except block catches the error and shows the user an error message that there is no IP address associated with the hostname, www.pytgo.org.

 

Converting an IPv4 address to different formats

When you would like to deal with low-level network functions, sometimes, the usual string notation of IP addresses are not very useful. They need to be converted to the packed 32-bit binary formats.

How to do it...

The Python socket library has utilities to deal with the various IP address formats. Here, we will use two of them: inet_aton() and inet_ntoa().

Let us create the convert_ip4_address() function, where inet_aton() and inet_ntoa() will be used for the IP address conversion. We will use two sample IP addresses, 127.0.0.1 and 192.168.0.1.

Listing 1.3 shows ip4_address_conversion as follows:

#!/usr/bin/env python
# Python Network Programming Cookbook,
Second Edition -- Chapter - 1 # This program is optimized for Python 2.7.12 and
Python 3.5.2. # It may run on any other version with/without
modifications. import socket from binascii import hexlify def convert_ip4_address(): for ip_addr in ['127.0.0.1', '192.168.0.1']: packed_ip_addr = socket.
inet_aton(ip_addr) unpacked_ip_addr = socket.inet_ntoa
(packed_ip_addr) print ("IP Address: %s => Packed: %s,
Unpacked: %s" %(ip_addr,
hexlify(packed_ip_addr),
unpacked_ip_addr)) if __name__ == '__main__': convert_ip4_address()

Now, if you run this recipe, you will see the following output:

$ python 1_3_ip4_address_conversion.py 
IP Address: 127.0.0.1 => Packed: 7f000001, Unpacked: 
127.0.0.1
IP Address: 192.168.0.1 => Packed: c0a80001, Unpacked: 192.168.0.1

How it works...

In this recipe, the two IP addresses have been converted from a string to a 32-bit packed format using a for-in statement. Additionally, the Python hexlify function is called from the binascii module. This helps to represent the binary data in a hexadecimal format.

 

Finding a service name, given the port and protocol

If you would like to discover network services, it may be helpful to determine what network services run on which ports using either the TCP or UDP protocol.

Getting ready

If you know the port number of a network service, you can find the service name using the getservbyport() socket class function from the socket library. You can optionally give the protocol name when calling this function.

How to do it...

Let us define a find_service_name() function, where the getservbyport() socket class function will be called with a few ports, for example, 80, 25. We can use Python's for-in loop construct.

Listing 1.4 shows finding_service_name as follows:

#!/usr/bin/env python 
# Python Network Programming Cookbook, Second Edition -- Chapter - 1 
# This program is optimized for Python 2.7.12 and Python 3.5.2. 
# It may run on any other version with/without modifications. 
 
import socket 
 
def find_service_name(): 
    protocolname = 'tcp' 
    for port in [80, 25]: 
        print ("Port: %s => service name: %s" %(port, socket.getservbyport(port, protocolname))) 
     
    print ("Port: %s => service name: %s" %(53, socket.getservbyport(53, 'udp'))) 
     
if __name__ == '__main__': 
    find_service_name() 
 

If you run this script, you will see the following output:

$ python 1_4_finding_service_name.py 
    
Port: 80 => service name: http
Port: 25 => service name: smtp
Port: 53 => service name: domain
    

This indicates that http, smtp, and domain services are running on the ports 80, 25, and 53 respectively.

How it works...

In this recipe, the for-in statement is used to iterate over a sequence of variables. So for each iteration, we use one IP address to convert them in their packed and unpacked format.

 

Converting integers to and from host to network byte order

If you ever need to write a low-level network application, it may be necessary to handle the low-level data transmission over the wire between two machines. This operation requires some sort of conversion of data from the native host operating system to the network format and vice versa. This is because each one has its own specific representation of data.

How to do it...

Python's socket library has utilities for converting from a network byte order to host byte order and vice versa. You may want to become familiar with them, for example, ntohl()/htonl().

Let us define the convert_integer() function, where the ntohl()/htonl() socket class functions are used to convert IP address formats.

Listing 1.5 shows integer_conversion as follows:

#!/usr/bin/env python 
# Python Network Programming Cookbook, Second Edition -- Chapter - 1 
# This program is optimized for Python 2.7.12 and Python 3.5.2. 
# It may run on any other version with/without modifications. 
 
import socket 
 
def convert_integer(): 
    data = 1234 
    # 32-bit 
    print ("Original: %s => Long  host byte order: %s, Network byte order: %s" %(data, socket.ntohl(data), socket.htonl(data))) 
    # 16-bit 
    print ("Original: %s => Short  host byte order: %s, Network byte order: %s" %(data, socket.ntohs(data), socket.htons(data))) 
 
     
if __name__ == '__main__': 
    convert_integer() 
 

If you run this recipe, you will see the following output:

$ python 1_5_integer_conversion.py 
Original: 1234 => Long  host byte order: 3523477504, 
Network byte order: 3523477504
Original: 1234 => Short host byte order: 53764,
Network byte order: 53764

How it works...

Here, we take an integer and show how to convert it between network and host byte orders. The ntohl() socket class function converts from the network byte order to host byte order in a long format. Here, n represents network and h represents host; l represents long and s represents short, that is, 16-bit.

 

Setting and getting the default socket timeout

Sometimes, you need to manipulate the default values of certain properties of a socket library, for example, the socket timeout.

How to do it...

You can make an instance of a socket object and call a gettimeout() method to get the default timeout value and the settimeout() method to set a specific timeout value. This is very useful in developing custom server applications.

We first create a socket object inside a test_socket_timeout() function. Then, we can use the getter/setter instance methods to manipulate timeout values.

Listing 1.6 shows socket_timeout as follows:

#!/usr/bin/env python 
# Python Network Programming Cookbook, Second Edition -- Chapter - 1 
# This program is optimized for Python 2.7.12 and Python 3.5.2. 
# It may run on any other version with/without modifications. 
 
 
import socket 
 
def test_socket_timeout(): 
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 
    print ("Default socket timeout: %s" %s.gettimeout()) 
    s.settimeout(100) 
    print ("Current socket timeout: %s" %s.gettimeout())     
     
if __name__ == '__main__': 
    test_socket_timeout() 
 
 

After running the preceding script, you can see how this modifies the default socket timeout as follows:

$ python 1_6_socket_timeout.py 
Default socket timeout: None
Current socket timeout: 100.0
  

How it works...

In this code snippet, we have first created a socket object by passing the socket family and socket type as the first and second arguments of the socket constructor. Then, you can get the socket timeout value by calling gettimeout() and alter the value by calling the settimeout() method. The timeout value passed to the settimeout() method can be in seconds (non-negative float) or None. This method is used for manipulating the blocking-socket operations. Setting a timeout of None disables timeouts on socket operations.

 

Handling socket errors gracefully

In any networking application, it is very common that one end is trying to connect, but the other party is not responding due to networking media failure or any other reason. The Python socket library has an elegant method of handing these errors via the socket.error exceptions. In this recipe, a few examples are presented.

How to do it...

Let us create a few try-except code blocks and put one potential error type in each block. In order to get a user input, the argparse module can be used. This module is more powerful than simply parsing command-line arguments using sys.argv. In the try-except blocks, put typical socket operations, for example, create a socket object, connect to a server, send data, and wait for a reply.

The following recipe illustrates the concepts in a few lines of code.

Listing 1.7 shows socket_errors as follows:

#!/usr/bin/env python 
# Python Network Programming Cookbook, Second Edition -- Chapter - 1 
# This program is optimized for Python 2.7.12 and Python 3.5.2. 
# It may run on any other version with/without modifications. 
 
import sys 
import socket 
import argparse  
 
 
def main(): 
    # setup argument parsing 
    parser = argparse.ArgumentParser(description='Socket Error Examples') 
    parser.add_argument('--host', action="store", dest="host",
required=False) parser.add_argument('--port', action="store", dest="port", type=int,
required=False) parser.add_argument('--file', action="store", dest="file",
required=False) given_args = parser.parse_args() host = given_args.host port = given_args.port filename = given_args.file # First try-except block -- create socket try: s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) except socket.error as e: print ("Error creating socket: %s" % e) sys.exit(1) # Second try-except block -- connect to given host/port try: s.connect((host, port)) except socket.gaierror as e: print ("Address-related error connecting to
server: %s" % e) sys.exit(1) except socket.error as e: print ("Connection error: %s" % e) sys.exit(1) # Third try-except block -- sending data try: msg = "GET %s HTTP/1.0\r\n\r\n" % filename s.sendall(msg.encode('utf-8')) except socket.error as e: print ("Error sending data: %s" % e) sys.exit(1) while 1: # Fourth tr-except block -- waiting
to receive
data from remote host try: buf = s.recv(2048) except socket.error as e: print ("Error receiving data: %s" % e) sys.exit(1) if not len(buf): break # write the received data sys.stdout.write(buf.decode('utf-8')) if __name__ == '__main__': main()

How it works...

In Python, passing command-line arguments to a script and parsing them in the script can be done using the argparse module. This is available in Python 2.7. For earlier versions of Python, this module is available separately in Python Package Index (PyPI). You can install this via easy_install or pip.

In this recipe, three arguments are set up—a hostname, port number, and filename. The usage of this script is as follows:

$ python 1_7_socket_errors.py --host=<HOST> 
--port=<PORT> --file=<FILE>
In the preceding recipe, msg.encode('utf-8')
encodes the message into UTF-8, and
buf.decode('utf-8') decodes the received UTF-8
format.

If you try the preceding recipe with a non-existent host, this script will print an address error as follows:

$ python 1_7_socket_errors.py 
--host=www.pytgo.org --port=8080
--file=1_7_socket_errors.py
Address-related error connecting to
server: [Errno -2] Name or service not known

If there is no service on a specific port and if you try to connect to that port, then this will throw a connection timeout error as follows:

$ python 1_7_socket_errors.py 
--host=www.python.org --port=8080
--file=1_7_socket_errors.py

This will return the following error since the host, www.python.org, is not listening on port 8080:

Connection error: [Errno 110] Connection timed out
  

However, if you send an arbitrary request as a correct request to a correct port, the error may not be caught at the application level. For example, running the following script returns no error, but the HTML output tells us what's wrong with this script:

$ python 1_7_socket_errors.py 
--host=www.python.org --port=80
--file=1_7_socket_errors.py
HTTP/1.1 500 Domain Not Found Server: Varnish Retry-After: 0 content-type: text/html Cache-Control: private, no-cache connection: keep-alive Content-Length: 179 Accept-Ranges: bytes Date: Thu, 01 Jun 2017 22:02:24 GMT Via: 1.1 varnish Connection: close <html> <head> <title>Fastly error: unknown domain </title> </head> <body> Fastly error: unknown domain: . Please check that this domain has been added to a service.</body></html>

In the preceding example, four try-except blocks have been used. All blocks use socket.error except for the second block, which uses socket.gaierror. This is used for address-related errors. There are two other types of exceptions—socket.herror is used for legacy C API, and if you use the settimeout() method in a socket, socket.timeout will be raised when a timeout occurs on that socket.

 

Modifying a socket's send/receive buffer sizes

The default socket buffer size may not be suitable in many circumstances. In such circumstances, you can change the default socket buffer size to a more suitable value.

How to do it...

Let us manipulate the default socket buffer size using a socket object's setsockopt() method.

First, define two constants: SEND_BUF_SIZE/RECV_BUF_SIZE and then wrap a socket instance's call to the setsockopt() method in a function. It is also a good idea to check the value of the buffer size before modifying it. Note that we need to set up the send and receive buffer size separately.

Listing 1.8 shows how to modify socket send/receive buffer sizes as follows:

#!/usr/bin/env python 
# Python Network Programming Cookbook, Second Edition -- Chapter - 1 
# This program is optimized for Python 2.7.12 and Python 3.5.2. 
# It may run on any other version with/without modifications. 
 
import socket 
 
SEND_BUF_SIZE = 4096 
RECV_BUF_SIZE = 4096 
 
def modify_buff_size(): 
    sock = socket.socket( socket.AF_INET, socket.SOCK_STREAM ) 
     
    # Get the size of the socket's send buffer 
    bufsize = sock.getsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF) 
    print ("Buffer size [Before]:%d" %bufsize) 
     
    sock.setsockopt(socket.SOL_TCP, 
socket.TCP_NODELAY, 1) sock.setsockopt( socket.SOL_SOCKET, socket.SO_SNDBUF, SEND_BUF_SIZE) sock.setsockopt( socket.SOL_SOCKET, socket.SO_RCVBUF, RECV_BUF_SIZE) bufsize = sock.getsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF) print ("Buffer size [After]:%d" %bufsize) if __name__ == '__main__': modify_buff_size()

If you run the preceding script, it will show the changes in the socket's buffer size. The following output may be different on your machine depending on your operating system's local settings:

$ python 1_8_modify_buff_size.py 
Buffer size [Before]:16384
Buffer size [After]:8192  

How it works...

You can call the getsockopt() and setsockopt() methods on a socket object to retrieve and modify the socket object's properties respectively. The setsockopt() method takes three arguments: level, optname, and value. Here, optname takes the option name and value is the corresponding value of that option. For the first argument, the needed symbolic constants can be found in the socket module (SO_*etc.).

 

Changing a socket to the blocking/non-blocking mode

By default, TCP sockets are placed in a blocking mode. This means the control is not returned to your program until some specific operation is complete. If you call the connect() API, the connection blocks your program until the operation is complete. On many occasions, you don't want to keep your program waiting forever, either for a response from the server or for any error to stop the operation. For example, when you write a web browser client that connects to a web server, you should consider a stop functionality that can cancel the connection process in the middle of this operation. This can be achieved by placing the socket in the non-blocking mode.

How to do it...

Let us see what options are available under Python. In Python, a socket can be placed in the blocking or non-blocking mode. In the non-blocking mode, if any call to API, for example, send() or recv(), encounters any problem, an error will be raised. However, in the blocking mode, this will not stop the operation. We can create a normal TCP socket and experiment with both the blocking and non-blocking operations.

To manipulate the socket's blocking nature, we should create a socket object first.

We can then call setblocking(1) to set up blocking or setblocking(0) to unset blocking. Finally, we bind the socket to a specific port and listen for incoming connections.

Listing 1.9 shows how the socket changes to blocking or non-blocking mode as follows:

#!/usr/bin/env python 
# Python Network Programming Cookbook, Second Edition -- Chapter - 1 
# This program is optimized for Python 2.7.12 and Python 3.5.2. 
# It may run on any other version with/without modifications. 
 
import socket 
 
def test_socket_modes(): 
    s = socket.socket(socket.AF_INET, 
socket.SOCK_STREAM) s.setblocking(1) s.settimeout(0.5) s.bind(("127.0.0.1", 0)) socket_address = s.getsockname() print ("Trivial Server launched on
socket: %s" %str(socket_address)) while(1): s.listen(1) if __name__ == '__main__': test_socket_modes()

If you run this recipe, it will launch a trivial server that has the blocking mode enabled as shown in the following command:

$ python 1_9_socket_modes.py 
Trivial Server launched on 
socket: ('127.0.0.1', 51410)

How it works...

In this recipe, we enable blocking on a socket by setting the value 1 in the setblocking() method. Similarly, you can unset the value 0 in this method to make it non-blocking.

This feature will be reused in some later recipes, where its real purpose will be elaborated.

 

Reusing socket addresses

You want to run a socket server always on a specific port even after it is closed intentionally or unexpectedly. This is useful in some cases where your client program always connects to that specific server port. So, you don't need to change the server port.

How to do it...

If you run a Python socket server on a specific port and try to rerun it after closing it once, you won't be able to use the same port. It will usually throw an error like the following command:

Traceback (most recent call last):
   File "1_10_reuse_socket_address.py", 
line 40, in <module>
reuse_socket_addr() File "1_10_reuse_socket_address.py",
line 25, in reuse_socket_addr
srv.bind( ('', local_port) ) File "<string>", line 1, in bind socket.error: [Errno 98] Address
already in use

The remedy to this problem is to enable the socket reuse option, SO_REUSEADDR.

After creating a socket object, we can query the state of address reuse, say an old state. Then, we call the setsockopt() method to alter the value of its address reuse state. Then, we follow the usual steps of binding to an address and listening for incoming client connections.

In the preceding example, when you close the Python script with Ctrl + C, you notice an exception:

^CTraceback (most recent call last):File "1_9_socket_modes.py", line 20, in <module>
test_socket_modes()
File "1_9_socket_modes.py", line 17, in test_socket_modes
s.listen(1)
KeyboardInterrupt

This indicates that there was a keyboard interrupt in the execution.

In this example, we catch the KeyboardInterrupt exception so that if you issue Ctrl + C, then the Python script gets terminated without showing any exception message.

Listing 1.10 shows how to reuse socket addresses as follows:

#!/usr/bin/env python 
# Python Network Programming Cookbook, Second Edition -- Chapter - 1 
# This program is optimized for Python 2.7.12 and Python 3.5.2. 
# It may run on any other version with/without modifications. 
 
import socket 
import sys 
 
 
def reuse_socket_addr(): 
    sock = socket.socket( socket.AF_INET, socket.SOCK_STREAM ) 
 
    # Get the old state of the SO_REUSEADDR option 
    old_state = sock.getsockopt(socket.SOL_SOCKET, 
socket.SO_REUSEADDR ) print ("Old sock state: %s" %old_state) # Enable the SO_REUSEADDR option sock.setsockopt( socket.SOL_SOCKET,
socket.SO_REUSEADDR, 1 ) new_state = sock.getsockopt(
socket.SOL_SOCKET, socket.SO_REUSEADDR ) print ("New sock state: %s" %new_state) local_port = 8282 srv = socket.socket(socket.AF_INET,
socket.SOCK_STREAM) srv.setsockopt(socket.SOL_SOCKET,
socket.SO_REUSEADDR, 1) srv.bind( ('', local_port) ) srv.listen(1) print ("Listening on port: %s " %local_port) while True: try: connection, addr = srv.accept() print ('Connected by %s:%s'
% (addr[0], addr[1])) except KeyboardInterrupt: break except socket.error as msg: print ('%s' % (msg,)) if __name__ == '__main__': reuse_socket_addr()

The output from this recipe will be similar to the outcomes produced here, by executing the program:

$ python 1_10_reuse_socket_address.py 
Old sock state: 0
New sock state: 1
Listening on port: 8282 
  

How it works...

You may run this script from one console window and try to connect to this server from another console window by typing telnet localhost 8282.

You will see an output printed in the program window as your telnet connects to it:

Connected by 127.0.0.1:46584  

Here the host and port will defer based on the telnet instance that you are sending this request from.

After you close the server program, you can rerun it again on the same port. However, if you comment out the line that sets the SO_REUSEADDR, the server will not run for the second time.

 

Printing the current time from the internet time server

Many programs rely on the accurate machine time, such as the make command in UNIX. Your machine time may be different and need synchronizing with another time server in your network.

Getting ready

In order to synchronize your machine time with one of the internet time servers, you can write a Python client for that. For this, ntplib will be used. Here, the client/server conversation will be done using Network Time Protocol (NTP). If ntplib is not installed on your machine, you can get it from PyPI with the following command using pip or easy_install:

$ pip install ntplib
  

If pip is not installed on your computer, first install it before executing the preceding command. In Debian-based Linux distributions such as Ubuntu, this can be installed by:

$ sudo apt install python-pip
  

Note that you will need to install pip for Python 3 separately if you are running it along side Python 2, as typically Python 2 is set as the default version:

$ sudo apt-get install python3-pip
  

Similarly, ntplib needs to be installed for python3-pip (also called pip3) separately:

$ pip3 install ntplib  

It is a good idea to upgrade pip to the latest version if you are running an outdated version, by issuing the following command:

$ pip install --upgrade pip  

or:

$ pip3 install --upgrade pip  

If Python 2 and Python 3 are installed alongside in your computer then use pip3.

I am using the pip version 9.0.1, for both Python 2 and Python 3. This is the latest version at the time of writing.

How to do it...

We create an instance of NTPClient and then we call the request() method on it by passing the NTP server address.

Listing 1.11 shows how to print the current time from the internet time server as follows:

    #!/usr/bin/env python
    # Python Network Programming Cookbook, 
Second Edition -- Chapter - 1 # This program is optimized for Python 2.7.12
and Python 3.5.2. # It may run on any other version with/without
modifications. import ntplib from time import ctime def print_time(): ntp_client = ntplib.NTPClient() response = ntp_client.request('pool.ntp.org') print (ctime(response.tx_time)) if __name__ == '__main__': print_time()

In my machine, this recipe shows the following output:

$ python 1_11_print_machine_time.py 
Fri Jun  2 16:01:35 2017
  

How it works...

Here, an NTP client has been created and an NTP request has been sent to one of the internet NTP servers, pool.ntp.org. The ctime() function is used for printing the response.

 

Writing an SNTP client

Unlike the previous recipe, sometimes, you don't need to get the precise time from the NTP server. You can use a simpler version of NTP called simple network time protocol.

How to do it...

Let us create a plain SNTP client without using any third-party library.

Let us first define two constants: NTP_SERVER and TIME1970. NTP_SERVER is the server address to which our client will connect, and TIME1970 is the reference time on January 1, 1970 (also called Epoch). You may find the value of the Epoch time or convert to the Epoch time at http://www.epochconverter.com/. The actual client creates a UDP socket (SOCK_DGRAM) to connect to the server following the UDP protocol. The client then needs to send the SNTP protocol data ('\x1b' + 47 * '\0') in a packet. Our UDP client sends and receives data using the sendto() and recvfrom() methods.

When the server returns the time information in a packed array, the client needs a specialized struct module to unpack the data. The only interesting data is located in the 11th element of the array. Finally, we need to subtract the reference value, TIME1970, from the unpacked value to get the actual current time.

Listing 1.12 shows how to write an SNTP client as follows:

    #!/usr/bin/env python
    # Python Network Programming Cookbook, 
Second Edition -- Chapter - 1 # This program is optimized for Python 2.7.12
and Python 3.5.2. # It may run on any other version with/without
modifications. import socket import struct import sys import time NTP_SERVER = "0.uk.pool.ntp.org" TIME1970 = 2208988800 def sntp_client(): client = socket.socket( socket.AF_INET,
socket.SOCK_DGRAM ) data = '\x1b' + 47 * '\0' client.sendto( data.encode('utf-8'),
( NTP_SERVER, 123 )) data, address = client.recvfrom( 1024 ) if data: print ('Response received
from:', address) t = struct.unpack( '!12I', data )[10] t -= TIME1970 print ('\tTime=%s' % time.ctime(t)) if __name__ == '__main__': sntp_client()

This recipe prints the current time from the internet time server received with the SNTP protocol as follows:

$ python 1_12_sntp_client.py 
('Response received from:', 
('192.146.137.13', 123))
Time=Sat Jun 3 14:45:45 2017

How it works...

This SNTP client creates a socket connection and sends the protocol data. After receiving the response from the NTP server (in this case, 0.uk.pool.ntp.org), it unpacks the data with struct. Finally, it subtracts the reference time, which is January 1, 1970, and prints the time using the ctime() built-in method in the Python time module.

 

Writing a simple TCP echo client/server application

After testing with basic socket APIs in Python, let us create a TCP socket server and client now. Here, you will have the chance to utilize your basic knowledge gained in the previous recipes.

How to do it...

In this example, a server will echo whatever it receives from the client. We will use the Python argparse module to specify the TCP port from a command line. Both the server and client script will take this argument.

First, we create the server. We start by creating a TCP socket object. Then, we set the reuse address so that we can run the server as many times as we need. We bind the socket to the given port on our local machine. In the listening stage, we make sure we listen to multiple clients in a queue using the backlog argument to the listen() method. Finally, we wait for the client to be connected and send some data to the server. When the data is received, the server echoes back the data to the client.

Listing 1.13a shows how to write a simple TCP echo client/server application as follows:

    #!/usr/bin/env python
    # Python Network Programming Cookbook,
Second Edition -- Chapter - 1 # This program is optimized for Python 2.7.12
and Python 3.5.2. # It may run on any other version with/without
modifications. import socket import sys import argparse host = 'localhost' data_payload = 2048 backlog = 5 def echo_server(port): """ A simple echo server """ # Create a TCP socket sock = socket.socket(socket.AF_INET,
socket.SOCK_STREAM) # Enable reuse address/port sock.setsockopt(socket.SOL_SOCKET,
socket.SO_REUSEADDR, 1) # Bind the socket to the port server_address = (host, port) print ("Starting up echo server on %s
port %s" % server_address) sock.bind(server_address) # Listen to clients, backlog argument
specifies the max no.
of queued connections sock.listen(backlog) while True: print ("Waiting to receive message
from client") client, address = sock.accept() data = client.recv(data_payload) if data: print ("Data: %s" %data) client.send(data) print ("sent %s bytes back
to %s" % (data, address)) # end connection client.close() if __name__ == '__main__': parser = argparse.ArgumentParser
(description='Socket Server Example') parser.add_argument('--port',
action="store", dest="port", type=int,
required=True) given_args = parser.parse_args() port = given_args.port echo_server(port)

On the client side code, we create a client socket using the port argument and connect to the server. Then, the client sends the message, Test message. This will be echoed to the server, and the client immediately receives the message back in a few segments. Here, two try-except blocks are constructed to catch any exception during this interactive session.

Listing 1-13b shows the TCP echo client as follows:

#!/usr/bin/env python 
# Python Network Programming Cookbook,
Second Edition -- Chapter - 1 # This program is optimized for Python 2.7.12
and Python 3.5.2. # It may run on any other version with/without modifications. import socket import sys import argparse host = 'localhost' def echo_client(port): """ A simple echo client """ # Create a TCP/IP socket sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # Connect the socket to the server server_address = (host, port) print ("Connecting to %s port %s" % server_address) sock.connect(server_address) # Send data try: # Send data message = "Test message. This will be
echoed" print ("Sending %s" % message) sock.sendall(message.encode('utf-8')) # Look for the response amount_received = 0 amount_expected = len(message) while amount_received < amount_expected: data = sock.recv(16) amount_received += len(data) print ("Received: %s" % data) except socket.error as e: print ("Socket error: %s" %str(e)) except Exception as e: print ("Other exception: %s" %str(e)) finally: print ("Closing connection to the server") sock.close() if __name__ == '__main__': parser = argparse.ArgumentParser
(description='Socket Server Example') parser.add_argument('--port', action="store",
dest="port", type=int, required=True) given_args = parser.parse_args() port = given_args.port echo_client(port)

How it works...

In order to see the client/server interactions, launch the following server script in one console:

$ python 1_13a_echo_server.py --port=9900 
Starting up echo server  on localhost port 9900 
    
Waiting to receive message from client 
  

Now, run the client from another Terminal as follows:

$ python 1_13b_echo_client.py --port=9900 
Connecting to localhost port 9900 
Sending Test message. This will be echoed 
Received: Test message. Th 
Received: is will be echoe 
Received: d 
Closing connection to the server
  

Upon receiving the message from the client, the server will also print something similar to the following message:

Data: Test message. This will be echoed 
sent Test message. This will be echoed 
bytes back to ('127.0.0.1', 42961)
Waiting to receive message from client
 

Writing a simple UDP echo client/server application

As we have developed a simple TCP server and client in the previous recipe, we will now look at how to develop the same with UDP.

How to do it...

This recipe is similar to the previous one, except this one is with UDP. The method recvfrom() reads the messages from the socket and returns the data and the client address.

Listing 1.14a shows how to write a simple UDP echo client/server application as follows:

#!/usr/bin/env python 
# Python Network Programming Cookbook,
Second Edition -- Chapter - 1 # This program is optimized for Python 2.7.12
and Python 3.5.2. # It may run on any other version with/without
modifications. import socket import sys import argparse host = 'localhost' data_payload = 2048 def echo_server(port): """ A simple echo server """ # Create a UDP socket sock = socket.socket(socket.AF_INET,
socket.SOCK_DGRAM) # Bind the socket to the port server_address = (host, port) print ("Starting up echo server
on %s port %s" % server_address) sock.bind(server_address) while True: print ("Waiting to receive message
from client") data, address = sock.
recvfrom(data_payload) print ("received %s bytes
from %s" % (len(data), address)) print ("Data: %s" %data) if data: sent = sock.sendto(data, address) print ("sent %s bytes back
to %s" % (sent, address)) if __name__ == '__main__': parser = argparse.ArgumentParser
(description='Socket Server Example') parser.add_argument('--port', action="store", dest="port", type=int, required=True) given_args = parser.parse_args() port = given_args.port echo_server(port)

On the client side code, we create a client socket using the port argument and connect to the server, as we did in the previous recipe. Then, the client sends the message, Test message. This will be echoed, and the client immediately receives the message back in a few segments.

Listing 1-14b shows the echo client as follows:

#!/usr/bin/env python 
# Python Network Programming Cookbook, Second Edition -- Chapter - 1 
# This program is optimized for Python 2.7.12 and Python 3.5.2. 
# It may run on any other version with/without modifications. 
 
import socket 
import sys 
import argparse 
 
host = 'localhost' 
data_payload = 2048 
 
def echo_client(port): 
    """ A simple echo client """ 
    # Create a UDP socket 
    sock = socket.socket(socket.AF_INET, 
socket.SOCK_DGRAM) server_address = (host, port) print ("Connecting to %s port %s" % server_address) message = 'This is the message. It will be
repeated.' try: # Send data message = "Test message. This will be
echoed" print ("Sending %s" % message) sent = sock.sendto(message.encode
('utf-8'), server_address) # Receive response data, server = sock.recvfrom(data_payload) print ("received %s" % data) finally: print ("Closing connection to the server") sock.close() if __name__ == '__main__': parser = argparse.ArgumentParser
(description='Socket Server Example') parser.add_argument('--port', action="store", dest="port", type=int, required=True) given_args = parser.parse_args() port = given_args.port echo_client(port)
Downloading the example code
Detailed steps to download the code bundle are mentioned in the Preface of this book. The code bundle for the book is also hosted on GitHub at: https://github.com/PacktPublishing/Python-Network-Programming-Cookbook-Second-Edition. We also have other code bundles from our rich catalog of books and videos available at: https://github.com/PacktPublishing/. Check them out!

How it works...

In order to see the client/server interactions, launch the following server script in one console:

$ python 1_14a_echo_server_udp.py --port=9900 
Starting up echo server on localhost port 9900
Waiting to receive message from client
  

Now, run the client from another terminal as follows:

$ python 1_14b_echo_client_udp.py --port=9900 
Connecting to localhost port 9900
Sending Test message. This will be echoed
received Test message. This will be echoed
Closing connection to the server
    
  

Upon receiving the message from the client, the server will also print something similar to the following message:

received 33 bytes from ('127.0.0.1', 43542)
Data: Test message. This will be echoed
sent 33 bytes back to ('127.0.0.1', 43542)
Waiting to receive message from client
  
About the Authors
  • Pradeeban Kathiravelu

    Pradeeban Kathiravelu is an open source evangelist. He is a Ph.D. researcher at INESC-ID Lisboa/Instituto Superior Tecnico, Universidade de Lisboa, Portugal, and Universite Catholique de Louvain, Belgium. He is a Fellow of Erasmus Mundus Joint Degree in Distributed Computing (EMJD-DC), researching a software-defined approach to quality of service and data quality in multi-tenant clouds. Pradeeban holds a master of science degree, Erasmus Mundus European Master in Distributed Computing (EMDC), from Instituto Superior Tecnico, Portugal and KTH Royal Institute of Technology, Sweden. He also holds a first class bachelor of science in engineering (Hons) degree, majoring in computer science and engineering, from the University of Moratuwa, Sri Lanka. His research interests include Software-Defined Networking (SDN), distributed systems, cloud computing, web services, big data in biomedical informatics, Network Functions Virtualizations (NFV), and data mining. He is very interested in free and open source software development and has been an active participant in the Google Summer of Code (GSoC) program since 2009, as a student and as a mentor. Pradeeban has published several conference papers and co-authored a few book chapters. He has also worked on OpenDaylight Cookbook and Learning OpenDaylight as a technical reviewer. Python Network Programming Cookbook, Second Edition (2017) is his first book as an author, and he is quite excited about it.

    Browse publications by this author
  • Gary Berger
  • Dr. M. O. Faruque Sarker

    Dr. M. O. Faruque Sarker is a software architect based in London; he has shaped various Linux and open source software solutions mainly on cloud computing platforms for various institutions. Over the past 10 years, he has led numerous Python software development and cloud infrastructure automation projects. In 2009, he started using Python and shepherded a fleet of miniature E-puck robots at the University of South Wales, Newport, UK. Later, he was invited to work on the Google Summer of Code (2009/2010) programs to contribute to the BlueZ and Tahoe-LAFS open source projects. He is the author of Python Network Programming Cookbook, Packt Publishing and received his PhD in multirobot systems at the University of South Wales.

    Browse publications by this author
Latest Reviews (6 reviews total)
awesome more detail would be better
Very informative and easy to follow examples even for a beginner who wants to try something new.
Wonderful book. I'm learning a lot from this book.
Python Network Programming Cookbook - Second Edition
Unlock this book and the full library FREE for 7 days
Start now