Home Networking-and-servers Python Network Programming Cookbook

Python Network Programming Cookbook

By Dr. M. O. Faruque Sarker
books-svg-icon Book
Subscription
$10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
BUY NOW $10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
Subscription
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
  1. Free Chapter
    Sockets, IPv4, and Simple Client/Server Programming
About this book

Python is an excellent language to use to write code and have fun by prototyping applications quickly. The presence of lots of third-party libraries, also known as batteries, makes it even more easier and faster to prototype an application or to implement a new algorithm. If you are interested in creating the building blocks for many practical web and networking applications that rely on networking protocols then this book is a must-have.

This book highlights major aspects of network programming in Python starting from writing simple networking clients, to developing complex screen-scraping and network security monitoring scripts. It creates the building blocks for many practical web and networking applications that rely on various networking protocols. This book presents the power and beauty of Python in solving the numerous real-world tasks in the area of network programming, system and network administration, network monitoring, and web-application development.

This book develops your ability to solve a wide range of network programming tasks in Python. We will start by exploring the Python standard library functions to create client/server network and manipulate your local networking resources available under both IPv4 and IPv6. The practical focus continues with creating web and email clients, scraping web pages, fetching information from various websites, and searching for information on the Web such as Amazon, Flickr, and other sites. It further develops your skills to analyze your network security vulnerabilities using advanced network packet capture and analysis techniques.

Publication date:
March 2014
Publisher
Packt
Pages
234
ISBN
9781849513463

 

Chapter 1. Sockets, IPv4, and Simple Client/Server Programming

In this chapter, we will cover the following recipes:

  • Printing your machine's name and IPv4 address

  • Retrieving a remote machine's IP address

  • Converting an IPv4 address to different formats

  • Finding a service name, given the port and protocol

  • Converting integers to and from host to network byte order

  • Setting and getting the default socket timeout

  • Handling socket errors gracefully

  • Modifying a socket's send/receive buffer size

  • Changing a socket to the blocking/non-blocking mode

  • Reusing socket addresses

  • Printing the current time from the Internet time server

  • Writing a SNTP client

  • Writing a simple echo client/server application

 

Introduction


This chapter introduces Python's core networking library through some simple recipes. Python's socket module has both class-based and instances-based utilities. The difference between a class-based and instance-based method is that the former doesn't need an instance of a socket object. This is a very intuitive approach. For example, in order to print your machine's IP address, you don't need a socket object. Instead, you can just call the socket's class-based methods. On the other hand, if you need to send some data to a server application, it is more intuitive that you create a socket object to perform that explicit operation. The recipes presented in this chapter can be categorized into three groups as follows:

  • In the first few recipes, the class-based utilities have been used in order to extract some useful information about host, network, and any target service.

  • After that, some more recipes have been presented using the instance-based utilities. Some common socket tasks, including manipulating the socket timeout, buffer size, blocking mode, and so on, have been demonstrated.

  • Finally, both class-based and instance-based utilities have been used to construct some clients, which perform some practical tasks, for example, synchronizing the machine time with an Internet server or writing a generic client/server script.

You can use these demonstrated approaches to write your own client/server application.

 

Printing your machine's name and IPv4 address


Sometimes, you need to quickly discover some information about your machine, for example, the host name, IP address, number of network interfaces, and so on. This is very easy to achieve using Python scripts.

Getting ready

You need to install Python on your machine before you start coding. Python comes preinstalled in most of the Linux distributions. For Microsoft Windows operating system, you can download binaries from the Python website: http://www.python.org/download/

You may consult the documentation of your OS to check and review your Python setup. After installing Python on your machine, you can try opening the Python interpreter from the command line by typing python. This will show the interpreter prompt, >>>, which should be similar to the following output:

~$ python 
Python 2.7.1+ (r271:86832, Apr 11 2011, 18:05:24) 
[GCC 4.5.2] on linux2 
Type "help", "copyright", "credits" or "license" for more information. >>> 

How to do it...

As this recipe is very short, you can try this in the Python interpreter interactively.

First, we need to import the Python socket library with the following command:

>>> import socket

Then, we call the gethostname() method from the socket library and store the result in a variable as follows:

>>> host_name = socket.gethostname()
>>> print "Host name: %s" %host_name
Host name: debian6
>>> print "IP address: %s" %socket.gethostbyname(host_name)
IP address: 127.0.1.1

The entire activity can be wrapped in a free-standing function, print_machine_info(), which uses the built-in socket class methods.

We call our function from the usual Python __main__ block. During runtime, Python assigns values to some internal variables such as __name__. In this case, __name__ refers to the name of the calling process. When running this script from the command line, as shown in the following command, the name will be __main__, but it will be different if the module is imported from another script. This means that when the module is called from the command line, it will automatically run our print_machine_info function; however, when imported separately, the user will need to explicitly call the function.

Listing 1.1 shows how to get our machine info, as follows:

#!/usr/bin/env python
# Python Network Programming Cookbook -- Chapter -1 
# This program is optimized for Python 2.7. It may run on any
# other Python version with/without modifications.

import socket

def print_machine_info():
    host_name = socket.gethostname()
    ip_address = socket.gethostbyname(host_name)
    print "Host name: %s" % host_name
    print "IP address: %s" % ip_address

if __name__ == '__main__':
    print_machine_info()

In order to run this recipe, you can use the provided source file from the command line as follows:

$ python 1_1_local_machine_info.py

On my machine, the following output is shown:

Host name: debian6
IP address: 127.0.0.1

This output will be different on your machine depending on the system's host configuration.

How it works...

The import socket statement imports one of Python's core networking libraries. Then, we use the two utility functions, gethostname() and gethostbyname(host_name). You can type help(socket.gethostname) to see the online help information from within the command line. Alternately, you can type the following address in your web browser at http://docs.python.org/3/library/socket.html. You can refer to the following command:

gethostname(...)
    gethostname() -> string 
    Return the current host name. 

gethostbyname(...) 
   gethostbyname(host) -> address 
    Return the IP address (a string of the form '255.255.255.255') for a host.

The first function takes no parameter and returns the current or localhost name. The second function takes a single hostname parameter and returns its IP address.

 

Retrieving a remote machine's IP address


Sometimes, you need to translate a machine's hostname into its corresponding IP address, for example, a quick domain name lookup. This recipe introduces a simple function to do that.

How to do it...

If you need to know the IP address of a remote machine, you can use a built-in library function, gethostbyname(). In this case, you need to pass the remote hostname as its parameter.

In this case, we need to call the gethostbyname() class function. Let's have a look inside this short code snippet.

Listing 1.2 shows how to get a remote machine's IP address as follows:

#!/usr/bin/env python
# Python Network Programming Cookbook -- Chapter – 1
# This program is optimized for Python 2.7.
# It may run on any other version with/without modifications.

import socket

def get_remote_machine_info():
    remote_host = 'www.python.org'
    try:
        print "IP address: %s" %socket.gethostbyname(remote_host)
    except socket.error, err_msg:
        print "%s: %s" %(remote_host, err_msg)
    
if __name__ == '__main__':
    get_remote_machine_info()

If you run the preceding code it gives the following output:

$ python 1_2_remote_machine_info.py 
IP address of www.python.org: 82.94.164.162

How it works...

This recipe wraps the gethostbyname() method inside a user-defined function called get_remote_machine_info(). In this recipe, we introduced the notion of exception handling. As you can see, we wrapped the main function call inside a try-except block. This means that if some error occurs during the execution of this function, this error will be dealt with by this try-except block.

For example, let's change the remote_host value and replace www.python.org with something non-existent, for example, www.pytgo.org. Now run the following command:

$ python 1_2_remote_machine_info.py 
www.pytgo.org: [Errno -5] No address associated with hostname

The try-except block catches the error and shows the user an error message that there is no IP address associated with the hostname, www.pytgo.org.

 

Converting an IPv4 address to different formats


When you would like to deal with low-level network functions, sometimes, the usual string notation of IP addresses are not very useful. They need to be converted to the packed 32-bit binary formats.

How to do it...

The Python socket library has utilities to deal with the various IP address formats. Here, we will use two of them: inet_aton() and inet_ntoa().

Let us create the convert_ip4_address() function, where inet_aton() and inet_ntoa() will be used for the IP address conversion. We will use two sample IP addresses, 127.0.0.1 and 192.168.0.1.

Listing 1.3 shows ip4_address_conversion as follows:

#!/usr/bin/env python
# Python Network Programming Cookbook -- Chapter – 1
# This program is optimized for Python 2.7.
# It may run on any other version with/without modifications.

import socket
from binascii import hexlify

def convert_ip4_address():
    for ip_addr in ['127.0.0.1', '192.168.0.1']:
        packed_ip_addr = socket.inet_aton(ip_addr)
        unpacked_ip_addr = socket.inet_ntoa(packed_ip_addr)
        print "IP Address: %s => Packed: %s, Unpacked: %s"\
	 %(ip_addr, hexlify(packed_ip_addr), unpacked_ip_addr)
    
if __name__ == '__main__':
    convert_ip4_address()

Now, if you run this recipe, you will see the following output:

$ python 1_3_ip4_address_conversion.py 

IP Address: 127.0.0.1 => Packed: 7f000001, Unpacked: 127.0.0.1
IP Address: 192.168.0.1 => Packed: c0a80001, Unpacked: 192.168.0.1

How it works...

In this recipe, the two IP addresses have been converted from a string to a 32-bit packed format using a for-in statement. Additionally, the Python hexlify function is called from the binascii module. This helps to represent the binary data in a hexadecimal format.

 

Finding a service name, given the port and protocol


If you would like to discover network services, it may be helpful to determine what network services run on which ports using either the TCP or UDP protocol.

Getting ready

If you know the port number of a network service, you can find the service name using the getservbyport() socket class function from the socket library. You can optionally give the protocol name when calling this function.

How to do it...

Let us define a find_service_name() function, where the getservbyport() socket class function will be called with a few ports, for example, 80, 25. We can use Python's for-in loop construct.

Listing 1.4 shows finding_service_name as follows:

#!/usr/bin/env python
# Python Network Programming Cookbook -- Chapter -  1
# This program is optimized for Python 2.7.
# It may run on any other version with/without modifications.

import socket

def find_service_name():
    protocolname = 'tcp'
    for port in [80, 25]:
        print "Port: %s => service name: %s" %(port, socket.getservbyport(port, protocolname))
    print "Port: %s => service name: %s" %(53, socket.getservbyport(53, 'udp'))
    
if __name__ == '__main__':
    find_service_name()

If you run this script, you will see the following output:

$ python 1_4_finding_service_name.py 

Port: 80 => service name: http
Port: 25 => service name: smtp
Port: 53 => service name: domain

How it works...

In this recipe, for-in statement is used to iterate over a sequence of variables. So for each iteration we use one IP address to convert them in their packed and unpacked format.

 

Converting integers to and from host to network byte order


If you ever need to write a low-level network application, it may be necessary to handle the low-level data transmission over the wire between two machines. This operation requires some sort of conversion of data from the native host operating system to the network format and vice versa. This is because each one has its own specific representation of data.

How to do it...

Python's socket library has utilities for converting from a network byte order to host byte order and vice versa. You may want to become familiar with them, for example, ntohl()/htonl().

Let us define the convert_integer() function, where the ntohl()/htonl() socket class functions are used to convert IP address formats.

Listing 1.5 shows integer_conversion as follows:

#!/usr/bin/env python
# Python Network Programming Cookbook -- Chapter - 
# This program is optimized for Python 2.7.
# It may run on any other version with/without modifications.
import socket
def convert_integer():
    data = 1234
    # 32-bit
    print "Original: %s => Long  host byte order: %s, Network byte order: %s"\
    %(data, socket.ntohl(data), socket.htonl(data))
    # 16-bit
    print "Original: %s => Short  host byte order: %s, Network byte order: %s"\
    %(data, socket.ntohs(data), socket.htons(data))
if __name__ == '__main__':
    convert_integer()

If you run this recipe, you will see the following output:

$ python 1_5_integer_conversion.py 
Original: 1234 => Long  host byte order: 3523477504, Network byte order: 3523477504
Original: 1234 => Short  host byte order: 53764, Network byte order: 53764

How it works...

Here, we take an integer and show how to convert it between network and host byte orders. The ntohl() socket class function converts from the network byte order to host byte order in a long format. Here, n represents network and h represents host; l represents long and s represents short, that is 16-bit.

 

Setting and getting the default socket timeout


Sometimes, you need to manipulate the default values of certain properties of a socket library, for example, the socket timeout.

How to do it...

You can make an instance of a socket object and call a gettimeout() method to get the default timeout value and the settimeout() method to set a specific timeout value. This is very useful in developing custom server applications.

We first create a socket object inside a test_socket_timeout() function. Then, we can use the getter/setter instance methods to manipulate timeout values.

Listing 1.6 shows socket_timeout as follows:

#!/usr/bin/env python
# Python Network Programming Cookbook -- Chapter - 1
# This program is optimized for Python 2.7. It may run on any   
# other Python version with/without modifications

import socket

def test_socket_timeout():
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    print "Default socket timeout: %s" %s.gettimeout()
    s.settimeout(100)
    print "Current socket timeout: %s" %s.gettimeout()    
    
if __name__ == '__main__':
    test_socket_timeout()

After running the preceding script, you can see how this modifies the default socket timeout as follows:

$ python 1_6_socket_timeout.py 
Default socket timeout: None
Current socket timeout: 100.0

How it works...

In this code snippet, we have first created a socket object by passing the socket family and socket type as the first and second arguments of the socket constructor. Then, you can get the socket timeout value by calling gettimeout() and alter the value by calling the settimeout() method. The timeout value passed to the settimeout() method can be in seconds (non-negative float) or None. This method is used for manipulating the blocking-socket operations. Setting a timeout of None disables timeouts on socket operations.

 

Handling socket errors gracefully


In any networking application, it is very common that one end is trying to connect, but the other party is not responding due to networking media failure or any other reason. The Python socket library has an elegant method of handing these errors via the socket.error exceptions. In this recipe, a few examples are presented.

How to do it...

Let us create a few try-except code blocks and put one potential error type in each block. In order to get a user input, the argparse module can be used. This module is more powerful than simply parsing command-line arguments using sys.argv. In the try-except blocks, put typical socket operations, for example, create a socket object, connect to a server, send data, and wait for a reply.

The following recipe illustrates the concepts in a few lines of code.

Listing 1.7 shows socket_errors as follows:

#!/usr/bin/env python
# Python Network Programming Cookbook -- Chapter – 1
# This program is optimized for Python 2.7. It may run on any   
# other Python version with/without modifications.

import sys
import socket
import argparse 


def main():
    # setup argument parsing
    parser = argparse.ArgumentParser(description='Socket Error Examples')
    parser.add_argument('--host', action="store", dest="host", required=False)
    parser.add_argument('--port', action="store", dest="port", type=int, required=False)
    parser.add_argument('--file', action="store", dest="file", required=False)
    given_args = parser.parse_args()
    host = given_args.host
    port = given_args.port
    filename = given_args.file
    
    # First try-except block -- create socket 
    try:
        s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    except socket.error, e:
        print "Error creating socket: %s" % e
        sys.exit(1)
    
    # Second try-except block -- connect to given host/port
    try:
        s.connect((host, port))
    except socket.gaierror, e:
        print "Address-related error connecting to server: %s" % e
        sys.exit(1)
    except socket.error, e:
        print "Connection error: %s" % e
        sys.exit(1)
    
    # Third try-except block -- sending data
    try:
        s.sendall("GET %s HTTP/1.0\r\n\r\n" % filename)
    except socket.error, e:
        print "Error sending data: %s" % e
        sys.exit(1)
    
    while 1:
        # Fourth tr-except block -- waiting to receive data from remote host
        try:
            buf = s.recv(2048)
        except socket.error, e:
            print "Error receiving data: %s" % e
            sys.exit(1)
        if not len(buf):
            break
        # write the received data
        sys.stdout.write(buf) 
    
if __name__ == '__main__':
    main()

How it works...

In Python, passing command-line arguments to a script and parsing them in the script can be done using the argparse module. This is available in Python 2.7. For earlier versions of Python, this module is available separately in Python Package Index (PyPI). You can install this via easy_install or pip.

In this recipe, three arguments are set up: a hostname, port number, and filename. The usage of this script is as follows:

$ python 1_7_socket_errors.py –host=<HOST> --port=<PORT> --file=<FILE>

If you try with a non-existent host, this script will print an address error as follows:

$ python 1_7_socket_errors.py --host=www.pytgo.org --port=8080 --file=1_7_socket_errors.py 
Address-related error connecting to server: [Errno -5] No address associated with hostname

If there is no service on a specific port and if you try to connect to that port, then this will throw a connection timeout error as follows:

$ python 1_7_socket_errors.py --host=www.python.org --port=8080 --file=1_7_socket_errors.py 

This will return the following error since the host, www.python.org, is not listening on port 8080:

Connection error: [Errno 110] Connection timed out

However, if you send an arbitrary request to a correct request to a correct port, the error may not be caught in the application level. For example, running the following script returns no error, but the HTML output tells us what's wrong with this script:

$ python 1_7_socket_errors.py --host=www.python.org --port=80 --file=1_7_socket_errors.py

HTTP/1.1 404 Not found
Server: Varnish
Retry-After: 0
content-type: text/html
Content-Length: 77
Accept-Ranges: bytes
Date: Thu, 20 Feb 2014 12:14:01 GMT
Via: 1.1 varnish
Age: 0
Connection: close

<html>
<head>
<title> </title>
</head>
<body>
unknown domain: </body></html>

In the preceding example, four try-except blocks have been used. All blocks use socket.error except the second block, which uses socket.gaierror. This is used for address-related errors. There are two other types of exceptions: socket.herror is used for legacy C API, and if you use the settimeout() method in a socket, socket.timeout will be raised when a timeout occurs on that socket.

 

Modifying socket's send/receive buffer sizes


The default socket buffer size may not be suitable in many circumstances. In such circumstances, you can change the default socket buffer size to a more suitable value.

How to do it...

Let us manipulate the default socket buffer size using a socket object's setsockopt() method.

First, define two constants: SEND_BUF_SIZE/RECV_BUF_SIZE and then wrap a socket instance's call to the setsockopt() method in a function. It is also a good idea to check the value of the buffer size before modifying it. Note that we need to set up the send and receive buffer size separately.

Listing 1.8 shows how to modify socket send/receive buffer sizes as follows:

#!/usr/bin/env python
# Python Network Programming Cookbook -- Chapter – 1
# This program is optimized for Python 2.7. It may run on any
# other Python version with/without modifications

import socket

SEND_BUF_SIZE = 4096
RECV_BUF_SIZE = 4096

def modify_buff_size():
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM )
    
    # Get the size of the socket's send buffer
    bufsize = sock.getsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF)
    print "Buffer size [Before]:%d" %bufsize
    
    sock.setsockopt(socket.SOL_TCP, socket.TCP_NODELAY, 1)
    sock.setsockopt(
            socket.SOL_SOCKET,
            socket.SO_SNDBUF,
            SEND_BUF_SIZE)
    sock.setsockopt(
            socket.SOL_SOCKET,
            socket.SO_RCVBUF,
            RECV_BUF_SIZE)
    bufsize = sock.getsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF)
    print "Buffer size [After]:%d" %bufsize

if __name__ == '__main__':
    modify_buff_size()

If you run the preceding script, it will show the changes in the socket's buffer size. The following output may be different on your machine depending on your operating system's local settings:

$ python 1_8_modify_buff_size.py 
Buffer size [Before]:16384
Buffer size [After]:8192

How it works...

You can call the getsockopt() and setsockopt() methods on a socket object to retrieve and modify the socket object's properties respectively. The setsockopt() method takes three arguments: level, optname, and value. Here, optname takes the option name and value is the corresponding value of that option. For the first argument, the needed symbolic constants can be found in the socket module (SO_*etc.).

 

Changing a socket to the blocking/non-blocking mode


By default, TCP sockets are placed in a blocking mode. This means the control is not returned to your program until some specific operation is complete. For example, if you call the connect() API, the connection blocks your program until the operation is complete. On many occasions, you don't want to keep your program waiting forever, either for a response from the server or for any error to stop the operation. For example, when you write a web browser client that connects to a web server, you should consider a stop functionality that can cancel the connection process in the middle of this operation. This can be achieved by placing the socket in the non-blocking mode.

How to do it...

Let us see what options are available under Python. In Python, a socket can be placed in the blocking or non-blocking mode. In the non-blocking mode, if any call to API, for example, send() or recv(), encounters any problem, an error will be raised. However, in the blocking mode, this will not stop the operation. We can create a normal TCP socket and experiment with both the blocking and non-blocking operations.

In order to manipulate the socket's blocking nature, we need to create a socket object first.

We can then call setblocking(1) to set up blocking or setblocking(0) to unset blocking. Finally, we bind the socket to a specific port and listen for incoming connections.

Listing 1.9 shows how the socket changes to blocking or non-blocking mode as follows:

#!/usr/bin/env python
# Python Network Programming Cookbook -- Chapter - 1
# This program is optimized for Python 2.7. It may run on any
# other Python version with/without modifications

import socket

def test_socket_modes():
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.setblocking(1)
    s.settimeout(0.5)
    s.bind(("127.0.0.1", 0))
    
    socket_address = s.getsockname()
    print "Trivial Server launched on socket: %s" %str(socket_address)
    while(1):
        s.listen(1)

if __name__ == '__main__':
    test_socket_modes()

If you run this recipe, it will launch a trivial server that has the blocking mode enabled as shown in the following command:

$ python 1_9_socket_modes.py 
Trivial Server launched on socket: ('127.0.0.1', 51410)

How it works...

In this recipe, we enable blocking on a socket by setting the value 1 in the setblocking() method. Similarly, you can unset the value 0 in this method to make it non-blocking.

This feature will be reused in some later recipes, where its real purpose will be elaborated.

 

Reusing socket addresses


You want to run a socket server always on a specific port even after it is closed intentionally or unexpectedly. This is useful in some cases where your client program always connects to that specific server port. So, you don't need to change the server port.

How to do it...

If you run a Python socket server on a specific port and try to rerun it after closing it once, you won't be able to use the same port. It will usually throw an error like the following command:

Traceback (most recent call last):
  File "1_10_reuse_socket_address.py", line 40, in <module>
    reuse_socket_addr()
  File "1_10_reuse_socket_address.py", line 25, in reuse_socket_addr
    srv.bind( ('', local_port) )
  File "<string>", line 1, in bind
socket.error: [Errno 98] Address already in use

The remedy to this problem is to enable the socket reuse option, SO_REUSEADDR.

After creating a socket object, we can query the state of address reuse, say an old state. Then, we call the setsockopt() method to alter the value of its address reuse state. Then, we follow the usual steps of binding to an address and listening for incoming client connections. In this example, we catch the KeyboardInterrupt exception so that if you issue Ctrl + C, then the Python script gets terminated without showing any exception message.

Listing 1.10 shows how to reuse socket addresses as follows:

#!/usr/bin/env python
# Python Network Programming Cookbook -- Chapter - 1
# This program is optimized for Python 2.7. It may run on any
# other Python version with/without modifications

import socket
import sys

def reuse_socket_addr():
    sock = socket.socket( socket.AF_INET, socket.SOCK_STREAM )

    # Get the old state of the SO_REUSEADDR option
    old_state = sock.getsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR )
    print "Old sock state: %s" %old_state

    # Enable the SO_REUSEADDR option
    sock.setsockopt( socket.SOL_SOCKET, socket.SO_REUSEADDR, 1 )
    new_state = sock.getsockopt( socket.SOL_SOCKET, socket.SO_REUSEADDR )
    print "New sock state: %s" %new_state

    local_port = 8282
    
    srv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    srv.bind( ('', local_port) )
    srv.listen(1)
    print ("Listening on port: %s " %local_port)
    while True:
        try:
            connection, addr = srv.accept()
            print 'Connected by %s:%s' % (addr[0], addr[1])
        except KeyboardInterrupt:
            break
        except socket.error, msg:
            print '%s' % (msg,)

if __name__ == '__main__':
    reuse_socket_e addr()

The output from this recipe will be similar to the following command:

$ python 1_10_reuse_socket_address.py 
Old sock state: 0
New sock state: 1
Listening on port: 8282 

How it works...

You may run this script from one console window and try to connect to this server from another console window by typing telnet localhost 8282. After you close the server program, you can rerun it again on the same port. However, if you comment out the line that sets the SO_REUSEADDR, the server will not run for the second time.

 

Printing the current time from the Internet time server


Many programs rely on the accurate machine time, such as the make command in UNIX. Your machine time may be different and need synchronizing with another time server in your network.

Getting ready

In order to synchronize your machine time with one of the Internet time servers, you can write a Python client for that. For this, ntplib will be used. Here, the client/server conversation will be done using Network Time Protocol (NTP). If ntplib is not installed on your machine, you can get it from PyPI with the following command using pip or easy_install:

$ pip install ntplib

How to do it...

We create an instance of NTPClient and then we call the request() method on it by passing the NTP server address.

Listing 1.11shows how to print the current time from the Internet time server is as follows:

 #!/usr/bin/env python
# Python Network Programming Cookbook -- Chapter - 1
# This program is optimized for Python 2.7. It may run on any
# other Python version with/without modifications

import ntplib
from time import ctime

def print_time():
    ntp_client = ntplib.NTPClient()
    response = ntp_client.request('pool.ntp.org')
    print ctime(response.tx_time)

if __name__ == '__main__':
    print_time()

In my machine, this recipe shows the following output:

$ python 1_11_print_machine_time.py 
Thu Mar 5 14:02:58 2012

How it works...

Here, an NTP client has been created and an NTP request has been sent to one of the Internet NTP servers, pool.ntp.org. The ctime() function is used for printing the response.

 

Writing a SNTP client


Unlike the previous recipe, sometimes, you don't need to get the precise time from the NTP server. You can use a simpler version of NTP called simple network time protocol.

How to do it...

Let us create a plain SNTP client without using any third-party library.

Let us first define two constants: NTP_SERVER and TIME1970. NTP_SERVER is the server address to which our client will connect, and TIME1970 is the reference time on January 1, 1970 (also called Epoch). You may find the value of the Epoch time or convert to the Epoch time at http://www.epochconverter.com/. The actual client creates a UDP socket (SOCK_DGRAM) to connect to the server following the UDP protocol. The client then needs to send the SNTP protocol data ('\x1b' + 47 * '\0') in a packet. Our UDP client sends and receives data using the sendto() and recvfrom() methods.

When the server returns the time information in a packed array, the client needs a specialized struct module to unpack the data. The only interesting data is located in the 11th element of the array. Finally, we need to subtract the reference value, TIME1970, from the unpacked value to get the actual current time.

Listing 1.11 shows how to write an SNTP client as follows:

#!/usr/bin/env python
# Python Network Programming Cookbook -- Chapter - 1
# This program is optimized for Python 2.7. It may run on any
# other Python version with/without modifications
import socket
import struct
import sys
import time

NTP_SERVER = "0.uk.pool.ntp.org"
TIME1970 = 2208988800L

def sntp_client():
    client = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    data = '\x1b' + 47 * '\0'
    client.sendto(data, (NTP_SERVER, 123))
    data, address = client.recvfrom( 1024 )
    if data:
        print 'Response received from:', address
    t = struct.unpack( '!12I', data )[10]
    t -= TIME1970
    print '\tTime=%s' % time.ctime(t)

if __name__ == '__main__':
    sntp_client()

This recipe prints the current time from the Internet time server received with the SNTP protocol as follows:

$ python 1_12_sntp_client.py 
Response received from: ('87.117.251.2', 123) 
      Time=Tue Feb 25 14:49:38 2014 

How it works...

This SNTP client creates a socket connection and sends the protocol data. After receiving the response from the NTP server (in this case, 0.uk.pool.ntp.org), it unpacks the data with struct. Finally, it subtracts the reference time, which is January 1, 1970, and prints the time using the ctime() built-in method in the Python time module.

 

Writing a simple echo client/server application


After testing with basic socket APIs in Python, let us create a socket server and client now. Here, you will have the chance to utilize your basic knowledge gained in the previous recipes.

How to do it...

In this example, a server will echo whatever it receives from the client. We will use the Python argparse module to specify the TCP port from a command line. Both the server and client script will take this argument.

First, we create the server. We start by creating a TCP socket object. Then, we set the reuse address so that we can run the server as many times as we need. We bind the socket to the given port on our local machine. In the listening stage, we make sure we listen to multiple clients in a queue using the backlog argument to the listen() method. Finally, we wait for the client to be connected and send some data to the server. When the data is received, the server echoes back the data to the client.

Listing 1.13a shows how to write a simple echo client/server application as follows:

#!/usr/bin/env python
# Python Network Programming Cookbook -- Chapter – 1
# This program is optimized for Python 2.7. It may run on any
# other Python version with/without modifications.

import socket
import sys
import argparse

host = 'localhost'
data_payload = 2048
backlog = 5 

def echo_server(port):
    """ A simple echo server """
    # Create a TCP socket
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    # Enable reuse address/port 
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    # Bind the socket to the port
    server_address = (host, port)
    print "Starting up echo server  on %s port %s" % server_address
    sock.bind(server_address)
    # Listen to clients, backlog argument specifies the max no. of queued connections
    sock.listen(backlog) 
    while True: 
        print "Waiting to receive message from client"
        client, address = sock.accept() 
        data = client.recv(data_payload) 
        if data:
            print "Data: %s" %data
            client.send(data)
            print "sent %s bytes back to %s" % (data, address)
        # end connection
        client.close() 
   
if __name__ == '__main__':
    parser = argparse.ArgumentParser(description='Socket Server Example')
    parser.add_argument('--port', action="store", dest="port", type=int, required=True)
    given_args = parser.parse_args() 
    port = given_args.port
    echo_server(port)

On the client-side code, we create a client socket using the port argument and connect to the server. Then, the client sends the message, Test message. This will be echoed to the server, and the client immediately receives the message back in a few segments. Here, two try-except blocks are constructed to catch any exception during this interactive session.

Listing 1-13b shows the echo client as follows:

#!/usr/bin/env python
# Python Network Programming Cookbook -- Chapter – 1
# This program is optimized for Python 2.7. It may run on any
# other Python version with/without modifications.

import socket
import sys

import argparse

host = 'localhost'

def echo_client(port):
    """ A simple echo client """
    # Create a TCP/IP socket
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    # Connect the socket to the server
    server_address = (host, port)
    print "Connecting to %s port %s" % server_address
    sock.connect(server_address)
    
    # Send data
    try:
        # Send data
        message = "Test message. This will be echoed"
        print "Sending %s" % message
        sock.sendall(message)
        # Look for the response
        amount_received = 0
        amount_expected = len(message)
        while amount_received < amount_expected:
            data = sock.recv(16)
            amount_received += len(data)
            print "Received: %s" % data
    except socket.errno, e:
        print "Socket error: %s" %str(e)
    except Exception, e:
        print "Other exception: %s" %str(e)
    finally:
        print "Closing connection to the server"
        sock.close()
    
if __name__ == '__main__':
    parser = argparse.ArgumentParser(description='Socket Server Example')
    parser.add_argument('--port', action="store", dest="port", type=int, required=True)
    given_args = parser.parse_args() 
    port = given_args.port
    echo_client(port)

How it works...

In order to see the client/server interactions, launch the following server script in one console:

$ python 1_13a_echo_server.py --port=9900 
Starting up echo server  on localhost port 9900 

Waiting to receive message from client 

Now, run the client from another terminal as follows:

$ python 1_13b_echo_client.py --port=9900 
Connecting to localhost port 9900 
Sending Test message. This will be echoed 
Received: Test message. Th 
Received: is will be echoe 
Received: d 
Closing connection to the server

Upon connecting to the localhost, the client server will also print the following message:

Data: Test message. This will be echoed 
sent Test message. This will be echoed bytes back to ('127.0.0.1', 42961) 
Waiting to receive message from client
About the Author
  • Dr. M. O. Faruque Sarker

    Dr. M. O. Faruque Sarker is a software architect based in London; he has shaped various Linux and open source software solutions mainly on cloud computing platforms for various institutions. Over the past 10 years, he has led numerous Python software development and cloud infrastructure automation projects. In 2009, he started using Python and shepherded a fleet of miniature E-puck robots at the University of South Wales, Newport, UK. Later, he was invited to work on the Google Summer of Code (2009/2010) programs to contribute to the BlueZ and Tahoe-LAFS open source projects. He is the author of Python Network Programming Cookbook, Packt Publishing and received his PhD in multirobot systems at the University of South Wales.

    Browse publications by this author
Latest Reviews (4 reviews total)
Good read! Learned better on how to integrate a GUI with tkinter.
Takes forever to ship, maybe when I receive it the new edition will be out
Atingiu e superou minhas expectativas.
Python Network Programming Cookbook
Unlock this book and the full library FREE for 7 days
Start now