Chapter 3. Convolution and Frequency Domain Filtering

In this chapter, we will continue with 2D convolution and understand how convolution can be done faster in the frequency domain (with basic concepts of the convolution theorem). We will see the basic differences between correlation and convolution with an example on an image. We will also describe an example from SciPy that will show how to find the location of specific patterns in an image with a template image using cross-correlation. Finally, we will describe a few filtering techniques (that can be implemented with convolution using kernels, such as box-kernel or Gaussian kernel) in the frequency domain, such as  high-pass, low-pass, band-pass, and band-stop filters, and how to implement them with Python libraries by using examples. We shall demonstrate with examples how some filters can be used for image denoising (for example, the band-reject or notch filters to remove periodic noise from an image, or the inverse or Wiener filters...

Convolution theorem and frequency domain Gaussian blur

In this section, we will see more applications of convolution on images using Python modules such as scipy signal and ndimage. Let's start with convolution theorem and see how the convolution operation becomes easier in the frequency domain.

Application of the convolution theorem

The convolution theorem says that convolution in an image domain is equivalent to asimple multiplication in the frequency domain:

Following diagram shows the application of fourier transforms:

The next diagram shows the basic steps in frequency domain filtering. We have the original image, F, and a kernel (a mask or a degradation/enhancement function) as input. First, both input items need to be converted into the frequency domain with DFT, and then the convolution needs to be applied, which by convolution theorem is just an (element-wise) multiplication. This outputs the convolved image in the frequency domain, on which we need to apply IDFT to obtain the reconstructed...

Filtering in the frequency domain (HPF, LPF, BPF, and notch filters)

If we remember from the image processing pipeline described in Chapter 1, Getting Started with Image Processing, the immediate next step after image acquisition is image pre-processing. Images are often corrupted by random variations in intensity and illumination or have poor contrast, and therefore can't be used directly and need to be enhanced. This is where the filters are used.

What is a filter?

Filtering refers to transforming pixel intensity values toreveal certain image characteristics such as:

• Enhancement: This image characteristic improves contrast
• Smoothing: This image characteristic removes noise
• Template matching: This image characteristic detects known patterns

The filtered image is described by a discreteconvolution and the filter is described by an x ndiscreteconvolution mask.

High-Pass Filter (HPF)

This filter allows only high frequencies from the frequency domain representation of the image (obtained with DFT)...

Summary

In this chapter, we discussed a few important concepts primarily related to 2D convolution and its related applications in image processing, such as filtering in the spatial domain. We also discussed a few different frequency domain filtering techniques and illustrated them with quite a few examples with the scikit-image numpy fft, scipy, fftpack, signal, and ndimage modules. We started with the convolution theorem and its application in frequency domain filtering, various frequency domain filters such as LPF, HPF, and notch filters, and finally deconvolution and its application in designing filters for image restoration, such as inverse and Wiener filters.

On completion of this chapter, the reader should be able to write Python code to do 2D convolution/filtering, and should also be able to write Python code to implement time/frequency domain filters with/without convolution.

In the next chapter, we will start on different image enhancement techniques based on the concepts introduced...

Questions

1. Plot the frequency spectrum of an image, a Gaussian kernel, and the image obtained after convolution in the frequency domain, in 3D (the output should be like the surfaces shown in the sections) using the mpl_toolkits.mplot3d module. (Hint: the np.meshgrid()function will come in handy for the surface plot). Repeat the exercise for the inverse filter too.
2. Add some random noise to the lena image, blur the image with a Gaussian kernel, and then try to restore the image using an inverse filter, as shown in the corresponding example. What happens and why?
3. Use SciPy signal'sfftconvolve() function to apply a Gaussian blur on a color image in the frequency domain.
4. Use the fourier_uniform() and fourier_ellipsoid() functions of the ndimage module of SciPy to apply LPFs with box and ellipsoid kernels, respectively, on an image in the frequency domain.

Further reading

• https://www.cs.cornell.edu/courses/cs1114/2013sp/sections/S06_convolution.pdf
• http://www.aip.de/groups/soe/local/numres/bookcpdf/c13-3.pdf
• http://www.cse.usf.edu/~r1k/MachineVisionBook/MachineVision.files/MachineVision_Chapter4.pdf
• https://web.stanford.edu/class/ee367/slides/lecture6.pdf
• https://pdfs.semanticscholar.org/presentation/50e8/fb095faf6ed51e03c85a2fcb7eb1ae1b1009.pdf
• http://www.robots.ox.ac.uk/~az/lectures/ia/lect2.pdf