Vision-Based Defect Detection System – Machines Can See Now!

Computer Vision (CV) is a field of artificial intelligence concerned with giving machines the ability to analyze and extract meaningful information from digital images, videos, and other visual input, as well as take actions or make recommendations based on the extracted information. Decades of research in the field of CV have led to the development of powerful Machine Learning (ML)-based vision algorithms that are capable of classifying images into some pre-defined categories, detecting objects from images, understanding written content from digital images, and detecting actions being performed in videos. Such vision algorithms have given businesses and organizations the ability to analyze large amounts of digital content (images and videos) and also automate processes to make instant decisions.

CV-based algorithms have changed the way we interact with smart devices in our day-to-day life – for example,...

Technical requirements

The code samples used in this chapter can be found at the following GitHub address: https://github.com/PacktPublishing/The-Definitive-Guide-to-Google-Vertex-AI/tree/main/Chapter16.

Vision-based defect detection

CV is capable nowadays of detecting visual defects on object surfaces or inconsistencies in their designs (such as dents or scratches on a car body), by just analyzing their digital photographs or videos. The manufacturing industry can leverage CV algorithms to automatically detect and remove low-quality and defected products from being packed and reaching customers. There are many possible ways to detect defects within digital content using CV-based algorithms. One simple idea is to solve defect detection as a classification problem, where a vision model can be trained to classify images such as good or defected. A more complex defect detection system will also locate the exact area of an image with a defect. The problem of identifying and locating visual defects can be solved using object-detection algorithms.

In this section, we will build and train a simple defect detection system step by step. In this example, we will use ML classification as a...

Deploying a vision model to a Vertex AI endpoint

In the previous section, we completed our experiment of training a TF-based vision model to identify detects from product images. We now have a trained model that can identify defected or broken bangle images. To make this model usable in downstream applications, we need to deploy it to an endpoint so that we can query that endpoint, getting outputs for new input images on demand. There are certain things that are important to consider while deploying a model, such as expected traffic, expected latency, and expected cost. Based on these factors, we can choose the best infrastructure to deploy our models. If there are strict low-latency requirements, we can deploy our model to machines with accelerators (such as Graphical Processing Units (GPUs) or Tensor Processing Units (TPUs)). Conversely, we don’t have the necessity of online or on-demand predictions, so we don’t need to deploy our model to an endpoint. Offline batch...

Getting online predictions from a vision model

In the previous section, we deployed our custom TF-based model to a Vertex AI Endpoint so that we could embed it into any downstream application, querying it for on-demand or online predictions. In this section, we will see how we can call this endpoint for online predictions programmatically using Python. However, the prediction requests can also be made by using a curl command and sending a JSON file with input data.

There are a few things to consider while making prediction requests; the most important part is pre-processing the input data accordingly. In the first section, when we trained our model, we did some pre-processing on our image dataset to make it compatible with the model. Similarly, while requesting the predictions, we should follow the exact same data preparation steps. Otherwise, either the model request will fail, due to an incompatible input format, or it will give bad results, due to training-serving skew. We already...

Summary

In this chapter, we created an end-to-end vision-based solution to detect visual defects from images. We saw how CNN-based deep learning architectures can be used to extract useful features from images and then use those features for tasks such as classification. After training and testing our model, we went ahead and deployed it to a Vertex AI endpoint, allowing it to serve online or on-demand prediction requests for any number of downstream applications.

After completing this chapter, you should be confident about how to approach vision-based problems and how to utilize ML to solve them. You should now be able to train your own vision-based classification models to solve real-world business problems. After completing the second section on deploying a custom model to a Vertex AI endpoint and the third section on getting online prediction from a Vertex endpoint, you should now be able to make your custom vision models usable for any downstream business application, by deploying...