Exploring Bias and Fairness

A biased machine learning model produces and amplifies unfair or discriminatory predictions against certain groups. Such models can produce biased predictions that lead to negative consequences such as social or economic inequality. Fortunately, some countries have discrimination and equality laws that protect minority groups against unfavorable treatment. One of the worst scenarios a machine learning practitioner or anyone who deploys a biased model could face is either receiving a legal notice imposing a heavy fine or receiving a lawyer letter from being sued and forced to shut down their deployed model. Here are a few examples of such situations:

• The ride-hailing app Uber faced legal action from two unions in the UK for its facial verification system, which showed racial bias against dark-skinned people by displaying more frequent verification errors. This impeded their work as Uber drivers (https://www.bbc.com/news/technology-58831373).
• Creators...

Technical requirements

This chapter includes some practical implementations in the Python programming language. To complete it, you will need to have a computer with the following libraries installed:

• pandas
• matplotlib
• scikit-learn
• numpy
• pytorch
• transformers==4.28.0
• accelerate==0.6.0
• captum
• catalyst

The code files are available on GitHub at https://github.com/PacktPublishing/The-Deep-Learning-Architect-Handbook/tree/main/CHAPTER_13.

Exploring the types of bias

Bias can be described as a natural tendency or inclination toward a specific viewpoint, opinion, or belief system, regardless of whether it is treated as positive, neutral, or negative. AI bias, on the other hand, specifically occurs when mathematical models perpetuate the biases embedded by their creators or underlying data. Be aware that not all information is treated as biases, as some information can also be knowledge. Bias is a type of subjective information, and knowledge refers to factual information, understanding, or awareness acquired through learning, experience, or research. In other words, knowledge is the truth without bias.

Note

Do not confuse bias in this book with the bias from the infamous “bias versus variance” concept in machine learning. Bias in this concept refers to the specific bias on how simple a machine learning model is concerning a certain task to learn. For completeness, variance specifies the sensitivity...

Understanding the source of AI bias

AI bias can happen at any point in the deep learning life cycle. Let’s go through bias at those stages one by one:

• Planning: During the planning stage of the machine learning life cycle, biases can emerge as decisions are made regarding project objectives, data collection methods, and model design. Bias may arise from subjective choices, assumptions, or the use of unrepresentative data sources. Project planners need to maintain a critical perspective, actively consider potential biases, engage diverse perspectives, and prioritize fairness and ethical considerations.
• Data preparation: This stage involves the following phases:
  • Data collection: During the data collection phase, bias can creep in if the collected data fails to represent the target population accurately. Several factors can contribute to this bias, including sampling bias, selection bias, or the underrepresentation of specific groups. These issues can lead to the creation...

Discovering bias and fairness evaluation methods

Fairness and bias are opposing concepts. Fairness seeks to ensure fair and equal treatment in decision-making for all individuals or groups, while bias refers to unfair or unequal treatment. Mitigating bias is a crucial step in achieving fairness. Bias can exist in different forms and addressing all potential biases is complicated. Additionally, it’s important to understand that achieving fairness in one aspect doesn’t guarantee the complete absence of bias in general.

To understand both how much bias and how fair our data and model are, what we need is a set of bias and fairness metrics to objectively measure and evaluate. This will then enable a feedback mechanism to iteratively and objectively mitigate bias and achieve fairness. Let’s go through a few robust bias and fairness metrics that you need to have in your arsenal of tools to achieve fairness:

• Equal representation-based metrics: This set of metrics...

Evaluating the bias and fairness of a deep learning model

In this practical example, we will be exploring the infamous real-world use case of face recognition. This practical example will be leveraged for the practical implementation of bias mitigation in the next section. The basis of face recognition is to generate feature vectors that can be used to carry out KNN-based classification so that new faces don’t need to undergo additional network training. In this example, we will be training a classification model and evaluating it using traditional classification accuracy-based metrics; we won’t be demonstrating the recognition part of the use case, which allows us to handle unknown facial identity classes.

The goal here is to ensure that the resulting facial classification model has low gender bias. We will be using a publicly available facial dataset called BUPT-CBFace-50, which has a diverse coverage of facial images that have different facial expressions, poses...

Tailoring bias and fairness measures across use cases

The process of figuring out bias and fairness metrics to use for our use case can flow similarly to the process of figuring out general model performance evaluation metrics, as introduced in Chapter 10, Exploring Model Evaluation Methods, in the Engineering the base model evaluation metric section. So, be sure to check that topic out! However, bias and fairness have unique aspects that require additional heuristical recommendations. Earlier, recommendations for metrics that belong to the same metric group were explored. Now, let’s explore general recommendations on the four metric groups:

• Equal representation is always desired when there is a sensitive and protected attribute. So, when you see these attributes, be sure to use equal representation-based metrics on both your data and the model. Examples include race, gender, religion, sexual orientation, disability, age, socioeconomic status, political affiliations...

Mitigating AI bias

AI bias is an algorithmic bias that either comes from the model itself through its learning process or the data it used to learn from. The most obvious solution to mitigate bias is not programmatic mitigation methods but ensuring fair processes when collecting data. A data collection and preparation process is only truly fair when it not only ensures the resulting data is balanced by sensitive attributes but also ensures all inherent and systematic biases are not included.

Unfortunately, a balanced dataset based on the sensitive attribute does not guarantee a fair model. There can be differences in appearance among subgroups under the hood or associative groups of the data concerning multiple factors, which can potentially cause a biased system. Bias, however, can be mitigated partially when the dataset is balanced compared to without concerning the observable sensitive groups. But what are all these attributes? It might be easier to identify data attributes in...

Summary

In this chapter, we focused on the critical issue of bias and fairness in machine learning models. The potential negative consequences of deploying biased models, such as legal actions and fines, were emphasized. We covered various types of biases and identified stages in the deep learning life cycle where bias can emerge, including planning, data preparation, model development, and deployment.

Several metrics for detecting and evaluating bias and fairness were also introduced, including equal representation-based metrics, equal error-based metrics, distributional fairness metrics, and individual fairness metrics. This chapter provided recommendations on selecting the right metrics for specific use cases and highlighted the importance of balancing opposing views, such as WAE and WYSIWYG, when evaluating fairness. This chapter also discussed programmatic bias mitigation methods that can be applied during the pre-processing, in-processing, and post-processing stages of model...