Computer Vision: Anthropology of Algorithmic Bias in Facial Analysis Tool

Abstract

The usage of Computer Vision (CV) has led to debates about the bias within the technology. Despite machines being labeled as autonomous, human bias is embedded in data labeling for effective machine learning. Proper training of neural network machines requires massive amounts of “relevant data,” however, not all data is collected. This contributes to a one-sided view and feeds a “standard of data that is not collected.” The machine develops algorithmic decision-making based on the data it is presented, which can create machinic biases such as differences in gender, race/ethnicity, and class. This raises questions about which bodies are recognized by machines and how they are taught to “see” beyond binary “male or female” limitations. The study aims to understand how Amazon’s Rekognition, a facial recognition and analysis tool, analyzes and classifies people of dissident genders who do not conform to “conventional” gender norms. Understanding the mechanisms behind the technology’s decision-making processes can lead to more equitable and inclusive outcomes.

1. Introduction

Artificial Intelligence (AI) is full of biological analogies we tend to recognize their machinic bodies and organs, through sci-fi stories, as in the case of Computer Vision (VC) from 2001: A Space Odyssey [1]. In the movie [2], HAL 9000, a sentient AI, a Heuristic Algorithmic programming computer, is composed of cameras with fisheye lenses and an internal structure constituted by the binomial <algorithms+data>, which in many branches of AI is impossible to dissociate [3]. This structure provides Hal with the visual inputs needed to scan and analyze the spacecraft Discovery, and is imbued with natural language (similar to a human voice) to interact with the interstellar mission crew.

As in history, we are surrounded by voices (Siri, Alexa, and Google Assistant) and machine eyes through cell phones, notebooks, cameras installed on poles, in ATMs, subways, cars, busses, and drones, whether autonomous or guided. All configured with objectives and must present results with the means shown to them; thus, the scanning of facial expressions goes through a series of tangles inspired by the human brain, the convolutional neural networks (CNN), which are responsible for the analysis and processing of data. Videos and images so for are an effective knowledge result, and a lot of data is needed for the neural network to learn about who or what is seeing these ramifications are made up of norms and models, which regulate the constitutive patterns of what to see, for whom to look, and what should be described about whom one is looking at. Commonly, these models are known as algorithms, in terms of computer science, and they are any computational, mathematical, and statistical procedures aligned to take values or set of values as input and produce set of values as output [4, 5].

In other words, machines learn by using models and analyzing patterns through algorithms, so the computer learns about what it is seeing according to the data presented to it, over and over again, so that it can understand what differs a leopard from a cat, for example. In this way, the objective of AI algorithms is learning, so when new information is presented, it knows how to classify, regardless of what was previously shown to the algorithm, analyzing patterns through the data articulated with each other to generate results. From this point of view, Russell [6] and Lee [7] argue that neural networks demonstrate effective recognition after proper training with labeled examples that connect the many data points to the expected result, and this action according to both requires massive amounts of “relevant data.”

Data considered relevant are those that are always contained in the machines, even with the massive production of existing data, not all are collected and if they are, they go through a screening. Thus, no matter how sophisticated they are, algorithms are useless in isolation, and part of their results is supported by the data and samples contained in them, as well as in the way it interacts with the environment [8]. Crucially, data are people [9] if a certain group is included and others are on a smaller scale, statistically the “data that is left out” does not exist according to the analysis of the machine, so the algorithms analyze that a certain group of people is considered hegemonic [10]. In this way, the machine develops algorithmic decision making based on what appears in the data, establishing parameters that express the machinic biases.

2. Anthropology of algorithmic bias

Medeiros [3] argues that: a) algorithms can be blamed for certain results, even when there is curation performed on the entered data. In other cases: b) algorithms specified under inappropriate assumptions generate inappropriate solutions, even though the data have been well configured. There is also the possibility that: c) both algorithms and data have human biases, which may have been unconsciously inserted by their programmers and creators. In this sense, from the anthropological point of view, Forsythe [11] argues that software embodies values that are tacitly maintained by those who build them.

You need to consider sources of bias throughout the data lifecycle – collection, curation, analysis, storage and archiving. […] the responsibility does not end with archiving data, or delivering software. Regardless of whether bias exists in data, algorithms, or in their combination, it always appears due to humans – in collection, analysis or interpretation, whether intentional or through ignorance. And when adding machines to the binomial, more questions arise. ([3]:12)

To a certain extent, AI algorithms are human extensions, so by extensions it is understood that there is an automated widening of biases and as noted earlier, even though machines are named as autonomous, such as AI algorithms, human bias [5, 12] is embedded for effective learning. It is with the premise of corrosive software in the social context that investigations about Computer Vision (VC) are consistent in several debates about its use and the biases inserted in the machines, therefore racist algorithmic biases [13, 14, 15, 16], as well as disparities related to gender, class, politics, democracy, and surveillance.

Like Hal 9000—the computer presented at the beginning of this article—the information contained in it constituted his decision making for what he should do and in which situations he should act, but his actions were based on the decisions of his creator Dr. Chandra. There is a lot of controversy about the story, but it still presents a scenario that demonstrates how the objectives embedded in analysis software can establish significant changes in the short/long term, if they do not go through a bias audit process. In this sense, when analyzing the bibliographies and documentary about the algorithmic gender bias, race¹, it was analyzed that a large part is articulated around the social binary “woman” or “man.”

2.1 AWS Amazon rekognition (facial analysis)

From the situations presented by Joy Buolamwini in the documentary Coded Bias [17], in which she describes the process of a project for the MIT Media Lab in which facial analysis software from conglomerates is used: Google Cloud, Microsoft, AWS Amazon, and IBM Watson, and in most of these software, her face is not recognized, which configures that the tools did not recognize her as a human only when she started to use the white mask²—symbol of Algorithmic Justice League—there is recognition and it is noticed that in the course of the documentary, there are changes in the algorithms/data of these companies that begin to detect the scientist’s face, in addition to classifying her, indicating gender, age, and emotional aspects.

Considering that algorithms are not static, conglomerates modify, add, or remove data, modifying nuances of the algorithms frequently, whether for user experience (UX) or to prevent stocks from fluctuating, after all CEO’s of Google Cloud, Microsoft, AWS Amazon, IBM Watson, and other companies would not allow their brands to be linked to racism/gender discrimination or any factor that harms the brand, not for reasons of social concern, but for monetary reasons.

As a result, when researching the aforementioned tools, AWS Amazon Rekognition (Facial Analysis) proved to be more accessible, offering a layout in which the user can choose which tools she needs to use, in addition to: “no monetary charges.” Undoubtedly, in this segment, the principles of advertising do not fail: “there is no free lunch,” if I do not pay cash, so my information is the bargaining chip and that includes email address, digital traces, photo, age, academic background, orders placed on Amazon, mobile number, geolocation, etc.… still, Google was tested, but not to the same degree as Amazon (Figure 1).

Figure 1.

AWS splash screen recognition-facial analysis.

Let us return to the tool, AWS Amazon Rekognition (Facial Analysis) is a tool based on computer vision and deep learning, as previously mentioned, and machine learning works by making various data look for patterns repeatedly to then discern and recognize certain images, with that it performs facial recognition and analysis, detection of objects and texts, information about where faces are detected in an image or video, assigning points on faces and eye position, and “detecting emotions” (e.g., happy or sad) [18]. With that, I tested the tool to see if it was able to give me a result that was different from the one shown in the documentary in which Buolamwini was made invisible, I inserted 15 photographs of women, black celebrities, who socially identify as straight people, and the machine recognized them, classified them according to the available parameters as follows: looks like a face 99.9%, looks like a woman 99.9%, age group 27–37 years old, smiling 96.1%, looks happy, 95.9% is not wearing glasses 97.4%, is not wearing sunglasses 99.9%, eyes open 97.5% mouth open 95.4%, no mustache 98.2%, and no beard 95.8%.

2.2 Algorithmic misgender

Thus, based on the results obtained, I asked if the same would happen if photographs of bodies were inserted that do not correspond (and do not need to correspond) to the expectations of “conventional gender norms” if the tool analyzes, identifies, and classifies characteristics of people of dissident genders, and if analyzed, how are they taught to “see”? would be through of the biological/social limits of the “male/female” binary? [19, 20] or would they be unfeasible?

The analysis was carried out from December 2021 to February 2022, then from March 2022 to July 2022, and these spaces were purposeful to verify if there would be changes in the classification algorithms in relation to gender labels, in all months 15 photographs were used, and this time of celebrities who socially identify as non-binary, I observed that the Amazon Rekognition facial analysis method includes the following: a) inverted pyramid: eyes, nose, lips from one end to the other, and b) pyramid straight: nose lips from one end to the other. Triangulations are part of the parameters in which algorithms focus on the biological for precision analysis, in all analyzed photographs there were no classifications that transposed the female/male binary.

According to AWS, a binary gender prediction (male/female) is based on the physical appearance of a face in a given image. It does not enforce gender identity [21].

},

"Gender": {

"Value": "Female",

"Confidence": 55.517173767089844

Here the machine returns the labeling as being 55.51% female, but the person considers himself genderless/gender fluid.

},

"Gender": {

"Value": "Female",

"Confidence": 99.7735595703125

Above, the machine returns the labeling as being 99.77% female, but the person self-identifies as non-binary and other factors that contribute to the analysis are not demonstrated, that is, we do not know what data makes up the analysis of the tool so that she arrived at this result.

The use, treatment, and/or mention of gender terms that do not correspond to the self-identity that a non-binary or trans-person self-identifies generates the misgender experience [22, 23, 24], even when recognition goes beyond the human social line, consisting in Automatic Gender Recognition (AGR) the automatism that algorithmically identifies the gender of individuals generating self-identification errors, and it is considered algorithmic misgender.

In fact, the joining of data and algorithms together with the company’s regulation determines the machinic vision indicating that the biological that is inserted in the social overlaps the self-identity; thus, the invisibility happens in the sense of the prominence of the “norm” that hegemonizes bodies and makes them invisible, in the sense of self-identity, and the dynamics of observation, analysis, and classification define labels that match the result. It seems to be a woman or it seems to be a man, according to the imposed biological/social norm.

Google, however, returned the result of gender-related labels such as: person. The “algorithmic adaptation” of this tool was to decode the labeling to indicate that the analyzed image is of a person, a human, a neutral, colorless, and genderless being. Indeed, the results indicate that these analysis tools do not learn and do not know that there are people who do not fit into the categorical double woman/man, so the catalog of possible identities [25] is not part of what machines need to learn.

The limitations presented in this research are in line with those of Keyes [24] and Scheuerman [26] in which the search for diversity in certain tools coincides with a large wall of data that cannot be analyzed and reviewed, but which still return with results. According to the objective imposed by the companies, the machinic eye is organized according to the categorical binary woman/man.

3. The BitLocker effect and the imponderables of academic life

This subtitle was not foreseen, but there is the so-called Murphy’s law³ or “the imponderables of real life,” [27] or even more the imponderables of academic life that is when the notebook believes it is suffering some kind of cyberattack and does not allow the human to have control of the situation, just like Hal 9000 my notebook had a single objective, not to allow anyone (not even me) to access the hardware.

Just over a week to send this article, I revised it, formatting, font, after exhausting days/weeks/hours of adjustments, the goal of finishing the article was overcome by my organic limitations. There are times when, after analyzing images in search engines, reading data lists, arguing, and referencing so much, this organic body that typed these lines runs out, unlike machines. I left the glasses on the corner of the table and lowered the notebook screen (I was condescending, after all I would like at least my machine to rest) a few minutes before sending the complete work when I returned to the workflow this around 30 minutes, which was my surprise when I lifted the notebook screen and noticed that instead of the wallpaper as usual, there was this warning (Figure 2):

Figure 2.

Author’s notebook screen.

From experience, the worst thing to do in these cases where our machines hijack our academic research and all the materials accumulated for the making of the doctoral thesis, projects, etc., is to consult online search engines, and that is because all the tutorials said the same thing: “click F2 directly, give this command and with that my friends the entire drive will be erased, with factory formatting and don’t forget to leave your likes.”

I had already gone through something similar to notebooks before this one, but seeing a work about to be submitted held hostage, my organic algorithms adjusted to deliver the result: Tears. I came up with this role so after so much pressure and desperation it was what I had at the moment. I could not think rationally, that was it, my own machine had betrayed me. I called my partner in tears and she said: “it would be comical if it wasn’t tragic, a work on machines hijacked by your own machine, leave him in the corner of thought and let’s have lunch.”

I was the one who kept thinking what was behind the notebook’s action? Until reaching this conclusion, the crying would not stop and the hours were passing, after lunch, I called the notebook company, the call center’s response was: “Bitlocker is when your machine thinks you may be suffering a cyberattack and is securing your information, you can only access it with Bitlocker recovery keys if you do not have a recovery key you will not be able to access your computer and if you cannot revert you will have to reset your device, this action removes all your files.”

I turned off my cell phone. A few minutes of silence until I recover and think of another way to recover the machine and files. I called Microsoft and was answered by an artificial intelligence who said he could only help if my case was related to XBOX, otherwise I should access the company chat. I accessed the Microsoft chat, I told the whole odyssey and to my surprise it was another artificial intelligence, that is, my regret had to be as objective as possible, I typed the whole story, but when I noticed the AI left the chat because “it didn’t understand my request.“ I went back to the chat, another AI, I typed everything again as objective and succinct as possible again, who knows then the AI would help me access my machine. After a few moments the answer came, I had to confirm my data and the BitLocker recovery keys were sent to my email. I entered the 48 digits and I have never been so relieved to see this article again.

4. Conclusions

The entanglement of human life with computer vision has been done through data, which is the main raw material of machine learning or Big Data, being the deluge of data that is made available to train computer vision. As they are inserted, they receive preliminary descriptions that consist of a tag, label, which enables standards, which regulate deep learning, the more data that is produced by us, the more data are selected, labeled, and each time more algorithmic performativity corresponds to what has been learned through datasets, but this has not given the guarantee that there is equity, diversity in facial recognition, as noted, Google chose not to describe people’s gender, it will be simpler to delete algorithms instead of teaching that there is diversity? Or remain in the hegemonic category like Amazon? Anyway bodies that do not adapt to the binary continue to be made invisible, whether by humans, increasingly sophisticated algorithms, but that do not work alone, we know that the standards that regulate this data tend to be biased by other humans.