Ethical Issues in Research with Artificial Intelligence Systems

Abstract

There are many definitions of what an artificial intelligence (AI) system is. This chapter emphasises the characteristics of AI to mimic human behaviour in the process of solving complex tasks in real-world environments. After introducing different types of AI systems, the chapter continues with a brief analysis of the distinction between research into what an AI system is in its inner structure and research into the uses of AI. Since much literature is already devoted to the ethical concerns surrounding the use of AI, this chapter addresses the problem of accountability with respect to opaque human-like AI systems. In addition, the chapter explains how research ethics in AI is fundamentally different from research ethics in any other field. Often, the goal of engineers in this field is to build powerful autonomous systems that tend to be opaque. The aim is therefore to build entities whose inner workings become unknown to their creators as soon as these entities start the learning process. A split accountability model is proposed to address this specificity.

1. Introduction

Contrary to what most people believe, it is not easy to find a consensus definition of artificial intelligence (AI). The public may have an intuitive understanding of the term AI. Influenced by the media and the film industry, lay people may implicitly see AI as something close to what might be called an ‘evil, mysterious machine’. Certainly, there is some sense of automation and hidden cognitive-like processes that people have about AI, but this would not cover a trustworthy definition. Therefore, we combine different approaches currently found in the literature. We begin our paper by defining artificial intelligence (AI) as a system that attempts to mimic human behaviour in the process of solving complex tasks in real-world environments that are not easily solved by other methods. These tasks can include activities such as perception, reasoning, decision-making, problem-solving, learning, and natural language processing. Such a system should be able to learn and adapt in order to become effectively independent of its prior knowledge and make decisions accordingly. Of course, becoming independent does not mean it will ignore initial knowledge. However, this system should be able to create new and original outputs.

The field of AI is broad and encompasses many different subdisciplines, including machine learning, neural networks, computer vision, natural language processing, and robotics. AI systems can be classified according to their level of autonomy and intelligence, ranging from simple rule-based systems to more advanced ones that can learn and adapt to new situations. Historically, AI can be divided into four distinct generations. The first generation of AI systems was characterised by rule-based systems designed to solve specific problems, such as a game of chess. The second generation of AI systems was characterised by expert systems, designed to solve complex problems using knowledge representation and reasoning. It used techniques of deep learning, natural language processing, etc. The third and current generation of AI systems is characterised by machine learning systems, which are designed to learn from data and adapt to new situations.

There are four approaches to developing AI, based on the goal of either a) thinking humanly, b) acting humanly, c) thinking rationally or d) acting rationally. Depending on the goal, thinking may be more relevant than acting, or similarly, humanly versus rationally. The approaches are not mutually exclusive and usually overlap. In this paper, we will focus more on the process of building an AI system that is capable of thinking and acting humanly. The success of such systems is measured in terms of fidelity to human performance [1].

‘Thinking humanly’ or the cognitive modelling approach can be defined as the automation of activities that we associate with human thinking, activities such as decision-making, problem-solving and learning [2]. This includes the subfield of AI called Artificial General Intelligence (AGI), which aims to create systems that can think and learn in ways similar to human intelligence. AGI research focuses on creating systems that can perform a wide range of tasks and adapt to new situations, as humans do. This would require capabilities in a) natural language processing, b) knowledge representation, c) automated reasoning and d) machine learning. ‘Acting humanly’ is an approach definable as the study of how to make computers do things at which people are better [3]. This area of research focuses on creating AI systems that can act in ways similar to humans. This includes creating systems that can understand and respond to natural language, recognise and respond to emotions, as well as interact with humans in a natural and intuitive way. This area of research is often referred to as human-robot interaction (HRI) or human-AI interaction (HAI).

AI systems that aim to think and act rationally include decision-making systems that can make optimal decisions based on (uncertain) information, including systems that can act in ways that are consistent with logical reasoning and probability theory. Ideally, they should always be able to do the ‘right thing’ given what they know at any given time.

In terms of approaches to building AI, there are two classes based on the transparency of their actions: black-box models and glass-box (or interpretable) models. Currently, the models with the best predictive accuracy, and therefore the most powerful, are paradoxically those with the most opaque black-box architectures [4]. A black-box AI system is based on a model or algorithm whose inner workings are not easily explained or understood by humans. This means that it is difficult or impossible to understand how the system arrived at a particular decision or prediction, even if the inputs and outputs are known. The term ‘black box’ is used to reflect the lack of transparency or visibility into the inner workings of the system. Examples of black-box AI models include deep neural networks and some complex forms of machine learning algorithms, such as random forests or gradient boosting. These models can be highly accurate and perform very well at certain tasks, but they can be difficult to interpret or understand why they make certain decisions. This can be an issue in areas such as healthcare or finance, where transparency and explainability are important for regulatory compliance and ethical considerations. In contrast, the inner workings of Glass Box AI models are transparent and explainable. They allow the user to see how the model arrived at a particular decision or prediction by providing access to the model’s logic, parameters and features. This transparency allows users to understand the model’s behaviour, identify potential problems and make adjustments to improve its performance. Examples of Glass Box AI models include decision trees, linear regression and rule-based systems.

Explainable AI (XAI) is also known as Interpretable AI or Explainable Machine Learning (XML). It is the current research direction that attempts to develop a set of processes and methods that aim to make AI systems more transparent, interpretable and explainable to humans. According to some authors, there are four main principles of XAI. The first is explanation: a system should provide evidence or reasons for its outputs and/or processes. The second is understandability: the explanations provided by the system should be understandable to the intended audience. The third is accuracy of explanation: the explanations should correctly reflect the reasons for generating the outputs and/or processes of the system. Finally, the principle of knowledge limits states that the system should only operate within the conditions for which it was designed and should only provide outputs when it has a sufficient level of confidence in them [5].

Some authors propose an alternative and complementary approach to XAI. It involves describing black-box processing in AI using experimental methods from psychology, in the same way that scientists do with humans. Using the experimental method, cognitive psychology uses carefully designed stimuli (input) and measures the corresponding behaviour (output) to make causal inferences about the structure and function of the human mind. The same approach could be applied to artificial minds [6].

2. Research with artificial intelligence systems

A difficult distinction to make is between different types of research on and with artificial intelligence systems. We propose a distinction between research on how artificial intelligence systems can be applied to various tasks and real-world problems, as well as research on the process of building artificial intelligence itself, or on what artificial intelligence is (or should be). Typically, AI systems are built with the goal of performing a specific task. It could be argued that the above distinction is artificial. We argue that focusing our ethical discussions on the various applications of AI is likely to miss more important issues. In particular, we argue that the ethical question of what artificial intelligence systems are is at least as important as the ethical question of how artificial intelligence systems can be used.

There is already a large body of literature on the ethics of the use of artificial intelligence systems. The most debated issues include privacy and surveillance, manipulation of behaviour, bias in decision-making systems, human-robot interaction, automation and employment, and artificial moral agents [7]. Topics dealing specifically with the ethics of research into what an AI system really is are mixed with topics on the ethics of different uses of AI.

We suggest emphasising the distinction between the research ethics of AI applications and the research ethics of what AI is (or becomes) in itself. For example, the research ethics of AI applications and uses include the weaponisation of AI systems [8, 9]. Another controversial topic is artificial intelligence as a moral advisor or decision-maker [10]. On the other hand, ethical discussions of what we might call the ‘ontology’ of an AI system are mostly concerned with the problem of opacity [11]. The most powerful systems are those with the most opaque black-box architectures [4]. It is therefore highly likely that research focused on increasing the power of artificial intelligence will build more opaque systems. Another issue that might fit uncomfortably into what we might call the ‘ontology’ of an AI is machine ethics. One way to define it is the ability of an AI to take human values into account in its decision-making process [7].

Ethicists are concerned about the problems of opacity. There are concerns about the creation of decision-making processes that limit opportunities for human participation [12]. These concerns include the fact that people affected by the decision of an AI will not be able to know how the system arrived at the decision in the first place. Therefore, the system is ‘opaque’ not only to the people affected by the decision, but also to the expert, especially when the systems involve machine learning. Most experts dealing with opaque AI systems will not know what the pattern that generated a decision is. The programme itself evolves in such a way that as new data comes in, the patterns used by the AI system change. Certain inputs can be allowed or not, and this generates (some degree of) control. However, what this artificial intelligence system becomes is not transparent to the people who create it [7].

3. Why research ethics in AI is completely different

Research ethics is rarely treated as a general, all-encompassing field. We usually talk about research ethics in a particular field. We discuss the ethics of clinical research, the ethics of bioengineering research, the ethics of gene sequencing, and so on. In most cases, principles derived from general ethics are applied to specific fields and issues. For example, biomedical research ethics often uses the checklist of the four ecumenical principles: respect for autonomy, nonmaleficence, beneficence and justice [13]. These are treated together with Kantian, virtue-based and utilitarian approaches to consider the nature of the choice per se, but also its consequences.

In almost all cases, the ethical analysis considers an explicit intention on the part of researchers to better understand the entity they are studying qua that entity. For example, when researching the molecule of a new drug, scientists have the explicit intention of creating (and patenting) a new molecule. Their aim is to understand that molecule, its structure and how it works (e.g. side effects and pharmacokinetics). Usually, the purpose is pragmatic: scientists try to achieve a better result (e.g. antidepressant effect) than other similar products or interventions already on the market. After previous investigations in cell cultures and animal models, the research progresses to the point where superiority could be demonstrated in a trial. In a classic design, consenting patients would be randomised into at least two equivalent groups. One group would receive the gold standard treatment for the condition. The other would receive the newly discovered molecule. The scientists try to control for all other variables. The dependent variable (usually the desired outcome) would be measured in both groups. Differences (if any) would be statistically tested to see if and how they were due to chance alone.

As we can see in the example above, the intention of scientists is to create something that they understand better. The reason for this is that better uses of the drug are closely linked to a better understanding of the entity being created. It is the same with engineering technologies. If a “better” engine is being researched, the intention is to get better results through a better understanding of its functions and structure. When a better processor is created, the intention is to achieve better performance through a better understanding of the technology within. When improvements are made to an engine, drug, or processor, those improvements are made from the outside. It is the scientists who come up with something new through a better understanding of the entity and this entity’s uses over time. We argue that AI research is fundamentally different from other types of research, as we will try to show. Other research efforts create things by understanding them better. Improvements are made from outside that entity, by the scientists themselves, using knowledge gathered by the same scientists or by others in the scientific community.

We began our paper by defining AI as a system that attempts to mimic human behaviour in the process of solving complex tasks, adapting to become effectively independent of its prior knowledge (but without ignoring it) and making decisions accordingly. A better degree of independence of its prior knowledge means that the inner workings of the AI become less a direct effect of an outside intervention by humans. This means that, for the first time in history, scientists are creating something with the explicit intention that it will become something other than what was created in the beginning. This means that people are creating something that has the distinct property of changing itself into something that cannot be described. It is not an attempt to create something that can be better understood qua what it is in the moment of its creation. When trying to create general AI with the purpose of it being human-like, it’s an attempt to create something whose structure becomes increasingly unknown as soon as it’s released. It could be argued that this kind of change does not happen ‘by itself’, since data must be input in order for the AI to ‘learn’ new paths. Data is necessary, but by the very definition of opaque AI, the nature of the data input will not control how the artificial intelligence system changes from within. Data input may have unexpected results as well. Some outputs may be satisfactorily predicted, but the pathways inside the system will remain potentially unknown to those who created it.

We can compare opaque AI systems to the personality and behaviour of a child given a certain education. The educational input can indeed be controlled. But this input is not the only thing that explains the structures that are being created in a child’s mind. This does not mean that the child’s ‘personality’ can ever be controlled and accurately predicted. As with humans, whose personalities are the result of nature, nurture and chance (chance being a variable both in ‘nurture’ and ‘nature’), the inner workings of an opaque AI system would change into something that might be unknown. And as with people whose parents want more autonomously thinking offspring, the research with artificial intelligence will aim the unknown which is subsequent to AI’s own development. It is the explicit intention of the researchers to create a ‘mind’ whose inner structure will (soon) become different than what it was when it was created. This is because it is supposed to have a degree of autonomy in its cognitive processes and decisions. It is, so to speak, a mind that creates itself. And the capacity for self-creation and development means that researchers want their creation to become something new as soon as it works.

This completely changes the ethics behind research ethics. In classical research ethics, responsibility and other principles are linked to the degree of control over what the research entity is. To control it, one must understand it. Classical research does just that. It considers what is known with respect to what the entity being created qua that entity. It links what that entity is with responsibilities about that entity’s uses and possible consequences. Of course, not all possible consequences are known to the researcher from the very beginning. For instance, nuclear fission had several uses, and not all of them were predicted from the beginning by Lise Meitner, one of the two scientists having discovered the phenomenon. But her intent as a researcher was to get a better grasp of the nuclear fission qua what it was even though she did not know at the time how her discoveries might be used.

In the case of some AI systems, things are different, because researchers create entities that intrinsically contain some degree of unknown changes as part of their own performance. This should not be confounded with various uses that the researcher can or cannot predict at the moment of AI system’s creation. This second topic is abundantly covered in literature. When dealing with AI, researchers try to discover more and more performant systems and, in some cases, this performance is intrinsically linked with the unpredictability itself. We could even say that the performance of the AI system is, in some cases, the very fact that they are unpredictable (and therefore autonomous or human-like). For instance, if one creates an AI system with the purpose of teaching it to generate original avatars from real-life photos, the performance of the system will be measured, among other things, by the degree of originality in combining various personal features with other elements that are not in the original photos. These elements, of course, belong to some other input. However, the resulted combination is original and unpredictable. One of the explicit aims of the researchers, in this simple example, is to create an entity which becomes more and more original in its outputs. Since it is original in its outputs, it is original in its inner structure. Its inner working can become fully unknown to the people who created it, just because they created it so it can become original and different qua what it is.

We believe that this dynamic changes the semantics we usually use in research ethics. In regular research ethics, principles are used in a situation where people research or create something they try to understand as much as possible, not only in the beginning but also afterwards. In AI research ethics, at least in some cases (e.g. black-box or human-like), principles cannot be used in the same way. The reason is that people in these situations create an entity with the explicit purpose that it becomes, as soon as it starts to ‘learn’, something else than what it was when it started to function. Consider the degree of researchers’ accountability with respect to an entity that is meant to become, as soon as it starts to function, something else in its inner workings than what was designed. Many AI systems are (partially), something else than what they were the moment researchers finished creating them as entities capable of learning. Should a researcher be held accountable in this situation in the same way he/she is held accountable when the purpose is the most comprehensive understanding of an entity or a phenomenon? Should researchers be held accountable only with respect to what the AI was before it started to learn? If they can be held accountable with respect to what they fed intentionally as inputs, can they also be held accountable with respect to inputs that were not intended by the researchers? For instance, connecting a black-box AI to the World Wide Web makes it unlikely for researchers to be able to control everything that the AI learns from the Internet.

On the other hand, even if input is 100% controlled by researchers, the manner in which input is processed (combined, structured and restructured) cannot be controlled. The very essence of an autonomous, human-like artificial intelligence system is the capacity to bring forth its own solutions and models. Therefore, not in all cases, the AI’s output is a ‘spitting image’ of the input, so we can draw responsibility from the output from the responsibility from the input. How can one be held accountable for something that is meant to change in its being as soon as it starts to function, and those changes are unpredictable?

A recent example is ChatGPT (https://chat.openai.com/chat) which became, quite quickly, an extraordinary instance of how opaque (and powerful) artificial intelligence works. In summary, the software is able to answer questions, build up cogent arguments, and write short essays on a given topics or even small pieces of literature. Being connected to the Internet, OpenAI chatbot has also found itself in the middle of several ethical controversies. For instance, many (underpaid) workers had to ‘manually’ input data in order to help the chatbot avoid interactions with users on unethical topics like illegal sexual content or racism [14]. ChatGPT, the chatbot for OpenAI, can also give us some examples about how unintended internal models can be created by artificial intelligence, promoting bias. The specialists working for the improvement of ChatGPT used seven known tests in order to verify the output for implicit biases [15, 16]. They admit: ‘we found that our models more strongly associate (a) European American names with positive sentiment, when compared to African American names, and (b) negative stereotypes with black women’ [17]. It is just an example of how unpredictable the inner working of artificial intelligence is. This is so in the absence of any type of known intention from the part of the developers, especially when the system is connected to the World Wide Web. If we are to get back to the child analogy, we can easily conclude that allowing a child to navigate the Internet unsupervised might create unexpected and strong tendencies in their personality, despite the parents never having intended it. It might be the same with artificial intelligence models.

The strategy of ‘manually patching’ artificial intelligence with interdictions, so it will not discuss inappropriate sexual content or stock investments, is just a temporary solution. The palette of desired and undesired possible outputs is almost infinite and unpredictable. Teams can be hired to ‘manually’ identify inappropriate content. Rules can be added, so the artificial intelligence will ‘refuse’ outputs on certain topics. This does not solve the problem by far. It is our opinion that in a situation like ChatGPT’s, it will be virtually impossible to control what models are created in its inner workings and what type of outputs can be generated. Without being too pessimistic about it, readers should admit that humans have not succeeded in satisfactorily understanding themselves. We know more about the way we make decisions, but not too much. We have general ideas about how personalities come to be, but we are not able to map the inner workings of our mind in processing large quantities of data or preferences. It is genuinely a challenge to already create entities that emulate humans when humans do not understand themselves very well. In the end, all complex data that is fed to an artificial intelligence system has some degree of cultural imbuement in it. It is likely that all output from artificial intelligence systems will carry with them something of the ‘wrongs’ and the ‘rights’ of humanity.

Current policies focus more on the accountability for AI’s applications and less on AI’s structure. For instance, the European Parliament states that AI can replace neither human decision-making nor human contact. It calls for a prohibition of lethal autonomous weapon systems. It also raises concerns over the use of AI in spreading disinformation and influencing elections [18]. More general recommendations also come from UNESCO [19]. Policies do not appear to address the problem of unpredictability of the inner workings of AI which might engender subsequent problems with the output. Policies do not seem to state who and how someone should be held accountable for what an artificial intelligence system is before what an artificial intelligence system does.

A possible split model of accountability could be imagined in the case of opaque systems. Researchers could be held accountable to the following: the inner structure of the artificial intelligence system at its tabula rasa moment of existence (before the learning process). Also, researchers could be held accountable for the data input that has been intentionally fed to the artificial intelligence system. The community, represented by governments, elected bodies representing the public could be held accountable for: the differences between the what the AI system was in its tabula rasa moment and all future moments, when input has been accounted for. Another collective accountability could be imagined for the outputs of an artificial intelligence system allowed to connect to uncontrolled sets of data (e.g. World Wide Web). In both cases, for opaque systems, the responsibility of the output cannot belong solely to the researchers. The reason is that researchers are unable, in some cases, to know what the entity they are creating becomes in its being, even if they take all precautions. If the public is unwilling to share collective responsibility for the above elements, then opaque system research should be banned. Making researchers accountable for all the developments of opaque systems is unfair, and it will simply not work for individuals and societies.

4. Conclusions

The purpose of this chapter has been to address the ethical concerns of accountability in AI research with respect to the inner workings of opaque AI and human-like systems. The ethics of research with opaque AI systems is fundamentally different from the ethics of any other area of research. This is because, for the first time in history, scientists are creating something with the explicit intention that it will almost automatically become something other than what was created at the beginning. Unpredictability relates to the inner workings of the system, but also to the input, when the system is connected to large amounts of uncontrolled data. Some examples are given. Engineers developing AI systems cannot be the only ones held accountable for something that is partly designed to be autonomous and unpredictable. We propose that accountability for the problem of opaque AI systems be divided between researchers and society. Society, through elected bodies, could express agreement or disagreement about what course of action should be taken. We can either restrict AI research to glass-box systems or we can share responsibility for research into black-box systems.