What is a moral machine?

April 1, 2020

Pär SegerdahlI recently read an article about so-called moral robots, which I found clarifying in many ways. The philosopher John-Stewart Gordon points out pitfalls that non-ethicists – robotics researchers and AI programmers – may fall into when they try to construct moral machines. Simply because they lack ethical expertise.

The first pitfall is the rookie mistakes. One might naively identify ethics with certain famous bioethical principles, as if ethics could not be anything but so-called “principlism.” Or, it is believed that computer systems, through automated analysis of individual cases, can “learn” ethical principles and “become moral,” as if morality could be discovered experientially or empirically.

The second challenge has to do with the fact that the ethics experts themselves disagree about the “right” moral theory. There are several competing ethical theories (utilitarianism, deontology, virtue ethics and more). What moral template should programmers use when getting computers to solve moral problems and dilemmas that arise in different activities? (Consider self-driving cars in difficult traffic situations.)

The first pitfall can be addressed with more knowledge of ethics. How do we handle the second challenge? Should we allow programmers to choose moral theory as it suits them? Should we allow both utilitarian and deontological robot cars on our streets?

John-Stewart Gordon’s suggestion is that so-called machine ethics should focus on the similarities between different moral theories regarding what one should not do. Robots should be provided with a binding list of things that must be avoided as immoral. With this restriction, the robots then have leeway to use and balance the plurality of moral theories to solve moral problems in a variety of ways.

In conclusion, researchers and engineers in robotics and AI should consult the ethics experts so that they can avoid the rookie mistakes and understand the methodological problems that arise when not even the experts in the field can agree about the right moral theory.

All this seems both wise and clarifying in many ways. At the same time, I feel genuinely confused about the very idea of ​​”moral machines” (although the article is not intended to discuss the idea, but focuses on ethical challenges for engineers). What does the idea mean? Not that I doubt that we can design artificial intelligence according to ethical requirements. We may not want robot cars to avoid collisions in city traffic by turning onto sidewalks where many people walk. In that sense, there may be ethical software, much like there are ethical funds. We could talk about moral and immoral robot cars as straightforwardly as we talk about ethical and unethical funds.

Still, as I mentioned, I feel uncertain. Why? I started by writing about “so-called” moral robots. I did so because I am not comfortable talking about moral machines, although I am open to suggestions about what it could mean. I think that what confuses me is that moral machines are largely mentioned without qualifying expressions, as if everyone ought to know what it should mean. Ethical experts disagree on the “right” moral theory. However, they seem to agree that moral theory determines what a moral decision is; much like grammar determines what a grammatical sentence is. With that faith in moral theory, one need not contemplate what a moral machine might be. It is simply a machine that makes decisions according to accepted moral theory. However, do machines make decisions in the same sense as humans do?

Maybe it is about emphasis. We talk about ethical funds without feeling dizzy because a stock fund is said to be ethical (“Can they be humorous too?”). There is no mythological emphasis in the talk of ethical funds. In the same way, we can talk about ethical robot cars without feeling dizzy as if we faced something supernatural. However, in the philosophical discussion of machine ethics, moral machines are sometimes mentioned in a mythological way, it seems to me. As if a centaur, a machine-human, will soon see the light of day. At the same time, we are not supposed to feel dizzy concerning these brave new centaurs, since the experts can spell out exactly what they are talking about. Having all the accepted templates in their hands, they do not need any qualifying expressions!

I suspect that also ethical expertise can be a philosophical pitfall when we intellectually approach so-called moral machines. The expert attitude can silence the confusing questions that we all need time to contemplate when honest doubts rebel against the claim to know.

Pär Segerdahl

Gordon, J. Building Moral Robots: Ethical Pitfalls and Challenges. Sci Eng Ethics 26, 141–157 (2020). https://doi.org/10.1007/s11948-019-00084-5

This post in Swedish

We like challenging questions - the ethics blog


Artificial intelligence and living consciousness

March 2, 2020

Pär SegerdahlThe Ethics Blog will publish several posts on artificial intelligence in the future. Today, I just want to make a little observation of something remarkable.

The last century was marked by fear of human consciousness. Our mind seemed as mystic as the soul, as superfluous in a scientific age as God. In psychology, behaviorism flourished, which defined psychological words in terms of bodily behavior that could be studied scientifically in the laboratory. Our living consciousness was treated as a relic from bygone superstitious ages.

What is so remarkable about artificial intelligence? Suddenly, one seems to idolize consciousness. One wallows in previously sinful psychological words, at least when one talks about what computers and robots can do. These machines can see and hear; they can think and speak. They can even learn by themselves.

Does this mean that the fear of consciousness has ceased? Hardly, because when artificial intelligence employs psychological words such as seeing and hearing, thinking and understanding, the words cease to be psychological. The idea of computer “learning,” for example, is a technical term that computer experts define in their laboratories.

When artificial intelligence embellishes machines with psychological words, then, one repeats how behaviorism defined mind in terms of something else. Psychological words take on new machine meanings that overshadow the meanings the words have among living human beings.

Remember this next time you wonder if robots might become conscious. The development exhibits fear of consciousness. Therefore, what you are wondering is not if robots can become conscious. You wonder if your own consciousness can be superstition. Remarkable, right?

Pär Segerdahl

This post in Swedish

We challenge habits of thought : the Ethics Blog


Neuroethics as foundational

January 28, 2020

Pär SegerdahlAs neuroscience expands, the need for ethical reflection also expands. A new field has emerged, neuroethics, which celebrated its 15th anniversary last year. This was noted in the journal AJOB Neuroscience through an article about the area’s current and future challenges.

In one of the published comments, three researchers from the Human Brain Project and CRB emphasize the importance of basic conceptual analysis in neuroethics. The new field of neuroethics is more than just a kind of ethical mediator between neuroscience and society. Neuroethics can and should contribute to the conceptual self-understanding of neuroscience, according to Arleen Salles, Kathinka Evers and Michele Farisco. Without such self-understanding, the ethical challenges become unclear, sometimes even imaginary.

Foundational conceptual analysis can sound stiff. However, if I understand the authors, it is just the opposite. Conceptual analysis is needed to make concepts agile, when habitual thinking made them stiff. One example is the habitual thinking that facts about the brain can be connected with moral concepts, so that, for example, brain research can explain to us what it “really” means to be morally responsible for our actions. Such habitual thinking about the role of the brain in human life may suggest purely imaginary ethical concerns about the expansion of neuroscience.

Another example the authors give is the external perspective on consciousness in neuroscience. Neuroscience does not approach consciousness from a first-person perspective, but from a third-person perspective. Neuroscience may need to be reminded of this and similar conceptual limitations, to better understand the models that one develops of the brain and human consciousness, and the conclusions that can be drawn from the models.

Conceptual neuroethics is needed to free concepts from intellectual deadlocks arising with the expansion of neuroscience. Thus, neuroethics can contribute to deepening the self-understanding of neuroscience as a science with both theoretical and practical dimensions. At least that is how I understand the spirit of the authors’ comment in AJOB Neuroscience.

Pär Segerdahl

Emerging Issues Task Force, International Neuroethics Society (2019) Neuroethics at 15: The Current and Future Environment for Neuroethics, AJOB Neuroscience, 10:3, 104-110, DOI: 10.1080/21507740.2019.1632958

Arleen Salles, Kathinka Evers & Michele Farisco (2019) The Need for a Conceptual Expansion of Neuroethics, AJOB Neuroscience, 10:3, 126-128, DOI: 10.1080/21507740.2019.1632972

This post in Swedish

We want solid foundations - the Ethics Blog


An extended concept of consciousness and an ethics of the whole brain

November 4, 2019

Pär SegerdahlWhen we visit a newly operated patient, we probably wonder: Has she regained consciousness? The question is important to us. If the answer is yes then she is among us, we can socialize. If the answer is negative then she is absent, it is not possible to socialize. We can only wait and hope that she returns to us.

Michele Farisco at CRB proposes in a new dissertation a more extensive concept of consciousness. According to this concept, we are conscious without interruption, basically, as long as the brain lives. This sounds controversial. It appears insensitive to the enormous importance it has for us in everyday life whether someone is conscious or not.

Maybe I should explain right away that it is not about changing our usual ways of speaking of consciousness. Rather, Michele Farisco suggests a new neuroscientific concept of consciousness. Science sometimes needs to use familiar words in unfamiliar ways. For example, biology cannot speak of humans and animals as an opposition, as we usually do. For biology, the human is one of the animals. Just as biology extends the concept of an animal to us humans, Michele Farisco extends the concept of consciousness to the entire living brain.

Why can an extended concept of consciousness be reasonable in neuroscience? A simple answer is that the brain continues to be active, even when in the ordinary sense we lose consciousness and the ability to socialize. The brain continues to interact with the signals from the body and from the environment. Neural processes that keep us alive continue, albeit in modified forms. The seemingly lifeless body in the hospital bed is a poor picture of the unconscious brain. It may be very active. In fact, some types of brain processes are extra prominent at rest, when the brain does not respond to external stimuli.

Additional factors support an extended neuroscientific concept of consciousness. One is that even when we are conscious in the usual sense, many brain processes happen unconsciously. These processes often do the same work that conscious processes do, or support conscious processes, or are shaped by conscious processes. When we look neuroscientifically at the brain, our black and white opposition between conscious and unconscious becomes difficult to discern. It may be more reasonable to speak of continuities, of levels of the same consciousness, which always is inherent in the living brain.

In short, neuroscience may gain from not adopting our ordinary concept of consciousness, which makes such an opposition between conscious and unconscious. The difference that is absolute when we visit a newly operated patient – is she conscious or not? – is not as black and white when we study the brain.

Does Michele Farisco propose that neuroscience should make no difference whatsoever between what we commonly call conscious and unconscious, between being present and absent? No, of course not. Neuroscience must continue to explore that difference. However, we can understand the difference as a modification of the same basic consciousness, of the same basic brain activity. Neuroscience needs to study differences without falling victim to a black and white opposition. Much like biology needs to study differences between humans and other animals, even when it extends the concept of an animal to the human.

The point, then, is that neuroscience needs to be open to both difference and continuity. Michele Farisco proposes a neuroscientific distinction between aware and unaware consciousness. It captures both aspects, the difference and the continuity.

Michele Farisco’s extended concept of consciousness also has ethical consequences. It can motivate an ethics of the whole brain, not just of the conscious brain, in the usual sense. The question is no longer, merely, whether the patient is conscious or not. The question is at what level the patient is conscious. We may need to consider ethically even unconscious brains and brain processes, in the ordinary sense. For example, by talking calmly near the patient, even though she does not seem to hear, or by playing music that the patient usually appreciates.

Perhaps we should not settle for waiting and hoping that the patient will return to us. The brain is already here. At several levels, this brain may continue to socialize, even though the patient does not seem to respond.

If you want to know more about Michele Farisco’s extended concept of consciousness and his ethics of the whole brain, read the dissertation that he recently defended. You can also read about new technological opportunities to communicate with patients suffering from severe disorders of consciousness, and about new opportunities to diagnose such disorders.

Pär Segerdahl

Farisco, Michele. 2019. Brain, consciousness and disorders of consciousness at the intersection of neuroscience and philosophy. (Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Medicine 1597.) Uppsala: Acta Universitatis Upsaliensis.

This post in Swedish

We challenge habits of thought : the Ethics Blog


Neuroethical reflection in the Human Brain Project

May 15, 2019

Arleen SallesThe emergence of several national level brain initiatives and the priority given to neuroscientific research make it important to examine the values underpinning the research, and to address the ethical, social, legal, philosophical, and regulatory issues that it raises.

Neuroscientific insights allow us to understand more about the human brain: about its dynamic nature and about its disorders. These insights also provide the basis for potentially manipulating the brain through neurotechnology and pharmacotherapy. Research in neuroscience thus raises multiple concerns: From questions about the ethical significance of natural and engineered neural circuitry, to the issue of how a biological model or a neuroscientific account of brain disease might impact individuals, communities, and societies at large. From how to protect human brain data to how to determine and guard against possible misuses of neuroscientific findings.

Furthermore, the development and applications of neuro-technology to alleviate symptoms or even enhance the human brain raise further concerns, such as their potential impact on the personality, agency, and autonomy of some users. Indeed, some empirical findings appear to even challenge long held conceptions about who we are, the capacity to choose freely, consciousness, and moral responsibility.

Neuroethics is the field of study devoted to examining these critical issues. Unfortunately, it has sometimes been reduced to a subfield of applied ethics understood as a merely procedural approach. However, in our understanding, neuroethics is methodologically much richer. It is concerned not just with using ethical theory to address normative issues about right and wrong, but notably with providing needed conceptual clarification of the relevant neuroscientific and philosophical notions. Only by having conceptual clarity about the challenges presented will we be able to address and adequately manage them.

So understood, neuroethics plays a key role in the Human Brain Project (HBP). The HBP is a European Community Flagship Project of Information and Computing Technologies (ICT). It proposes that to achieve a fuller understanding of the brain, it is necessary to integrate the massive volumes of both already available data and new data coming from labs around the world. Expected outcomes include the creation and operation of an ICT infrastructure for neuroscience and brain related research in medicine and computing. The goal is to achieve a multilevel understanding of the brain (from genes to cognition), its diseases and the effects of drugs (allowing early diagnoses and personalised treatments), and to capture the brain’s computational capabilities.

The HBP is funded by the European Commission in the framework of the EU’s Horizon 2020 research-funding programme. The programme promotes responsible research and innovation (RRI). RRI is generally understood as an interactive process that engages social actors, researchers, and innovators who must be mutually responsive and work towards the ethical permissibility of the relevant research and its products. The goal is to ensure that scientific progress and innovation are responsible and sustainable: that they increase individual and societal flourishing and maximize the common good.

To develop, broaden, and enhance RRI within the project, the HBP established the Ethics and Society subproject. Ethics and Society  is structured around a number of RRI activities such as foresight analysis (to identify at an early stage ethical and social concerns), citizens’ engagement (to promote involvement with different points of view and to strengthen public dialogue), and ethics support (to carry out research in applied ethics and to develop principles and mechanisms that ensure that ethical issues raised by research subprojects are communicated and managed and that HBP researchers comply with ethical codes and legal norms).

Neuroethical reflection plays a key role in this integration of social, scientific, and ethical inquiry. Notably, in the HBP such reflection includes conceptual and philosophical analysis. Insofar as it does, neuroethics aims to offer more than assistance to neuroscientists and social scientists in identifying the social, political, and cultural components of the research. Via conceptual analysis, neuroethics attempts to open a productive space within the HBP for examining the relevant issues, carrying out self-critical analysis, and providing the necessary background to examine potential impacts and issues raised. Neuroethical reflection in the HBP does not exclusively focus on ethical applications and normative guidance. Rather, it takes as a starting point the view that the full range of issues raised by neuroscience cannot be adequately dealt with without also focusing on the construction of knowledge, the meaning of the relevant notions, and the legitimacy of the various interpretations of relevant scientific findings.

At present, the importance of neuroethics is not in question. It is a key concern of the International Brain Initiative, and the different international brain projects are trying to integrate neuroethics into their research in different ways. What continues to be unique to neuroethics in the HBP, however, is its commitment to the idea that making progress in addressing the host of ethical, social, legal, regulatory and philosophical issues raised by brain research to a great extent depends on a conceptual neuroethical approach. It enables constructive critical alertness and a thought-out methodology that can achieve both substantial scientific ground and conceptual clarity.

If you want to read more, see below a list of publications on which this post is based.

Arleen Salles

Delegates eaGNS. Neuroethics Questions to Guide Ethical Research in the International Brain Initiatives. Neuron. 2018.

Evers K, Salles A, Farisco M. Theoretical Framing for Neuroethics: The Need for a Conceptual Aproach. In: Racine E, Aspler, J., editor. Debates About Neuroethics: Springer; 2017.

Salles A, Evers K. Social Neuroscience and Neuroethics: A Fruitful Synergy. In: Ibanez A, Sedeno, L., Garcia, A., editor. Social Neuroscience and Social Science: The Missing Link: Springer; 2017. p. 531-46.

Farisco M, Salles A, Evers K. Neuroethics: A Conceptual Approach. Camb Q Healthc Ethics. 2018;27(4):717-27.

Salles A, Evers K, Farisco M. Neuroethics and Philosophy in Responsible Research and Innovation: The Case of the Human Brain Project. Neuroethics. 2018.

Salles A, Bjaalie JG, Evers K, Farisco M, Fothergill BT, Guerrero M, et al. The Human Brain Project: Responsible Brain Research for the Benefit of Society. Neuron. 2019;101(3):380-4.


Neuroethics goes global

February 12, 2019

The complicated meaning, powerful assumptions, and boundless hopes about what can be revealed through neuroscience have made this discipline a national funding priority around the globe. A growing cohort of large-scale brain research initiatives aim to unravel the mysteries of the basis of feelings, thinking, and ultimately the mind. Questions formerly in the domain of the philosophical world have become part and parcel to neuroscience.

Just as science has so clearly become a global enterprise, ethics must keep pace. Cultural misunderstandings have nontrivial consequences for the scientific enterprise. Gaps in understanding negatively impact opportunities for collaboration and sharing, ultimately slowing scientific advancement. Too narrow of a view on science can limit our ability to reap the benefits of discoveries and, perhaps most damning for science, can result in a failure to anticipate and recognize the full consequences and risks of research.

To date, neuroethics discussions have been dominated by Western influences. However, the rapid neuroscientific development in East Asia in particular and the not-so-gradual relocation of a number of cutting-edge research projects from the West to East Asia, has made it clear that exploration and understanding of the ethics and cultural values informing research will be critical in engaging science as a collaborative global enterprise.

The Neuroethics Workgroup of the International Brain Initiative is comprised of members of each of the existing and emerging large-scale brain research initiatives. Leveraging the fellowship of the IBI and using an intentional culturally aware approach to guide its work, the Neuroethics Workgroup completes rapid deliverables in the near (within one year) and short-term (within two years).

With the inaugural 2017 summit, leading scientists, ethicists, and humanist co-created a universal list of neuroethics questions, Neuroethics Questions for Neuroscientists (NeQN) that should be addressed by scientists in each brain project. These NeQN were published in Neuron in 2018 and can be found here.

The neuroethics questions themselves were not necessarily unfamiliar neuroethics questions; however, these NEQN were designed to be adapted and informed by the cultural values and frameworks of each country.

The 2018 meeting served as a workshop, where each of the brain projects discussed why and how they will integrate neuroethics into their brain projects with particular recognition of the five questions from the 2018 Neuroethics Questions for Neuroscientists (NeQN) featured in Neuron. The product is the first neuroethics special issue in a high impact neuroscience journal.

Each perspective offers topics and context for their engagement with and practice of neuroethics. The issue features the seven existing and emerging large-scale brain research projects organized in alphabetical order.

The Australian Brain Alliance describes how neuroethics has been integrated into their research ethos as featured in their public outreach and advocacy efforts as well as their explorations in the public domains such as neurolaw and industry. A key component for the Australian project is diversity and inclusion, and there is a particular interest in engaging brain health with vulnerable Indigenous populations in Australia.

The Canadian Brain Research Strategy paper illustrates the rich historical efforts in pioneering neuroethics and future plans of a national collaboration to carefully consider public discourse and patient engagement as they pursue deeper knowledge of the how the brain learns, remembers, and adapts. A fundamental recognition of the neuroethics backbone of the Canadian project is that “The powerful ability of the brain to change or rewire itself in response to experience is the foundation of human identity.”

The China Brain Project discusses potential models for important public outreach campaigns and the balance of considering traditional Chinese culture and philosophy, particularly in the areas of brain death, conceptualizations of personhood and individual rights, and stigma for mental illness. The authors describe commitments for integrating neuroethics as the China Brain Project is being designed.

The EU Human Brain Project outlines its bold leadership and addresses the conceptual and philosophical issues of neuroethics and the implementation of philosophical insights as an iterative process for neuroscience research. A project with an extremely sophisticated neuroethics infrastructure, this paper provides examples of managing issues related to the moral status of engineered entities, how interventions could impact autonomy and agency, and dual use.

The Japan Brain/MINDS paper describes plans to reinvigorate historical efforts in neuroethics leadership as it expands the scope of its research and launches Japan Brain/MINDS Beyond. In particular, the project will integrate neuroethics to address issues related to privacy and data collection as well as in considering stigma and biological models of psychiatric disease.

The Korea Brain Initiative paper nicely demonstrates how advocacy for neuroscience and neuroethics at the government and policy levels go hand in hand. As Korea aims to advance its neuroscience community, the Korean government has seen neuroethics as integral to neuroscientists’ development. The Korea Brain Initiative is exploring ethical issues related to “intelligent” brain technologies, brain banking, cognitive enhancement, and neural privacy in the milieu of traditional and contemporary cultural traditions in Korea.

The US BRAIN Initiative outlines its efforts in building an infrastructure for neuroethics in research and policy and for funding research as it plans its roadmap for the next phase of BRAIN to 2025. Example of ethical issues that arise from the project’s goals of understanding neural circuitry include the moral relevance and status of ex vivo brain tissue and organoids as well as unique ethical concerns around informed consent in brain recording and stimulation in humans.

Each project illustrates that neuroethics is important regardless of the scope and methodologies inherent in its research goals and demonstrates the utility of the NeQNs for today’s and future scientists within and beyond the large-scale neuroscience research projects.

Karen Rommelfanger

PhD, Director, Neuroethics Program Emory Center for Ethics, Co-chair International Brain Initiative Neuroethics Workgroup


Drug addiction as a mental and social disorder

December 4, 2018

Michele FariscoCan the brain sciences help us to better understand and handle urgent social problems like drug addiction? Can they even help us understand how social disorder creates disorderly, addicted brains?

If, as seems to be the case, addiction has a strong cerebral base, then it follows that knowing the brain is the key to finding effective treatments for addiction. Yet, what aspects of the brain should be particularly investigated? In a recent article, co-authored with the philosopher Kathinka Evers and the neuroscientist Jean-Pierre Changeux, I suggest that we need to focus on both aware and unaware processes in the brain, trying to figure out how these are affected by environmental influences, and how they eventually affect individual behavior.

There is no doubt that drug addiction is one of the most urgent emergencies in contemporary society. Think, for instance, of the opioid crisis in the US. It has become a kind of social plague, affecting millions of people. How was that possible? What are the causes of such a disaster? Of course, several factors contributed to the present crisis. We suggest, however, that certain external factors influenced brain processes on an unaware level, inviting addictive behavior.

To give an example, one of the causes of the opioid crisis seems to be the false assumption that opioid drugs do not cause addiction. Taking this view of opioid drugs was an unfortunate choice, we argue, likely favored by the financial interests of pharmaceutical companies. It affected not only physicians’ aware opinions, but also their unaware views on opioid drugs, and eventually their inclination to prescribe them. But that is not all. Since there is a general disposition to trust medical doctors’ opinions and choices, the original false assumption that opioid drugs do not cause addiction spread and affected also public opinion, especially at the unaware level. In other words, we think that there is a social responsibility for the increase in drug addiction, if not in ethical terms, at least in terms of public policies.

This is just an example of how external factors contribute to a personal disposition to use potentially addictive drugs. Of course, the factors involved in creating addiction are multifarious and not limited to false views about the risk of addiction associated with certain drugs.

More generally, we argue that in addition to the internal bases of addiction in the central nervous system, socio-economic status modulates, through unaware processing, what can be described as a person’s subjective “global well-being,” raising in some individuals the need for additional rewards in the brain. In the light of the impact of external factors, we argue that some people are particularly vulnerable to the pressures of the political and socio-economical capitalistic system, and that this stressful condition, which has both aware and unaware components, is one of the main causes of addiction. For this reason, we conclude that addiction is not only a medical and mental disorder, but also a social disorder.

Michele Farisco

Farisco M, Evers K and Changeux J-P (2018) Drug Addiction: From Neuroscience to Ethics. Front. Psychiatry 9:595. doi: 10.3389/fpsyt.2018.00595


%d bloggers like this: