As AI learns, it evolves and changes its output. In this sense, it differs radically from conventional medical devices and pharmaceuticals, which are required to deliver stable performance and effects, so the ethical issues it entails are complex. In clinical practice, physicians must take the greatest care to avoid excessive reliance on AI in supporting clinical decisions, lest they lose sight of their essential mission as physicians. AI also raises many issues for those receiving medical care, including the handling of personal information and the role of patients as users. Now is the time to discuss the kind of medical care we should aim to deliver by leveraging new technologies.

The social impact of some new scientific technologies is without precedent. The progressive introduction of AI to the field of medical care has led to discussion of its ethical issues. This is because, in medicine in particular, AI usage is developing in a different way from other fields, as it involves two categories of users: physicians and patients (i.e., the general public).

Ethical issues at every stage

While those who directly use AI-based tools in medical care are probably physicians, decisions about the suitability of a course of treatment —— such as deciding on a treatment method and prescribing drugs —— are not solely based on the information that physicians have about patients, but also take into account the patient’s preferences and lifestyle. Consequently, in undertaking such discussions, we need to keep in mind that these are tools used by people with different standpoints, namely physicians and patients. While respecting each patient’s views on life, death, and the body, physicians, patients, and society as a whole need to engage in ongoing discussion of the extent to which the new attributes introduced to medical care by AI should be tolerated, and how AI should be positioned in its use (Table).

The ethical issues that should be considered in regard to the introduction of AI are many and varied.

First, let us look at research and development settings. The development of AI requires a huge volume of data. This data is collected from individual medical institutions. However, the details entered in medical records are sensitive health records that affect an individual’s privacy, so they need to be managed rigorously in clinical practice. Fundamentally, there is a relationship of trust between physician and patient, in which the patient can fully entrust their personal health data to the physician. It is precisely because of this trust that patients can speak candidly about their suffering. Would patients really feel able to speak candidly to their physician if they did not know who might access their data? A patient might feel anxious that their medical information, which may offer no direct benefit to them, might be used without their knowledge for the future development of medical care.

However, the world is already heading in the direction of using individuals’ medical information as a key resource for research and development, in order to leverage AI for better medical care, and such moves will accelerate further. Even in Japan, development efforts using personal information are likely to intensify; when this occurs, the critical point in terms of moving forward with development will be how to secure the understanding of society. Several countries have already seen numerous situations in which government-led research and development initiatives have been discontinued due to insufficient explanation to the public regarding the use of personal information. I believe we must communicate with sincerity to the public about the fact that using patient medical records in research and development will be crucial to improving medical care.

How might AI reshape the conventional role of physicians?

Approaches to academic-industrial collaboration in medical care also need to be considered in the context of research and development. Even in Japan, there have been instances where academic-industrial collaboration in drug development was conducted without appropriate management of conflicts of interest. Taking this situation very seriously, the government tightened regulations on clinical research, on the basis that merely expecting researchers to demonstrate an ethical perspective was insufficient. However, compared with the development of pharmaceuticals and medical devices, individuals from industries that have not previously engaged with medical care are more likely to be involved in research focused on the development of AI, particularly in app development. I believe the question of how to move forward with activities within the context of academic-industrial collaboration in a transparent manner is crucial.

Next, there is the question of how AI-based tools are positioned when used in clinical practice.

A government report has drawn attention by suggesting that one way to leverage AI in medical care would be the use of AI-based medical tools to assist local physicians in regions lacking pathologists, for example. This is certainly a meaningful way of using AI, but on the other hand, there are a number of foreseeable hurdles, such as the fact that local physicians would be entrusting matters beyond their control to AI, while still being required to take professional responsibility for the diagnosis. This is because the Medical Practitioners’ Act imposes an obligation to provide a direct diagnosis. In 2018, the Ministry of Health, Labour and Welfare stated that “when providing medical care using artificial intelligence-based programs to support diagnosis and treatment, the physician remains accountable”; in other words, responsibility for the final judgment lies with the physician. If this view were interpreted narrowly and adhered to rigidly, it could discourage physicians from using AI. Another conceivable future direction would involve reviewing the principles and limitations of the fulfillment of the physician’s duty to reach a direct diagnosis, while protecting the institutional role of physicians and their expertise.

The role of physicians in using AI tools while maintaining their autonomy has been actively debated by the American Medical Association (AMA).

In the AMA’s view, the purpose of using AI in medical care is not to develop tools that mechanize medical care and replace physicians, but rather to enhance the quality of the essential duties performed by physicians by compensating for those areas where labor shortages affect those duties. The objective of using AI is not to pursue the complete replacement of physicians, nor should this be actively pursued. Recognizing the problem that physicians’ autonomous judgment could potentially be distorted, the AMA has, since 2018, argued that AI should be termed “augmented intelligence.” This, too, would seem to stem from the basic idea that AI is not intended to replace physicians, but rather is simply a tool to complement their expertise. The AMA Code of Medical Ethics states that when, for example, a physician’s results differ from those of an AI, the physician must verify that the transparency of AI-based decision-support tools has been adequately ensured, and should ultimately reach a conclusion that prioritizes the viewpoints of physicians and patients alike.

• *1 Transparency of AI-based decision-support tools: Clarity concerning what kind of data the AI was trained on and how it reached its conclusions.

Respecting shared professional judgment

While the U.S. has a positive attitude toward the use of AI, it was noteworthy that the AMA pointed out the need to ensure that physicians’ professionalism is not undermined by over-reliance on AI or other clinical decision-support tools. Based on an awareness of issues relating to a physician’s discretion, the AMA has also recommended that physicians should respect the clinical judgment and the accepted standards of professional practice shared among medical professionals, rather than being swayed solely by the results generated by AI. In a sense, this is the overriding principle of evidence-based medicine (EBM).

• *2  Evidence-based medicine (EBM): An approach to medical decision-making that prioritizes scientific evidence to provide patients with the optimal treatment.

Rather than prioritizing evidence, the idea underpinning EBM is that evidence should be carefully evaluated in light of physicians’ collective professional judgment. In other words, this means that the principles of EBM have been applied to AI-based decision-support tools as well. The AMA also stresses that, in the current situation, where the utility of AI has not been fully established due to insufficient clinical validation, physicians should not be forced or obliged to use such tools. This situation in the U.S. also offers valuable insights for Japan, where AI is likely to proliferate going forward.

The AMA’s opinion could also change. Should the rollout of AI in medical care be endorsed, even if it has aspects that remain somewhat opaque? To what extent should measures that gain public acceptance be taken? The premise that physicians are held accountable for the use of AI might eventually reach its limits. I believe it is essential to engage in widespread debate about how physicians and patients can collectively leverage AI-based tools.

As I stated at the outset, physicians are not the only users of AI. The general public, too, will likely use the AI-based apps available to engage in various activities relating to medical care. There could conceivably be a variety of developments in this realm, too. For example, there is the question of who holds what kind of responsibility in the event that use of a foreign app results in harm to someone’s health. Another issue is whether the concept of personal responsibility becomes excessive if the use of apps becomes a fundamental premise in healthcare and related areas.

The U.S. and the U.K. are often cited when considering future approaches to medical care in Japan. As each individual medical institution in the U.S. is a management entity, deregulation progresses, and the market brings about changes in approaches to medical care. This differs quite considerably from Japan’s public healthcare system, but if Japan were to move toward an American-style medical care system in the future, the pressure exerted on physicians’ diagnoses by AI and other such phenomena occurring in the U.S. would likely be a useful point of reference.

On the other hand, the U.K. is a country with a more comprehensive national healthcare system than Japan’s. Consequently, the emphasis is placed on how to sustain this system. Currently, the U.K. is leaning toward recommending that patients use apps to ascertain how to engage with medical care and manage their diseases. As the U.K.’s National Health Service faces severe financial constraints, the advice is for people to do what they can for themselves, in an effort to reduce the number of people visiting medical institutions. This is called prehospital triage; in the context of answering the question “Is your reason for going to hospital really legitimate?” it is recommended that people first use an app to check. However, there are concerns that if using such apps to manage one’s health becomes standardized, the idea that the inability to manage one’s own health is a person’s own responsibility will become excessively prevalent.

Civil society must recognize that it is undesirable for AI-based tools to impose undue pressure on patients and physicians.

How should we address the flexibility and plasticity inherent in AI learning?

Once again, I would note that a major topic when discussing the ethics of AI is AI’s characteristic ability to evolve.

To begin with, when discussing devices that use AI, there is the issue that there are a variety of perspectives on AI, particularly in terms of what distinguishes augmented intelligence from artificial intelligence. Opinions are divided: some view programs with AI-learned results as AI, while others regard entities that learn and evolve as AI; however, one could say that if something does not have a learning function, it is not AI, but rather simply software.

Although differences in definitions of AI do complicate the debate, if we think of AI as something that learns and is constantly transforming, it introduces a new element into the use of AI in medical care. This is because the fundamental premise of conventional medical devices and pharmaceuticals is that they are consistently manufactured in the same form in which their performance and effects were evaluated, and that the identical items are distributed in society. However, if AI learns and becomes more accurate, it may diverge from what was initially approved. It would be difficult to say that the government had evaluated that device, and if it were permitted, it would directly conflict with the existing framework of pharmaceutical regulation.

Even if such evolution were tolerated, the results presented by the AI would differ depending on the kind of data it was trained on. For example, let us assume that a hospital treating many patients with a particular lifestyle and adjusts the data to tailor it to those patients and modifies the software. From another perspective, customizing training data with a particular patient group in mind reduces effectiveness for individuals outside that group. This gives rise to the question of whether the hospital should prioritize usage optimized for specific patients or aim to ensure that the tool can be used in a wide variety of people, even if performance is somewhat reduced.

There is lively debate around such issues as whether the government should carry out regular assessments of approved medical devices, but this is a complex issue, as it could significantly impact the administration of medical care. In order to harness the multifaceted potential of AI and maximize the benefits for the public, the best approach is to engage the public in broad deliberations on the ideal form of medical care.

When AI is used in medical care, there will be a variety of new developments at all levels. However, when using AI, we must ensure that we do not lose sight of the most basic principle of what constitutes the evolution of medical care, and that we do not allow ourselves to be swayed by the novelty of technology.

I had the opportunity to visit an “AI hospital” in the U.K. While I had envisioned observing advanced applications of AI, I was surprised when I heard that AI was used in the system for managing patient appointments. However, this was because my own outlook was too narrow. While it is, of course, crucial to explore advanced therapies via the introduction of AI, improving the routine patient experience, which many people often find inconvenient or burdensome, is another realm where AI can make a substantial contribution. This is a topic that merits deeper consideration in Japan than it has received overseas. I believe that another vital role of medical ethics is to point out the increasingly diverse needs that should be addressed to improve medical care as a whole, rather than focusing solely on the development of advanced technology.