The term “human augmentation” tends to call to mind images of the kind of devices you find in science fiction that allow people to exhibit superhuman abilities, like a robot suit that enables you to fly through the sky or gives you the strength to lift objects that would normally be too heavy for a person to carry. However, human augmentation is already supporting our work and our daily lives; for example, at building sites, remotely controlled construction machinery is lessening the burden on workers. In rehabilitation settings, too, assistive exercise equipment based on artificial muscles are reportedly now enabling people to bring out their innate abilities.

Human augmentation technology aims to enhance the abilities that humans have. It does not simply refer to wearing a device in order to be able to carry extremely heavy objects, or to place your life in the hands of a machine so that you can travel through the air. A diverse array of mechanisms are being developed alongside advanced engineering-based technologies, including research in such fields as psychology.

The ability to carry out work without being physically present

Professor Yuichi Kurita of Hiroshima University’s Graduate School of Advanced Science and Engineering is working on the social implementation of a variety of technologies. In partnership with Hiroshima-headquartered company Kobelco Construction Machinery, Kurita developed a system for the remote control of construction machinery. We asked Kurita about the human augmentation technology that is actually in use or progressing toward commercialization.

“There are a number of factors contributing to the need for human augmentation technology at sites where construction machinery is used,” he says. “The first is an acute shortage of workers. Construction sites are highly diverse, and the degree of difficulty involved in operating construction machinery also varies. For example, in the case of leveling slopes, which is considered to be the most difficult task, several years of training are needed to reach the particularly high level of skills proficiency required. However, it’s proving hard to attract young people to work on construction sites. Accordingly, there are strong calls to develop an environment that enables tasks to be carried out without being physically present at a site, as a means of enabling older, retired operators to keep working.”

This remote-controlled construction machinery system is also being used in the vehicle dismantling industry, among others.

“The work at dismantling sites includes tasks carried out in intense heat, as high temperatures are required to melt down the metal. Some tasks are also dangerous, involving the use of a grapple (a device like a big pair of scissors, for cutting metal),” Kurita explains. “As such, there was demand for a system that could be operated remotely.”

Kobelco Construction Machinery’s collaborative research laboratory at Hiroshima University has a hydraulic excavator simulator equipped with the remote control system (Figure 1).

Figure 1. Remote-controlled construction machinery①Simulator operation environment.
② Image of the site actually shown on the display.
③The same lever is used on the actual construction machinery.

The lever fitted to the remote-controlled construction machinery is the same as that used on actual construction machinery. When the operator manipulates the lever, an image of the site is shown on the monitor in front of them (computer graphics are used on the simulator). To replicate the sensations the operator would experience if working at a real-life construction site, the movements of a hydraulic excavator are reproduced with the aid of a computer in the remote control system. The images when operating the remote-controlled construction machinery are reportedly virtually the same as those seen when actually working at a site. Thus, you can operate the remote-control machinery smoothly.

Sounds and vibration at the site are also conveyed to the operator

“In fact, there is hardly any haptic feedback from the lever on construction machinery at a real-life site,” Kurita says. “When sitting in the control position, the operator can hear a variety of noises and feel the engine vibration. In the remote-controlled construction machinery system, the seat has been equipped with a converter called a motion platform, which feeds back vibration. This makes the seat move in response to the construction machinery’s movements, so it gives the operator the sensation of their body tilting when working on sloped ground, for example.”

However, it is not necessarily appropriate simply to replicate exactly the kind of movements and sounds likely to occur in reality.

“One thing that could be described as a key point in human augmentation research: not all of the sounds on a building site increase the efficiency of operation or are required in order to detect danger. There’s a lot of unnecessary noise, too,” Kurita points out. “In other words, the vibrations of the real engine and the surrounding noises are not all necessarily required for operation. In fact, providing information without the extraneous and even uncomfortable elements should actually increase the operator’s efficiency. We can also feed back information that only a seasoned operator would normally be able to perceive in a manner that’s easily comprehensible even for novice users.”

In real construction machinery operated on-site with a human in the cab, it is undesirable to install more instruments —— that is to say, sensors and monitors —— than necessary, as they can hinder operation. However, various instruments can be installed in remotely operated devices. If remote-controlled construction machinery displayed information such as “Machine off-balance” on a monitor, even an operator with little experience would be able to ascertain the situation around the machine. Dangerous incidents that could result in a major accident are colloquially referred to as “near-miss events.” Various measures have been put in place to avoid these.

“Using remote-controlled construction machinery will enable us to reduce near-miss events while operators are still lacking in experience, and to build up their training safely,” Kurita says. “Another feature of remote-controlled construction machinery is that we can customize it according to the operator’s proficiency level. As they build up experience, they might find the information displayed on the monitor to be a nuisance. If that happens, we can use the software to switch it off. Providing functions tailored to the user like this is another concept in human augmentation.”

The ability to operate construction machinery remotely could have advantages in terms of work styles, too, as it would eliminate the burden of having to physically travel to each site. Instead, an operator could work on a site up in Hokkaido in the morning and down in Kyushu in the afternoon. It would also make it easier to accommodate needs for limited working hours per day. In addition to compensating for labor shortages and ensuring safety, Kurita also aims to build work environments in which there are advantages to using remote-controlled construction machinery over being physically present at a site.

Technologies that reinforce muscle strength are one widely used form of human augmentation technology.

The most commonly used type at present are robot suits, but they present a number of issues, including being difficult to put on without assistance, being made of a hard material with which collisions are painful, and only working with certain muscles.

“In our laboratory, we developed an assistive exercise device using an artificial muscle,” Kurita explains. “It’s very soft, and it’s easy to put on, too. The artificial muscle takes the form of a rubber tube that contracts when compressed air enters it. Joints move when muscles contract; this mechanism assists the movements of muscles by supporting the force of the contraction in this process. While it does not provide as much reinforcement as a robot suit, neither does it impede the wearer’s movements. This means it gives the wearer the sensation of moving their body as they choose.” (Figure 2)

Figure 2. Assistive devices①This device can also be easily customized by altering the position of the tube according to the muscle you wish to support.
②An assistive boot that supports ankle movements in response to the wearer’s gait.

Awareness of moving one’s own body

When people walk, they perform movements known as ankle dorsiflexion (bending toward the instep) and plantarflexion (bending downward), like kicking off from the ground. However, if muscle strength in this area weakens, the ankle becomes unstable, leading to falls. Accordingly, people undergoing rehabilitation wear this assistive device behind the ankle to reinforce the strength of the kick. An important feature of this assistive device is that minor adjustments can be made according to the abilities of the user.

The strength and position of the device are altered as the user’s walking ability improves, with the ultimate aim that they will recover their prior ability to walk without the device. This is because the objective of human augmentation technology is not only to reinforce the wearer’s abilities, but to augment human abilities themselves.

In nursing care and rehabilitation settings, maintaining the care recipient’s motivation is a challenge.

“Even though exercise is almost guaranteed to result in improvements if continued, people just can’t seem to keep it up,” Kurita points out. “This is because people have to get through a period where it feels like a laborious task for little reward. They also worry about falling and hurting themselves during the rehabilitation period, and about being a bother to those caring for them. People need to be mentally prepared for this, but unfortunately, the reality is that some give up. That’s why I think that if we could use human augmentation technology to reduce the struggle involved, more people would become able to walk unaided again.”

Rehabilitation inevitably tends to be left up to the patient, but human augmentation technology offers the idea that it might be possible to leverage the power of technology to assist in removing the daunting elements of the rehabilitation process, rather than simply branding patients who cannot continue with rehabilitation as lacking motivation.

Another feature of human augmentation technology is the importance attached to perception and sensation.

“When we move, we need to have a corresponding sensation that it’s we ourselves who are moving our bodies, rather than simply that our muscles are moving,” Kurita stresses. “If a person feels as though their body is being moved without their involvement, they’ll stop trying to move by themselves once they take this device off. Those who provide the device must not forget to take into consideration the sensation of actually using one’s own body and one’s own strength to perform the movement. I believe human augmentation is a service for people. When all’s said and done, the agency lies with humans, and it’s a research domain that seeks to expand what humans can do.”

One thing that has been developed with a focus on human psychology is an exercise support app designed to help people keep taking moderate exercise on their own (Figure 3).

Figure 3. Exercise support appThe user copies the instructor’s movements and the app displays messages, such as “GREAT” if they do it well.

“Many of those with pre-frailty are elderly, and they’re encouraged to exercise to prevent developing frailty,” he says. “While it’s certainly true that even being a little more active in daily life can help them to be healthier, not everyone is proactive about exercise. There are obvious reasons why people’s enthusiasm might be diminished, even when told they should exercise, including not knowing what to do and being uncertain as to whether they’re doing it right. However, we know that people’s motivation grows simply from having someone beside them to tell them what to do and praise them when they get it right. That’s why we developed a smartphone app.”

• * Frailty: A state of diminished mental and physical vitality that renders individuals more prone to requiring nursing care.

An environment equipped with an AI-based personal trainer

The user opens the app, sits with the screen facing them, and starts the workout. First of all, the screen shows the instructor demonstrating the exercise. Next, the user performs the movements themselves, following the lines displayed on the screen. The app picks up the user’s movements and responds with an evaluation, such as “Great,” “Good,” or “Bad,” which means that the user can concentrate, and it creates the feeling of actually having someone else there in the room. Features such as recording of the user’s score, which shows how much of an effect the workout has had, also seem likely to help maintain the user’s motivation.

With the assistance of physical therapists, Kurita and his team have created 52 different exercises, and their research has found that exercising with the app twice a day improved body movements in elderly people.

“The idea behind the system probably isn’t new, but previous systems weren’t easy for individuals to use, as they required elaborate equipment, including numerous cameras installed in the user’s room to capture the movements of their joints, and a motion capture system with lots of sensors worn on the user’s body,” Kurita says. “However, thanks to AI technology, we can now record the movements of people’s joints with the aid only of a two-dimensional image camera on a smartphone. In addition, the app saves the user’s score for each exercise, so they can obtain information in real time about such matters as whether their exercise capacity is improving or deteriorating. It isn’t merely an exercise app; we also envisage its being used as an app that enables people to monitor elderly family members.”

Even those who are active when they visit a seniors’ day care center or who do exercise at a facility with a physical therapist present might find they lack motivation once back home on their own. And just as in the case of the aforementioned construction machinery operators, personnel shortages mean that environments in which physical therapists are present to assist with rehabilitation are becoming like gold dust, especially in the countryside and other disadvantaged areas.

“I want to use the latest technology like this to provide an environment equipped with what you might call an AI personal trainer,” he explains. “We’ll have it evaluate exercises and encourage the user, telling them they’re making a good effort. It will share data with medical institutions and urge the user to seek medical attention if needed. I want to be able to provide a service tailored to the individual, rather than having them exercise in an aimless way. And I want to ensure that they don’t lose their innate potential just because they haven’t been exercising. That might mean people can continue working longer into old age. This is exactly what I think human augmentation is all about.”

Kurita is developing various devices that use human augmentation technology and can be used in industry; what they all have in common is that their objective is not simply to improve productivity.

“If achieving results is the sole purpose, one should simply think about a system that automates everything and eliminates the need for humans to do things,” he says. “It’s true that there are settings where that’s the more suitable approach, but I think ensuring that the result of increasing what humans can do is not merely enhanced performance, but also subjective comfort on the part of the individual.”

According to Kurita, human augmentation technology should achieve three key goals: making things easier, helping people do things better, and making things more enjoyable.

“Even if something becomes easier, it doesn’t necessarily mean people become better at it or enjoy it more. In that situation, we can’t say that people’s abilities have increased,” he says.

The emergence of innovative technologies —— AI being top of the list —— has sparked questions about the way in which they interact with humans. The concept of human augmentation might offer one pointer for exploring how we can harness advanced technologies of this kind for the purpose of human wellbeing.