© Furhat Robotics AB

FRAnny the concierge

Artificial intelligence with a smile

04/2018 – At Frankfurt Airport, you can step into the future today. DB Systel and Fraport have developed an electronic concierge using a robotic head from Furhat Robotics. During test operation, it answers passengers' questions – and not just by voice.

The robot gives a friendly smile as she tells the passenger which gate he has to go to. The scene seems futuristic, almost as if the robot had walked out of a sci-fi movie. But FRAnny is quite real and, standing behind her desk at Frankfurt Airport, is impressive testimony to what is now technologically feasible. The socially intelligent robot and its interaction platform were developed by Stockholm-based Furhat Robotics. But the artificial intelligence behind FRAnny comes from DB Systel and was created as a pilot project of the SEMMI team (socio-empathetic human-machine interaction) from the company’s own Skydeck Accelerator programme for innovations. The name FRAnny refers to the location: Frankfurt Airport has the international code FRA.

Dr. Kais Samkari, software architect at DB Systel, who programmed what you might call FRAnny’s brain, says: “Normally, you build systems that the user has to learn how to work with. We wanted to take a different approach.”  Samkari believes it makes more sense if people can talk to the system, and it understands what the user wants. The only problem is that until recently this was not possible, because neither the AI nor the required computing resources were available. “But over the past three years, AI has developed very rapidly. That’s why we wanted to create a machine that people can talk to freely,” says Samkari.

Pilot project for higher quality service

It was this idea of the SEMMI team that gave rise to the digital concierge for travel information. This human-computer interface consists of a moveable head with chest and shoulders. FRAnny’s face and facial expressions are projected onto a plastic mask, enabling her to take on different characters and adapt to any situation. Thanks to an integrated camera, the robot starts to interact as soon as someone enters her interaction radius, she recognises the person she is talking to and maintains eye contact. A built-in microphone supports voice input and voice recognition, and FRAnny’s body contains a loudspeaker.

Staff from Fraport AG, the company that operates Frankfurt Airport, attended a presentation by the SEMMI team. At that time, both parties were looking into robotics and voice-activated assistants. It soon became clear that DB Systel and Fraport shared similar ideas and expectations of how artificial intelligence and robotics can help improve service quality. So it was only logical that they decided to join forces for a joint test phase. While Fraport handles the design of functions and content, DB Systel is responsible for technical implementation in the joint project.

More than just voice

With voice control, the machine generally helps to make the underlying system unrecognisable to users. Computing operations are usually no longer performed on the individual device but are handled using computing power from the network. But SEMMI aims to use much more than just voice; after all, people also communicate using looks and gestures. “We wanted a combination of voice control and facial expressions that would allow us to offer a human-like combination,” says Kais Samkari. “And the interaction platform from Furhat Robotics makes this possible.” However, this is still a one-way street. The concierge can underscore her words with specific facial expressions, but she currently understands only the voice of the person interacting with her. While emotion recognition is possible, the focus here must be squarely on the framework, user acceptance and added value. If the test at the airport proves successful, the next step could be for FRAnny to also speak Spanish or Japanese, or to recognise the facial expressions of the person talking to her, for example.

Video: Interview with the project partners
Credit: DB Systel GmbH

For now, the most important question is whether people will actually want to talk to a machine. Even if many people already talk to “assistants” like Siri or Alexa, FRAnny is a system that is installed in a public place. “This solution certainly won’t appeal to all passengers,” says Christan Bartel from Fraport AG, who, together with the SEMMI team from DB Systel and his Fraport colleagues, developed the test scenario. “We want to find out whether and how a robot can integrate into the airline passenger environment.

One question, two methods

The robot not only supplements existing information media, but also provides support for staff. The ultimate goal is to increase service quality. To achieve that, the system first has to learn. This involves two approaches. On the one hand, an undefined number of sentences are entered and trained as a basis, using machine learning, with other sentence being added over time. On the other, natural language processing is deployed. Here, grammars (for example, linguistic patterns indicating a question) are used for voice recognition. With FRAnny, this works as follows: she is given a voice signal, which is converted into text. She may then recognise that the person talking to her is looking for a gate. The same text is also processed by the machine learning component. “We still don’t know which approach is the right one. That’s why we’re using both in the proof of concept so that they can complement each other,” says Kais Samkari. “Afterwards, we’ll look at the cases where there was no match and further train both system accordingly.”

Video: FRAnny in action at the airport
© DB Systel GmbH

The dialogues containing a basic corpus of information were created jointly by DB Systel and Fraport. And thanks to the integration of Google’s highly accurate voice recognition solution, the system is able to understand English even during the testing phase. The test will initially run for four weeks in the airport’s international transit area. FRAnny is also being supported by real people, who observe and log the interactions and conduct user surveys. In addition, the interactions will be exported from the system and evaluated. “Among other things, this shows us whether there are specific target groups who are more sympathetic to the robot, and if so, which,” explains Christian Bartel. “Or perhaps this approach will enable us to find a version that almost all users accept.”

A model with a future

Above all, the test is intended to develop expertise. One potential goal would then be to offer a “Robot as a Service”. Once it has been equipped with the intelligence and taught the appropriate business expertise for the specific purpose, “we deliver the machine and make sure that even employees with little IT experience can set it up correctly. In addition, we ensure interaction with the system and maintenance of the robot go as smoothly as possible,” says Kais Samkari. And all this for a monthly charge, just like for a mobile phone –  another technology that was science fiction not so long ago.

Other articles on the featured topic of artificial intelligence: