© DB Systel GmbH/AdobeStock blackzheep

Mobile AR

Smartphones as windows on augmented reality

11/2018 – In a DB Systel test project, a smartphone camera shows much more than just reality. The "Companion" AR app is designed to supplement the device's real field of view with all the relevant information and data – superimposed on objects.

Whether on station platforms, on trains or during maintenance and servicing tasks, questions continually arise. For example, passengers want to know where there are free seats on the train, how to get to the right compartment and where to find the on-board restaurant. Group employees also constantly face new challenges. For example, maintenance technicians may want to know when a specific part was last repaired. Sometimes they may need a colleague’s advice, have to find technical documentation or consult a manual. Most of this information is available somewhere or other, but not at the location where it is needed.

The Immersive Technology team at DB Systel has been working for some time to make information digitally tangible. “In the area of virtual reality, for example, we’re already well positioned with EVE learning applications,” says Johannes Hillebrenner, Product Owner at DB Systel. “We’ve recognised that immersive technology also has vast potential above and beyond this. Via the Skydeck Accelerator, we’re supporting another team with Companion to develop additional expertise in mobile AR solutions.”

A new approach to information

In cooperation with Swedish railways, the team is working in the Skydeck Accelerator on a proof of concept (PoC) that will test the feasibility of applications for passengers and DB employees. “Information from existing systems can be linked to a 3D map of the world called the AR Cloud,” explains Arne Kupetz, also Product Owner at DB Systel and jointly responsible for the project with Johannes Hillebrenner. Data records from very different sources flow into this AR cloud. They include not only IoT data from sensors, but also object descriptions or points of interest. The cloud consolidates this data as needed and makes it available for AR applications, enriching objects with real-time information. To receive this additional information, all a user needs is a smartphone or tablet with a camera and Internet connection. “AR glasses are still very expensive, but almost all employees and passengers have a smartphone or tablet. That’s why the team focused on mobile solutions in this pilot project,” says Johannes Hillebrenner.

© DB Systel GmbH

For example, we want to use an app to provide consumers with relevant travel information where it is needed without time-consuming searches – simply by pointing their device at the empty track.

Arne Kupetz, DB Systel

Ideally, users will no longer have to search for the required information. It will be enough to simply point the mobile end-user device at the object. “For example, we want to use an app to provide consumers with relevant travel information where it is needed without time-consuming searches – simply by pointing their device at the empty track,” says Arne Kupetz. Up to now, they have had to click their way through different pages to get all the information they needed.

A hologram of the train and relevant information are displayed directly in the camera image of the smartphone.

© DB Systel GmbH

The app shows all the information about the platform in the camera image on the user’s mobile phone. So if a passenger wants to know which car the on-board restaurant is in, how full the train is or where exactly the doors will be, they only have to point their phone in the direction of the track. The display then shows the train together with the most important data.

Maintenance employees can read information directly from the object and leave virtual comments.

© DB Systel GmbH

Another target group is maintenance staff. “Using an app, employees could view the data collected by IoT sensors and superimposed on the object in real time,” says Johannes Hillebrenner. Or a technician could hold his smartphone over a switch during maintenance work on a track. On the display, he can see when the last repair was performed. What’s more, employees can also leave notes about each object, which can then be called up by all other technicians. The aim is to enable DB employees to intuitively find information about objects – superimposed on the real object, with no need to search through systems. This can make it considerably faster and easier to find information, perform work processes and collaborate.

© DB Systel GmbH

Using an app, employees could view the data collected by IoT sensors and superimposed on the object in real time.

Johannes Hillebrenner, DB Systel

But this process needs to be tackled one step at a time. “We’re operating in a very innovative field. Some of the technology is still in its infancy,” says Arne Kupetz. It is first necessary to demonstrate the opportunities to customers, spark their enthusiasm for the idea and jointly develop new mobile AR solutions.

Visit the Skydeck Demo Day in Frankfurt on 29 November 2018 to see for yourself how the “Companion” team is progressing with the implementation of mobile Augmented Reality solutions.