© AdobeStock (Lilli)

Automatic image recognition

Hunting down graffiti with artificial intelligence

02/2019 – Damage caused by graffiti is a nuisance, but also costs millions to repair. In order to discover it more quickly, Deutsche Bahn is now resorting to intelligent means: in future, a combination of cameras and artificial intelligence should detect graffiti automatically.

The question “Is that art or can it be removed?” is never asked at Deutsche Bahn. If graffiti is discovered on a train, it must be removed as soon as possible. For the Group, however, it is above all a high cost factor. The annual cost of removing graffiti, for example, is almost ten million euros. The Deutsche Bahn Safety Report estimates the number of incidences of graffiti damage in 2017 at 18,120 – a rise of four percent over the previous year.

On the S-Bahn rapid transit system in Hamburg graffiti on the trains is virtually the order of the day. This is not only a matter of cost. “Dirty and defaced rail vehicles are perceived by our customers as a lowering of safety standards”, says Sven Krayl, head of IT and Security for the S-Bahn. “It is also a matter of subjective safety, by which the S-Bahn in the Hamburg Transport Association (HVV) is measured in quality control procedures.” Also working in his department are the security managers who must address the topic of graffiti, and with whom he continuously exchanges views about the problem and seeks solutions.

From snow to graffiti

The solution comes from the IoT/M2M department of DB Systel. On the basis of the Visual Recognition Service in the DB IoT Cloud a solution has been developed which can automatically detect graffiti. The Visual Recognition Service is already doing a good job in other areas, for example in detecting snow at stations. But this time the task is more complex, as graffiti in widely differing forms is to be detected on moving vehicles. To ensure success, an artificial intelligence system is trained using images, so that it can recognise specific things such as objects, nameplates, structures and any details in the images. For this purpose, the DB IoT Cloud Visual Recognition Service uses deep learning algorithms, in order to analyse images and scenes, objects and other content. The information that shows whether a rail vehicle has been defaced or not is then assigned to a virtual image, the digital twin of the rail vehicle. This additional information, e.g. the graffiti-covered status of the train, can then be accessed by all departments connected to the system.

Graffiti-covered rail vehicles are perceived by our customers as being less safe.

Sven Krayl, Head of IT and Security for S-Bahn Hamburg GmbH

Recognition test in Hamburg

For this reason, it did not take long to convince Sven Krayl to check the reliability of the recognition (Proof of Concept) in a test on site. The intelligent visual recognition should recognise graffiti on S-Bahn trains automatically and send corresponding information to the cleaning services. For this purpose, standard cameras will be installed on two platforms of Hamburg main station to photograph every incoming train. No more cameras are needed, as all S-Bahn trains pass through this station at least once on their route. “This meant that, after a certain time, we had recorded every train in service”, says Krayl. So the cameras only have to be installed at one site, thereby keeping down the capital investment costs.

An automated recognition and optimized delivery of sprayed vehicles to the depot ensures that the cleaning process can be better planned. Automatically created, digitised job lists result in a transparent and optimized performance of the cleaning.

© DB Systel GmbH

As the train passes through, not only is the wagon identification number recognised, but also all images, drawings and disfigurements that do not belong to the train. In order to differentiate between advertisements and graffiti, an artificial intelligence (AI) system must first be taught a little art lesson. In addition, it must be able to recognise when a train is coming in and which train it is. Even in the very first test, good results were achieved. Using an affordable outdoor camera from Axis, a total of 2,500 images per day were supplied and a recognition quota of 97 percent already achieved. “It may be assumed that with further learning over time, this performance can be further improved”, says Jörn Petereit, Vice President IoT/M2M at DB Systel.

With the aid of artificial intelligence and machine learning, self-optimising models can be developed in DB IoT Cloud in the course of time, so that findings and recommendations obtained there can be fed back faster into the decision and solution-finding process.

© DB Systel GmbH

Open to further applications

Yet it not only recognises whether a train has been sprayed, because the DB IoT Cloud also supports other options: by interacting with the digital twin, new knowledge is created, for example on the size of graffiti on the basis of the image data, statistics on advertising or also on perpetrator profiles by means of graffiti forensics. This new intelligence can also be fed back subsequently to the physical asset or also the digital process where necessary.

Primarily, however, the focus is on recognition. The system will be entering service not later than during the first quarter of 2019. It is clear that the system will not prevent vehicles being vandalised. But the early recognition of graffiti enables Deutsche Bahn to remove it even faster. “For the culprits, this makes it less attractive to use our rail vehicles as travelling graffiti trophies”, asserts Sven Krayl confidently.