Replacing white sticks with electronic devices: new technologies for the visually impaired

Translate images into sounds and you have a valuable navigation tool for blind people: increasing their mobility, safety and autonomy.

The white cane is still one of the most common tools that blind people use to stay mobile. But think instead how valuable an electronic device would be if it could communicate on the surrounding environment to its user.

Objects communicated as sound

Catalin gradually lost his sight during childhood. Nowadays he teaches in a school for visually impaired persons and he’s among the volunteers at the Politehnica University of Bucharest testing a new kind of navigator for blind people. His experience will help researchers customize the device to the needs of the users.

"The system includes a 3D camera that continuously scans the environment, very fast, 20 times a second,” says Alin Moldoveanu, Technical Coordinator of the Sound of Vision Project and Professor at the Politehnica University of Bucharest. “Then we process that information with computer vision algorithms and we break down every individual object. Each individual object is presented to the user through 3D computer-generated sounds and he hears the object through vibrations on a belt on his abdomen."

Application reads signs

The blind person is guided in real-time by those 3D sounds together with vibrations or haptic feedback. The intensity of the feedback varies depending on the user’s distance from the object. Researchers on this European project say it’s like creating a new sense for visually impaired people, who are also able to read signs thanks to another application.

"The user when he wants to read a text just presses the button and the system look for texts and indicates him where the texts are,” says Professor Moldoveanu. “He can look toward that direction and hear the text: pharmacy or supermarket or metro station"

The system identifies the most important objects like stairs, doors or people moving around. Every change in the surrounding environment is immediately detected. Users can also customize some functions according to their needs.

"Blind people can only 'see' what the white cane touches, it means what surrounds them one meter in front and up to one meter in height,” says Runar Unnporsson, Sound of Vision Project Coordinator. “What the Sound of Vision prototype does is to allow them to see further and higher up."

Handling life-critical demands

Research on outdoor use of the device is being led by the Technical University of Iasi. For the likes of Eusebiu to be able to use it in public spaces, the challenge is to handle sudden modifications: changes in lighting conditions, rain or vehicles and people moving around.

"Regarding outdoor use, we rely on two stereo cameras, two cameras that see the environment at the same time, and based on these two images we estimate distances from the objects in the environment,” says Simona Caraiman, Professor of Computer Engineering at the Technical University of Iasi. “We need to understand that this is a life-critical application, so the system cannot fail when signalling dangerous situations like a hole in the ground, for example."

Visually-impaired tester Eusebiu Toma is completely won over.

"From the moment I used this device, I felt a much greater sense of confidence and safety than I got from anything else I’ve used before," he says.

The three-year project started in 2015 with the participation of nine partners from five European countries – two from Iceland, three from Poland, one partner from Hungary and Italy, and three from Romania.

Researchers are currently fine-tuning the indoor and outdoor systems to combine them together. The next step will be to miniaturise the device for commercialization - within the next two years.

View Gallery

7 Photos