Please view the main text area of the page by skipping the main menu.


Postponed Tokyo Olympics to start July 23, 2021: Organizing committee

AI-based app with face and text recognition helping guide visually impaired in Japan

Akiko Ishii uses the Seeing AI text reading app to check letters and other information from her daughter's day care center, in Tokyo's Nerima Ward on Dec. 19, 2019. (Mainichi/Tomoko Igarashi)

TOKYO -- A free smartphone app for the blind and visually impaired that uses artificial intelligence to identify and audibly describe people and objects using the phone's camera is receiving praise from users.

The app, "Seeing AI" was originally developed in English by Microsoft in 2017, and is now available in Japanese. Users say it allows them to feel as if they can see the things in front of them.

The app is used with a smartphone function that reads out what the user touches on the screen. When the application's function to read out text is turned on, it will automatically read out the text in front of the camera, such as that on mail and food packaging. Then when the user points the phone in the direction of another person and takes a photo, the app can identify the person and their expression, with a description such as, "There is a smiling woman aged about 20 two meters away." If the name of another person, such as a friend or family member, is registered in advance, the app can inform the user that it is that person.

The app's eight functions include one to identify and read out the colors of objects, and another to inform the user of the level of brightness inside a room. According to Microsoft Japan Co., there have been similar apps in the past, but it has been unusual for an app to have all eight functions.

Akiko Ishii, a 46-year-old blind resident from Tokyo's Nerima Ward, uses the application to check the use-by dates on food products, to make selections from restaurant menus, and when she receives consultation slips at hospitals, and so on.

"It has enabled me to do things by myself in those moments when the need arises," she says.

Ishii has also found the app helpful as she raises her 3-year-old daughter.

"When my daughter is quiet, there are times when I don't know where she is. But when I wave my phone around, it tells me, which is helpful," she says.

The app was developed in 2014 by a visually impaired worker at Microsoft's headquarters in the United States as part of a contest, and completed over the course of three years. In 2017 the English version was released, and the Japanese version came out in December 2019. At present it is restricted to iPhones, and can be downloaded from Apple Inc.'s App Store. Telephone inquiries can be made to Microsoft Japan on 0120-54-2244.

(Japanese original by Tomoko Igarashi, Tokyo City News Department)

Also in The Mainichi

The Mainichi on social media