Google rang in the Lunar New Year with a couple of AI-powered treats: a new Live Transcribe service to help the deaf and hard of hearing, and a Google Doodle showcasing the ancient Chinese art of Shadow Puppetry.
Real-time Transcription
Voice-to-text technology is designed to recognize audio in various languages and turn it into text. It is sometimes referred to as Automatic Speech Recognition (ASR), and has been applied in areas such as automated captioning on YouTube. Existing ASR-based transcription services however are usually server-based and require heavy computing resources.
Last October Google unveiled a “Call Screen” feature on their Pixel devices, which brought similar voice-to-text functions to mobile devices. Call Screen can answer incoming calls, ask for additional information, and transcribe the responses in real-time to allow users to decide how to handle the call.
Google’s new Live Transcribe service presents a similar real-time transcription function accessible on Android phones, which will help hearing-impaired users with phone tasks in their everyday lives. The free Android service performs smooth automatic captioning on a conversational level and supports more than 70 languages, covering over 80 percent of the world’s population.
Google AI partnered with Gallaudet University for the related research on user experience (UX) such as how to display transcription confidence, which they did via colour-coding.
Researchers acknowledge that environmental noise levels can also affect Live Transcribe performance due to “the cocktail party effect.”
Technical details of the Live Transcribe can be found in Google AI’s Blog post. The Live Transcribe service is available on Google Play.
Shadow Puppetry of the Chinese Zodiac
Google Doodle is known for logo alterations commemorating holidays, achievements, people, etc. On February 5 in multiple countries, the Google Doodle took the form of a paper-cut window decoration featuring a shadow puppetry animation transitioning from the Year of the Dog to the Year of the Pig.
Shadow puppetry is a traditional Chinese storytelling art which employs paper cutouts or hand gestures in front of a light source to project figures and actions on a translucent screen.
Google’s Lunar New Year doodle links to Shadow Art, an AI-powered interactive web app that guides users in the creation and performance of Chinese Zodiac-inspired hand gestures using their webcam. It is powered by the open-source library TensorFlow.js, which runs an in-browser (on-device) machine learning model to recognize users’ silhouetted hand gestures and match them with Chinese Zodiac animals.
Shadow Art is similar the earlier Google AI web app experiment Quick, Draw!, which uses a neural network to learn and recognize users’ line drawings. Quick, Draw! was so popular that its users collectively built the largest doodling dataset in the world. Might we expect the same thing from the Shadow Art app, with regard to boosting AI’s hand gesture recognition abilities?
Author: Mos Zhang | Editor: Michael Sarazen
0 comments on “Google’s Lunar New Year AI Surprises”