The vOICe for Android

The description of The vOICe for Android

Site Editorial Commentary：Augmented Reality for the Blind: See with your Ears!

See the world with your ears! "The vOICe for Android" translates live camera views into soundscapes, providing augmented reality Experiences and unprecedented visual detail for the totally blind through sensory substitution and computer vision techniques. This innovative app also features live talking OCR, a talking color identifier, a talking compass, a talking face detector, and a talking gps locator. Furthermore, Microsoft Seeing AI and Google Lookout object recognition can be easily launched from "The vOICe for Android" by tapping the left or right screen edge.

Is it a cutting-edge augmented reality game or a serious assistive tool? It's both, depending on your needs! The primary goal is to offer a form of synthetic vision to the blind, but sighted users can also enjoy the unique experience of sight-without-eyesight. Visually impaired users with severe tunnel vision can explore whether the auditory feedback helps them perceive changes in their visual periphery. "The vOICe for Android" is designed for smartphones and tablets and is also compatible with most smart glasses. It leverages the tiny camera in these glasses and a tailored user interface to create a live sonic augmented reality overlay, all hands-free! For extended use with smart glasses, consider using an external battery connected via USB to prevent rapid battery drain. Share your experiences, use cases, and learning journey of seeing with sound through blog posts and tweets to help others discover the benefits.

How does "The vOICe" work? The app uses pitch to represent height and loudness to represent brightness in one-second left-to-right scans of any view. A rising bright line is sonified as a rising tone, a bright spot as a beep, a bright filled rectangle as a noise burst, and a vertical grid as a rhythm. Using stereo headphones is highly recommended for the most Immersive experience and the highest level of auditory resolution.

Start by experimenting with simple visual patterns, as real-life imagery can be very complex. For example, randomly place a bright object, such as a DUPLO brick, on a dark surface and practice reaching for it using only the sound cues (close your eyes if you have eyesight). Then, explore your familiar home environment, associating the complex sound patterns with the objects you already know are present. Sighted users can also use the app with Google cardboard-compatible devices by swiping down on the main screen to toggle the binocular view.

For dedicated users: learning to see with sound is akin to learning a new language or a musical instrument, demanding perseverance and leveraging brain plastiCity. It can be considered an ultimate brain training system, bridging the senses through artificial synesthesia. A comprehensive training manual for "The vOICe" (not specific to the Android version) is available online at:

https://www.seeingwithsound.com/manual/The_vOICe_Training_Manual.htm

Usage notes for running "The vOICe for Android" hands-free on smart glasses can be found at:

https://www.seeingwithsound.com/android-glasses.htm

Don't be overwhelmed by the many options in "The vOICe for Android." Just as human eyes have no buttons or options, "The vOICe" is designed to perform its primary function effectively out-of-the-box, requiring no initial configuration. Some of the most commonly used options are revealed by slowly sliding your finger across the main screen.

Why is "The vOICe" available for free? Because our primary objective is to create a significant impact by minimizing barriers to access. Competing technologies often cost upwards of $10,000 with lower specifications. The perceptual resolution provided by "The vOICe" is unmatched, even by $150,000 "bionic eye" retinal implants (PLoS ONE 7(3): e33136).

"The vOICe for Android" supports English, Dutch, German, French, Spanish, Italian, Estonian, Hungarian, Polish, Slovak, Turkish, Russian, Chinese, Korean, and Arabic (accessible via menu Options | Language).

Please report any bugs to [email protected], and visit the web page http://www.seeingwithsound.com/android.htm for a detailed description and disclaimers. Follow us on Twitter at @seeingwithsound.

Thank you!