Translating sign language into audible English sounds like a technology that has a lot of potential, right? However, the sign language glove is not an end-all solution for the Deaf and Hard of Hearing communities, as this technology does not consider the needs and the goals of the sign language user.
Note: Words written in capital letters are gloss. Gloss is what you call it when you write a language in another. Since English is not an interpretation of American Sign Language (ASL), gloss will be used to transcribe the language.
For the purpose of this article I will be referring specifically to ASL.
What is a Sign Language Glove?
A sign language glove is a wearable device that translates sign language into audible English in real time. There have been a few iterations of the glove since 1988. One of the first devices was developed by James Kramer at Stanford University where fingerspelling in ASL outputted the words in audible English. Recently, Navid Azodi and Thomas Pryor, two undergraduate students at the University of Washington, have developed the SignAloud , which can translate American Sign Language into audible English or text.
Here is a demonstration of the SignAloud Glove in action:
How the Sign Language Glove Works
This glove has sensors that run along the fingers and the thumb to identify finger and palm positions and movements. This data is sent wirelessly via Bluetooth to a computer that analyzes the hand-movement data though sequential statistical regression, similar to a neural network. If the hand-movement is matched to a word or phase in the English language, the word or phrase is spoken through a speaker on the glove. This is repeated for all the movements done by the signer.
Limitations
The current designs of the sign language glove have many limitations, that I will explain. First, we must define what is a sign in sign language.
Definition of a Sign
Sign language is a visual language and is meant to be processed visually. There are five parts to a sign, these are called parameters. The five parameters must be performed correctly to sign the word accurately. The parameters of a sign are:
Movement
Handshape
Location
Palm orientation
Non-manual markers (non-hand actions & movements)
The sign language glove cannot detect non-manual markers.
Missing Non-manual Markers
Non-manual markers are action or movement made with something other than the hands to add or change the meaning of the sign. These can include mouth morphemes, eye gazes, facial expressions, body shifting, shoulder raising, and head titling.
For example, the signs for LATE and NOT-YET have the same movement, handshape, location, and palm orientation, but differ in their non-manual marker. The sign NOT-YET requires a mouth morpheme ‘TH’, done by placing the tongue over the bottom teeth and slightly shaking the head left/right, while LATE requires no mouth morpheme or head shake.
In the photo below you can see for the NOT-YET sign the signer has their tongue placed over their bottom teeth making the “TH” mouth position.
Photos from Handspeak.com
This means the sign language glove would not be able to accurately identify the difference between these two signs because it cannot detect the non-manual markers by the signer.
Sentence Structure
Sign language is not a direct translation of English; the vocabulary and grammar used in sign language are different. For example, in sign language, the question word is typically placed at the end of a sentence. If you wanted to sign “Where do you attend school?” you would sign YOU ATTEND SCHOOL WHERE. However, the glove sequentially says each word as they are signed, which would not result in proper English sentence structure or syntax. The sign language glove is only able to detect and translate fingerspelling and simple signs to English; it does not translate entire sentences. The order of the signs allows meaning to be expressed in ways that cannot be expressed in spoken languages, and these important subtleties are lost with the signing glove.
Missing Classifiers
In sign language, a classifier is a sign used to represent a general category or class of things. They are used to convey meaning through common understanding. Classifiers are handshapes that are associated with specific categories (classes) such as shape, size, amount, etc. A classifier handshape can be incorporated with a movement, location, or/and palm orientation to convey additional information. Without classifiers, there is no sign language.
Here is an example: in the frozen form, the sign in the figure below means CATERPILLAR, WORM, MEET, MEET-him/her, BOARD based on context. Once the sign is unfrozen and moving it becomes a classifier which could be the general categories such as:
Long skinny object, such as a person
Small cylindrical object, such as a pen, stick, pole
Trajectory path, such as ball flying, snake moving
Photo from Lifeprint.com
You can see that the signing glove cannot differentiate between these classes, because the same movement has different meanings. More so, classifiers are not structured signs that could be picked up by even the most robust neural network.
One Sided Conversation
The sign language glove allows for the person using sign language to communicate with a hearing person. However, the glove does not allow communication in the other direction, resulting in one-direction conversation. The fingerspelling and simple signs that the glove can translate could have easily been written down by the person using sign language without having to use a sign language glove.
Possible Applications
Despite its flaws, the sign language glove does have a few possible beneficial applications. The sign language glove could be used for teaching purposes for learning signs. However, there is no broad application of this device in the Deaf and Hard of Hearing communities.
It would be better if these technologies focused on user-driven designs, rather than proposing solutions that are not needed.
"What inventions like signing gloves totally ignore is the fact that deaf people are the ones living these daily lives in the hearing communities the inventors try to serve," said Julie A. Hochgesang, an associate professor in the Department of Linguistics at Gallaudet University.
The current sign language glove designs are flawed. However, with improvements, consultation with the Deaf and Hard of Hearing communities, and technological advancements, they could be beneficial.
How do you think this device could be improved? Comment below!
References
Erard, Michael. “Why Sign-Language Gloves Don't Help Deaf People.” The Atlantic, Atlantic Media Company, 10 Nov. 2017, www.theatlantic.com/technology/archive/2017/11/why-sign-language-gloves-dont-help-deaf-people/545441/.
“Non-Manual Signals Used in Sign Language.” Handspeak, www.handspeak.com/learn/index.php?id=158.
Picheta, Rob. “High-Tech Glove Translates Sign Language into Speech in Real Time.” CNN, Cable News Network, 1 July 2020, www.cnn.com/2020/06/30/health/sign-language-glove-ucla-scn-scli-intl/index.html.
Univ., James Kramer Stanford, et al. “The Talking Glove.” ACM SIGCAPH Computers and the Physically Handicapped, 1 Apr. 1988, dl.acm.org/doi/10.1145/47937.47938.
“UW Undergraduate Team Wins $10,000 Lemelson-MIT Student Prize for Gloves That Translate Sign Language.” UW News, www.washington.edu/news/2016/04/12/uw-undergraduate-team-wins-10000-lemelson-mit-student-prize-for-gloves-that-translate-sign-language/.
Vicars, William. “American Sign Language.” ASL - American Sign Language, www.lifeprint.com/asl101/pages-signs/classifiers/classifiers-frame.htm.
Wanshel, Elyse. “Students Invented Gloves That Can Translate Sign Language Into Speech And Text.” HuffPost Canada, HuffPost Canada, 28 Apr. 2016, www.huffingtonpost.ca/entry/navid-azodi-and-thomas-pryor-signaloud-gloves-translate-american-sign-language-into-speech-text_n_571fb38ae4b0f309baeee06d?ri18n=true.