Anna C. Cavender, Daniel S. Otero, Jeffrey P. Bigham† and Richard E. Ladner
American Sign Language (ASL) currently lacks agreed-upon signs for complex terms in scientific fields, causing deaf students to miss or misunderstand course material. Furthermore, the same term or concept may have multiple signs, resulting in inconsistent standards and strained collaboration. The ASL-STEM Forum is an online, collaborative, video forum for sharing ASL signs and discussing them. An initial user study of the Forum has shown its viability and revealed lessons in accommodating varying user types, from lurkers to advanced contributors, until critical mass is achieved.
K.4.2 Social Issues: Assistive technologies for persons with disabilities; H.5.3 Information Interfaces and Presentation: Group and Organization Interfaces
Human Factors, Design
American Sign Language, Deaf, STEM, Video, Forum
American Sign Language (ASL) is very young relative to other languages, originating in the early 1800s, and only recently recognized as a language with the pioneering work of Stokoe in 1960 . With small numbers of deaf students in advanced science being geographically dispersed, the growth of ASL has been severely inhibited in Science, Technology, Engineering, and Math (STEM) fields [9, 14].
Often deaf scientists, students, and professionals decide on signs to use on a local (regional) or temporal (the duration of a class or conference) basis. As a result, alternative signs for the same term are developed, and developed signs are lost. Lack of standardization has been recognized for some time  and creates obstacles for collaboration and learning.
Recent efforts have tackled this problem of developing STEM signs
through careful consideration by expert committees who then disseminate
video sign language dictionaries either online or on DVD [4, 11, 15].
These dictionairies are an important first step, but natural languages
evolve by consensus not by committee . New vocabulary arises from
language-using populations, with standardization resulting from popular
adoption over time. In the case of ASL, the small size and geographic
dispersion of the signers presents major obstacles to this sort of
natural language growth.
Figure 1. ASL-STEM Forum on the topic “Arrays.” Topics are found in a searchable index (left). Topic pages show a text definition (top center), highest rated sign (right), all signs posted and text comments.
The ASL-STEM Forum is a community-based, video-enabled web resource that provides a venue for natural language collaboration and discussion about ASL terminology for STEM topics . We have started to seed the forum by inviting a select group of ASL users with STEM backgrounds, although anyone can participate.
To succeed, the Forum depends greatly on thoughtful participation from ASL users. We see two crucial design requirements. The ASL-STEM Forum should:
Increasing the contributor base of the Forum also needs to be balanced with encouragement to develop quality contributions. Although signs should be easy to contribute, contributors should feel obligated to add quality signs. While the forum should be inviting to all, community members should also be able to collectively converge on agreed upon signs.
At this early stage in the life of the Forum, we conducted a 10-day
user study with geographically dispersed ASL users who are students,
faculty, and scientists to investigate the ease of use and types of
discussions that may occur. We found that while achieving a critical
mass of contributors is key for long term success of the Forum, the
positive feedback and likelihood of continued use indicate that more
contributors will come to the Forum in the future.
We do not consider the ASL-STEM Forum to be a dictionary, but do
hope that it will become a reference for terms lacking an established
standard. Signs from the Forum may eventually be added to sign language
We conducted a study to gauge the potential of contributions from community-driven resources (like our Forum) becoming additions to official dictionaries. We looked to see when the 657 English words added to the Oxford English Dictionary (OED) in 2008 as main entries first appeared in the following community-driven resources: wiktionary.org, wikipedia.org, and urbandictionary.com. Of the 657 new terms, 450 were already included in community-driven resources (as shown below), suggesting that community-driven online resources can be a valuable tool for providing definitions quickly before the formal process has completed.
|Resource||# Included||Average Delay (years)|
The ASL-STEM Forum is the first online video-based collaboration
tool with the goal of expanding American Sign Language in STEM fields.
The design and creation of the Forum has been inspired by existing
video catalogs or dictionaries, has leveraged the increasing use of
video-enabled social networking sites, and is based in a long history
of collaborative and community-driven systems.
Online video references for technical and scientific signs indicate a strong need for consensus among members of the deaf community about signing for STEM topics [4, 10, 11, 15]. These resources are an important first step, but lack the fluid natural language evolution that informally occurs as other written, spoken, and signed languages develop. In contrast, ASL-STEM Forum supports community-centered discussion with geographically-dispersed community members. Because ASL is a visual language, video discussion and collaboration best enables this natural process.
Our ASL-STEM Forum builds on the deaf community’s rapid acceptance of video phones, deaf-oriented vlogs (video blogs), and video-enabled social networks, such as facebook.com and camfrog.com. Video technologies have already affected ASL by bringing together dispersed deaf communities and accelerating the natural process of discussion about differing vocabulary and idioms, resulting in agreement on common vocabulary or alternatives for the same concept.
Community-driven online resources, such as wikipedia.org, have proven a popular and expansive way to share information online. Their success is due to the participation of many users, made possible by the theory that the contributions of large groups will converge toward truth . Problems of bootstrapping and discontinuance plague collaborative systems and can stop growth before reaching critical mass . With the small group of participants in our study, the cost of contributing (time, effort, preparation, and upload) likely outweighed the benefits on the thus far sparsely-populated site (looking up unknown signs, collaborating with others) . A key aspect of promoting group involvement is supporting participation at a number of levels and roles in the system, from beginning editors to full-blown advanced editors . Our forum allows members to upload their preferred sign via an integrated video capture, discuss and comment on signs via either video or text, and anonymously rate signs on a scale from 1 to 5 stars. The community ratings are used to emphasize the highest rated signs for each concept using a collaborative filtering approach .
We have designed and implemented the ASL-STEM Forum (Figure 1) to enable members of the community to upload videos of signs and sign concepts, discuss alternatives, and rate others’ contributions. We designed the Forum to maximize speed of contribution and simplicity of video interaction, and to allow easy tracking of recent content changes.
The ASL-STEM Forum is built using the Ruby programming language and the Rails web application framework, or "Ruby On Rails." While data and user-created content are maintained in a local database, user-created video content is hosted by and streamed from YouTube.
The Forum’s organization is reminiscent of both typical web fora and Wiki content organizations. The Forum is divided hierarchically into “topics.” The four root topics match the mission, and name, of the Forum: Science, Technology, Engineering, and Mathematics.
Each topic has its own Forum page displaying the English word for that topic and an English definition. Users can add video sign suggestions and/or discussion comments to the page. Each topic page hosts a single thread of forum discussion regarding signing for the topic.
Registered users can also rate sign videos on a five-star scale. The highest-rated sign is displayed prominently, while poorly ranked signs are shown on a separate page.
Based on comments of pilot users, we simplified the capture and upload process, implementing our own web-based video capture applet (Figure 2). The applet allows users to seamlessly record video directly from a webcam that then becomes available on both YouTube and the Forum. Uploading videos to the ASL-STEM Forum is made possible by the YouTube external data upload API, which allows applications to dynamically interact with its video service. YouTube allows embedding of its videos in third-party HTML pages, a feature we use to integrate user videos into the Forum.
Figure 2. ASL-STEM Forum’s QuickCapture applet. The applet automatically connects to the user’s webcam, records and re-records with a single click, then uploads when the user submits.
The success of the forum hinges on community interest and involvement. For this reason, it is crucial that people can easily contribute signs, discuss signs, and access those signs. We conducted the following study to evaluate the usability of the forum as well as the ways in which people use the forum to discuss and develop signs.
For 10 days, we tracked study participants using the Forum. After a short preliminary questionnaire asking demographics and experience with sign language, STEM fields, and Internet usage, participates were given a Forum account and asked to explore and contribute to any area they wished.
We asked participants to contribute 3 signs in the first 5 days and 3 signs in the last 5 days. We tracked: (i) date and time the Forum was accessed, (ii) number of video signs contributed, (iii) number of signs rated by other members, and (iv) number of text comments on sign topics. At the end, participants answered a post-study questionnaire about their opinions of and experience using the Forum.
We recruited 14 participants for our study (3 female, 11 male); all were ASL users and involved in STEM majors and/or careers. The group was diverse in age, average age was 36.3 (SD = 14.7) and represented a broad range of interests in science and levels of expertise. The participants were geographically dispersed throughout the U.S., although half were either at Gallaudet (a liberal arts university for deaf and hard of hearing students) in Washington, DC or the National Technical Institute for the Deaf (NTID) at Rochester Institute of Technology (RIT) in Rochester, NY. All participants were extremely fluent in a sign language: nearly all reported life-long exposure to sign language with an average 26.1 years (SD = 6.7). Participants were also technically savvy, reporting an average of 25.3 hours per week accessing the Internet (SD = 17.5): all reported using email, 13 use instant messaging, 9 use social networking (such as facebook and myspace), 5 reported blogging, 6 use online video conferencing, and 2 reported vlogging.
During the 10 days, participants contributed a total of 106 video signs, 24 text comments, 18 ratings, and 9 new topics, totaling 163 combined contributions. Of the 14 who signed up for an account, 5 completed our minimum contribution requirement for the study, 3 contributed less, 2 contributed comments and/or rating but not signs, and 4 did not contribute at all. For the 4 non-contributers, 2 had technical difficulties with their webcams or our system, 1 cited “I did not have time,” and 1 dropped out for unknown reasons.
Contributing signs appears to be an easy task; participants took an average of 2.27 minutes to contribute one sign (SD = 1:38), with the fastest contribution at 0.78 minutes and the longest at 6.15 minutes. This is especially quick considering the steps required: the QuickCapture loads, the user records their sign (perhaps re recording if they are unhappy with the first), the user enters meta-data (description, title, and/or keywords) for the sign, and the sign video uploads to YouTube. The ease of use was supported by the post-study questionnaire: participants responded on average 4.0 on a 5 point scale (SD = 0.8) to the statement "The ASL-STEM forum is easy to contribute to," corresponding to "4-Agree."
Participants rated the statements "The ASL-STEM Forum is ... a valuable resource" and "easy to access," on average 4.8 (SD = 0.4) and 3.7 (SD = 0.8) respectively with options "5-Strongly Agree," "4-Agree," and "3-Neutral." No one chose "2-Disagree" or "1-Strongly Disagree."
Feedback from the post survey was also generally positive:
We noticed what may be varying levels of user contribution: advanced contributors who submitted a large amount of both signs and comments, mid-level contributors who only submitted a few signs or comments, and lurkers who submitted no signs and only rated or commented on other’s signs, not unlike other online communities .
Based on both behavior and survey results, some participants felt that commenting on signs was more valuable and preferable to rating. The six participants who contributed the most signs also commented on others’ signs and indicated in the post survey that ratings do not contain enough information. For example, a low rating could convey many different types of opinions: perhaps the sign is inaccurate, too iconic, only appropriate for some situations, or perhaps the video is blurry or the hands obscured. A comment could convey these issues and perhaps result in an appropriate fix, whereas ratings are not as constructive. This is interesting to us because our system uses ratings to display the most favorable signs prominently, allowing the Forum to serve as a mechanism for signs to converge on a community accepted basis. Comments are very useful and important for discussion but certainly can not be used to rank the Forum’s signs.
Four participants used ratings, often instead of signs or comments. The participant who used ratings most did not post signs or comments and later selected “I didn’t want my name/face associated with the topics/signs” and “I didn’t want to come across as attacking another Forum member” in the post-study survey. We think ratings may be an attractive alternative for users who do not want to setup a webcam, or who would prefer a more anonymous expression of opinion.
The current focus of the Forum is community participation.
A balance of contributors and viewers may happen naturally, but we can catalyze initial participation in a number of ways. There is some evidence that early contributors drive the growth of collaborative systems by creating norms, standards, and a content base for further contributions . With this goal in mind, we are working with Gallaudet and the National Technical Institute for the Deaf to seed the forum with initial signs; both wellknown standards and signs that may elicit discussion. This will free users to contribute where the language is sparse and provide new users with examples of community norms (how to present a sign in a video, whether to include an example sentence, how to add subtopics, etc.). Existing ratings will make the rating system more visible.
Recommending relevant places to contribute based on profile, past contributions, and topic subscriptions may encourage further contribution . Also, recognizing the social benefits of online communities and the finding that real world acquantaince positively affects online engagement (the deaf community is a tight-knit crowd) may also entice members to contribute, for example by including peers’ usernames in recommendations .
We will also investigate ways to motivate quality through member reputation, based on the types and numbers of signs contributed, the ratings of those signs, and general sign acceptance by the community .
The ASL-STEM Forum offers a vehicle for growing a language in a virtual environment, overcoming the geographic dispersion currently central to its real world problems in achieving critical mass. In a study with a fully-implemented forum, we observed significant advantages in terms of the rate at which signs were added and the number of people included over existing formalized resources.
Most participants wanted a greater sense of Forum community. We believe this can be reached by including different types participants, providing domain-specific encouragement, and leveraging the social benefits of the Forum.
We thank our participants for their time, contributions, and helpful feedback. Thanks to Professors E. William Clymer and Jorge Diaz-Herrera for ongoing support. We are thankful for excellent feedback from CHI reviewers. This work has been supported by a Boeing endowment, gifts from Google, and NSF Grant IIS-0915268.
Permission to make digital or hard copies of all or part of this
work for personal or classroom use is granted without fee provided that
copies are not made or distributed for profit or commercial advantage
and that copies bear this notice and the full citation on the first
page. To copy otherwise, or republish, to post on servers or to
redistribute to lists, requires prior specific permission and/or a fee.
CHI 2010, April 1015, 2010, Atlanta, Georgia, USA.
Copyright 2010 ACM 978-1-60558-929-9/10/04...$10.00.