Anna C. Cavender
Computer Science and Engineering, University of Washington, Seattle, Washington
Educational technology has the potential to better include deaf and hard of hearing students in the academic mainstream. This research involves development and testing of a classroom platform for deaf and hard of hearing students to access remote interpreters and captioners, avoid visual dispersion, and facilitate interaction in the classroom.
Entering mainstream universities (at all levels) involves extra challenges for people who are deaf and hard of hearing: skilled sign language interpreters and captioners with advanced domain knowledge can be difficult to find; multiple visual channels of information in the classroom can be difficult to juggle; and collaboration inside and outside the classroom is often strained due to language barriers.
Classroom technology research is currently improving educational experiences for all students and this creates opportunities to better include deaf and hard of hearing students. Wireless networks, data projectors, and portable computing devices can allow remote interpreters, support sharing and capture of instructional materials, and provide additional communication channels for all. A more digital academic environment creates an opportunity for customization to better suit the needs of individual students.
This research will investigate and develop technology to help manage the many academic tasks required of deaf and hard of hearing students. Development will parallel other educational technologies (namely Classroom Presenter  and ConferenceXP ) so that technology for deaf students will be similar to those used by all students.
Students currently utilize an array of accommodations in academic settings including: interpreters, real-time captioners, hearing aids, FM systems, and note takers. However, several issues currently create extra obstacles for deaf and hard and hearing students in university-level classes:
As more deaf students enter mainstream universities, there is a growing need for skilled sign language interpreters and real-time captioners with specialized knowledge in advanced courses. Finding an appropriate interpreter considering the variety of topics available can be a challenge, especially for complex courses such as Complexity Theory. The best interpreter for a given student may be located at another university in another state or country.
Because deaf students receive nearly all classroom information visually, they must juggle their visual attention between instructor, slides, interpreter and/or captioner, and personal notes or handouts. Due to this juggling, information can easily be missed. In fact, many deaf students request a student note-taker in order to eliminate at least one of their many visual tasks. While this helps ease visual burdens during class, the student may miss out on the value of taking and studying personal notes.
In spite of the plethora of possible accommodations, communication and participation within the classroom can be strained. Access to information is not equivalent, especially in terms of delivery time, and this makes call-and-response or question-and-answer techniques unfair. Furthermore, involvement outside the classroom (such as project groups meeting and impromptu study groups) can inadvertently exclude deaf or hard of hearing students.
Technology has potential to alleviate these problems, encourage participation, and enhance learning for all students.
Collaboration between universities through the existing multimedia cyberinfrastructure (software, hardware and other technologies including human expertise) would allow better access to skilled interpreters familiar with specialized, university-level topics, creating more opportunity for matching the best interpreter with specific content. This approach will also allow different types of students to receive differing accommodations based on preference. For example, one student may prefer a remote sign language interpreter while another student prefers realtime captioning.
Consolidating visual content into one device may prevent missed information due to the visual juggling act that many deaf students experience. Miller et. al. proposed using transparent video and overlaid digital ink to reduce the visual distance from the interpreter (video) and the student's notes (digital ink) . Taking this concept one step further, imagine a Tablet PC application that allows students to see the instructor, presentation, and accommodation of choice all on one screen and all within visual distance of personal annotations.
Technology has been shown to enhance education in the classroom and these “digital” environments open up new possibilities for leveling the academic playing field. The University of Washington's Classroom Presenter uses a system of networked Tablet PCs allowing students to electronically submit their work and/or questions to the instructor who can then choose to display submissions and digital ink on lecture slides . ConferenceXP, developed at Microsoft Research, provides the infrastructure for networking the Tablet PCs, and is also used for audio and video distance learning and classroom capture . These two projects will provide a backbone for accessible classrooms for deaf students.
Given the scenario where all students are equipped with a networked Tablet PC, an additional opportunity exists for student collaboration. Kam et. al.'s LiveNotes uses digital ink over lecture slides to encourage group conversations and cooperative note-taking during lectures . This idea could be used to bridge the cultural and language gap between hearing and deaf students and encourage group work.
Figure 1: Here, the cyberinfrastructure brings remote interpreters and captioners into the classroom. The instructor uses a microphone, earpiece, and laptop camera to relay audio, video, and presentation materials to the remote interpreter. Students have access to presentation, instructor, accommodation of choice, and their own notes. Students' webcams relay questions and discussions through the interpreter to the rest of the class.
As academic environments become more digital, capture and retrieval introduce interesting areas to improve content accessibility. Synchronization of video feeds, digital ink, and presentation materials could result in better preservation and easier post-class access, much like eClass  and other classroom capture techniques .
Our primary research goal is to find ways to increase involvement of deaf and hard of hearing students in university academics. Improving access to resources and providing additional communication channels will involve participatory design and iterative development. Solutions will be viable for traditional classroom environments as well as for lab sessions, study groups, and project meetings.
We are currently collaborating with Rochester Institute of Technology (RIT), home of the National Technical Institute for the Deaf (NTID) supporting over 400 deaf students in the academic mainstream, over 120 sign language interpreters, over 50 captioners. NTID has conducted extensive research on interpreting and captioning in tertiary education. Collaboration will jump start the design of a “cybercommunity” to enable deaf and hard of hearing students to advance academically through technological access. Participants will include deaf students, interpreters, captioners, sign language linguistic researchers, educational technology researchers, and cyber-infrastructure experts.
An excellent opportunity for evaluation and feedback of the proposed research is the Summer Academy for Deaf and Hard of Hearing Students hosted each summer at the University of Washington. The top ten college freshmen or sophomore applicants join the program to take college courses focused on computer science and related fields. Because the academy involves mainstream, 9-week courses, it presents an ideal test-bed.
1. R. Anderson, J. Chen, L. Jie, N. Li, N. Linnell, V. Razmov, and F. Videon. Supporting an interactive classroom environment in a cross-cultural course. In ASEE/IEEE Frontiers in Education, to appear., 2007.
2. J. Beavers, T. Chou, R. Hinrichs, C. Moffatt, M. Pahud, and J. V. Eaton. The learning experience project: Enabling collaborative learning with ConferenceXP. Microsoft Research Technical Report MSR-TR-2004-42, April 2004.
3. J. A. Brotherton and G. D. Abowd. Lessons learned from eclass: Assessing automated capture and access in the classroom. In CHI ‘04: Proceedings of the SIGCHI conference on Human factors in computing systems, 121–155, New York, NY, USA, 2004. ACM Press.
4. M. Kam, J. Wang, A. Iles, E. Tse, J. Chiu, D. Glaser, O. Tarshish, and J. Canny. Livenotes: a system for cooperative and augmented note-taking in lectures. In CHI ‘05: Proceedings of the SIGCHI conference on Human factors in computing systems, 531–540, New York, NY, USA, 2005. ACM Press.
5. D. Miller, J. Culp, and D. Stotts. Facetop tablet :: note-taking assistance for deaf persons. In Assets ’06: Proceedings of the ACM SIGACCESS conference on Computers and accessibility, 247–248, New York, NY, USA, 2006. ACM Press.
6. S. Mukhopadhyay and B. Smith. Passive capture and structuring of lectures. In MULTIMEDIA ’99: Proceedings of the ACM international conference on Multimedia (Part 1), 477–487, New York, NY, USA, 1999. ACM Press.
Anna Cavender is pursuing a Ph.D. in Computer Science and Engineering at the University of Washington. She graduated with a B.S. in Computer and Information Science from the University of Oregon in 2004. Her research interests lie in Human-Computer Interaction, specifically assistive technology for people with disabilities. She was the recipient of the Computing Research Association (CRA) Outstanding Undergraduate Award in 2004 and is a National Science Foundation (NSF) Fellow.