Clique: Perceptually Based, Task Oriented Auditory Display for GUI Applications

Public Deposited
Analytics

Downloadable Content

Download PDF
Request Version for Screen Reader
Last Modified
  • March 20, 2019
Creator
  • Parente, Peter
    • Affiliation: College of Arts and Sciences, Department of Computer Science
Abstract
  • Screen reading is the prevalent approach for presenting graphical desktop applications in audio. The primary function of a screen reader is to describe what the user encounters when interacting with a graphical user interface (GUI). This straightforward method allows people with visual impairments to hear exactly what is on the screen, but with significant usability problems in a multitasking environment. Screen reader users must infer the state of on-going tasks spanning multiple graphical windows from a single, serial stream of speech. In this dissertation, I explore a new approach to enabling auditory display of GUI programs. With this method, the display describes concurrent application tasks using a small set of simultaneous speech and sound streams. The user listens to and interacts solely with this display, never with the underlying graphical interfaces. Scripts support this level of adaption by mapping GUI components to task definitions. Evaluation of this approach shows improvements in user efficiency, satisfaction, and understanding with little development effort. To develop this method, I studied the literature on existing auditory displays, working user behavior, and theories of human auditory perception and processing. I then conducted a user study to observe problems encountered and techniques employed by users interacting with an ideal auditory display: another human being. Based on my findings, I designed and implemented a prototype auditory display, called Clique, along with scripts adapting seven GUI applications. I concluded my work by conducting a variety of evaluations on Clique. The results of these studies show the following benefits of Clique over the state of the art for users with visual impairments (1-5) and mobile sighted users (6): 1. Faster, accurate access to speech utterances through concurrent speech streams. 2. Better awareness of peripheral information via concurrent speech and sound streams. 3. Increased information bandwidth through concurrent streams. 4. More efficient information seeking enabled by ubiquitous tools for browsing and searching. 5. Greater accuracy in describing unfamiliar applications learned using a consistent, task-based user interface. 6. Faster completion of email tasks in a standard GUI after exposure to those tasks in audio.
Date of publication
DOI
Resource type
Rights statement
  • In Copyright
Advisor
  • Bishop, Gary
Language
Access right
  • Open access
Date uploaded
  • October 11, 2010

Relations

Parents:

This work has no parents.

Items

Thumbnail Title Date Uploaded Visibility Actions
file details Clique : perceptually based, task oriented auditory display for GUI applications 2019-04-11 Public Download