Emacspeak
Direct Speech Access
T. V. Raman
Senior Computer Scientist
Adobe Systems

(C)1996 Adobe Systems Incorporated.All rights reserved.
1

Outline
O Problem overview.
O Speech enabling applications.
O Emacspeak --Architecture.
O Emacspeak --The user experience.
O WWW access and speech style-sheets.

(C) 1996 Adobe Systems Incorporated.All rights reserved.
2

Screen Access
Applications provide visual feedback.
Screen-access renders this feedback aurally.
User explores visual display to:
O Construct a mental model of the interface.
O And interpret intent of the UI.
Aural output --Consequence of the visual display.

(C) 1996 Adobe Systems Incorporated.All rights reserved.
3

Screen Access Design
Goal --Speak visual display.
Steps:
O Encapsulate screen appearance in an off-screen model.
O Present this model aurally.
O Enable navigation of this model.
Aural feedback is application independent.

(C) 1996 Adobe Systems Incorporated.All rights reserved.
4

Available Screen Access
Character-based interfaces :::GUI.
Richer layout of GUI
l
Harder to build OSM
l
But richer OSM
l
Enables possibly richer feedback

(C) 1996 Adobe Systems Incorporated.All rights reserved.
5

Screen Access Innovations
Innovations include:
O Application specific customizations.
-- Speak specific areas of the screen.
-- Attach meaningful labels to icons.
O Navigate window hierarchy instead of geometric layout.
Access separate from application.

(C) 1996 Adobe Systems Incorporated.All rights reserved.
6

What Is UI?
Applications:
1. Obtain user input
2. Compute on the information
3. Display the results
UI = Input+Output

(C) 1996 Adobe Systems Incorporated.All rights reserved.
7

Speech Enabling Applications
O Treat speech as a first class medium.
O Application produces its own feedback.
O Exploit features of the spoken medium.
O Context-specific, audio formatted output.
Aural feedback independent of visual display.

(C) 1996 Adobe Systems Incorporated.All rights reserved.
8

Contrasting Approaches
Screen Reading Speech Enabling
Speak display Speak information
Minimal context Contextual feedback
Application independent Tightly integrated
Global context Application context
Examines display Examines environment

(C) 1996 Adobe Systems Incorporated.All rights reserved.
9

Advantages
Speech enabled applications:
O Produce intuitive feedback.
O Provide a simpler user model.
O Reduces users cognitive load.
User works with one --not two-- applications.

(C) 1996 Adobe Systems Incorporated.All rights reserved.
10

Example
March 1996
Sun Mon Tue Wed Thu Fri Sat
1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31
Easy to see relevant information.

(C) 1996 Adobe Systems Incorporated.All rights reserved.
11

EMACS
A customizable, extensible user environment:
O Editing and browsing electronic information
-- WWW, Email and Usenet.
O Software development.
-- Code browser and debugger.
Unified interface enables cooperating applications.

(C) 1996 Adobe Systems Incorporated.All rights reserved.
12

Emacspeak
Speech enables all of GNU Emacs 19.
Using Emacs' extension mechanism:
O Device independent interface to speech hardware.
O Core modules for basic spoken feedback.
O Extensions for various applications.
Does not modify Emacs code-base.

(C) 1996 Adobe Systems Incorporated.All rights reserved.
13

Advice Overview
Advice can extend or modify existing code.
BEFORE Called before advised function.
AFTER Called after advised function.
AROUND Called instead of advised function.
Advice enables code re-use.

(C) 1996 Adobe Systems Incorporated.All rights reserved.
14

Advice
Let x ! f ! y.
Advice fragments
BEFORE AROUND AFTER
Result in function f 0
x ! f
--- #z #
f 0
! y 0
That computes y 0
.

(C) 1996 Adobe Systems Incorporated.All rights reserved.
15

The User Experience
Emacspeak provides feedback as you work:
O Succinct contextual speech feedback.
O Auditory icons augment interaction.
O User action flushes prior speech immediately.
O Audio formatting provides rich feedback.
User focuses on task at hand.

(C) 1996 Adobe Systems Incorporated.All rights reserved.
16

Demonstration --Editing
O Simple editing, search and replace.
O Completion and spell checking.
O Syntax coloring using voice-lock mode.
O Comparing files.
Intuitive interface enables fluent interaction.

(C) 1996 Adobe Systems Incorporated.All rights reserved.
17

Demonstration --Browsing Information
O Browsing the file system.
O Reading and responding to email.
O Browsing Usenet news.
O Browsing the WWW.
An integrated window into digital information.

(C) 1996 Adobe Systems Incorporated.All rights reserved.
18

WWW --Speech Style Sheets
Cascaded speech style-sheet specifies:
O Voice properties for specific document elements.
O Amount of pause around document elements.
O Auditory icons for document elements.
O Background audio for document elements.
Generate richly formatted audio documents.
c

O1996 Adobe Systems Incorporated.All rights reserved.
19

Demonstration --System Tasks
O Running a shell.
O Running terminal based applications.
Behaves like a traditional screen-reader.

(C) 1996 Adobe Systems Incorporated.All rights reserved.
20

Device Independent Speech
TCL scripts interface to different speech devices.
Implemented interfaces include:
O Dectalk Express
O MultiVoice
O Software Dectalk --DEC-ALPHA
O DECFACE --A Talking Face
Supporting other devices is straightforward

(C) 1996 Adobe Systems Incorporated.All rights reserved.
21

Core Modules
Basic audio services used to advise
O Navigation commands e.g., forward-word, next-line, :::
O Editing commands e.g., typing, deleting :::
O Prompting commands --functions that ask questions.
O File and buffer manipulation commands.
Sufficient to implement a working system.

(C) 1996 Adobe Systems Incorporated.All rights reserved.
22

Application Specific Extensions
The icing on the cake!
O Calendar
O Spell checker
O Mail and News reader
O WWW browser
Extension code-size is less than 1% of original.

(C) 1996 Adobe Systems Incorporated.All rights reserved.
23

Advice Statistics
Package Code Extension Per Cent Advice
Core 217,295 3,152 1.45% 188
W3 17,384 515 2.96% 12
Mail 17,943 270 1.50% 14
GNUS 35,528 501 1.41% 37
Emacspeak --A model of code re-use.

(C) 1996 Adobe Systems Incorporated.All rights reserved.
24

Conclusion
O Treat speech I/O as a first-class medium.
O Shows advantage of separating UI and computation.
O Enables rich, well-structured interfaces.

(C) 1996 Adobe Systems Incorporated.All rights reserved.
25