With a grant from the National Science Foundation, Texas A&M Professor of Visualization Francis Quek has developed technology for talking books that allows people who are blind to access more literature with increased command over their reading experiences.
STAAR Description Format (SDF), the novel technology developed by Quek, converts any portable document format (PDF) to a version the blind can read on an iPad. The users scan the text with their fingers to hear the words, control the pace of their reading, keep their place on the page, refer back to text, highlight important information and take notes, much like the experience enjoyed by those who are sighted.
“None of us who are sighted who are studying for an exam use audio recordings because we want to be able to read at our own pace, keep our place on the page, and refer back to text,” Quek said. “I may want to read slower when I’m studying for an exam, while I read a newspaper more quickly.”
Before literacy was widespread, ancient writings were designed to be read aloud for aural consumption. Mnemonic aids were embedded in the language structure to guide conceptualization and help the hearers keep track of the information. For example, the Bible is repetitive in its use of language. Sequential passages in the beginning of the Book of Genesis all conclude with, “And the evening and the morning were,” punctuated by different days, “the first day,” “the second day,” “the third day,” until the text reaches the sixth day.
“It doesn’t make sense until you realize that a Levite had to read the text on a hill to a group of illiterate people,” Quek said. “Actually, the only place where structured spoken language is commonly used today is in churches where some ministers still use alliteration, rhyme and beat in their sermons.”
Lightening the cognitive load
Since Johannes Gutenberg invented the printing press around 1440, books have steered away from the poetic style as literacy has increased. For this reason, audiobooks alone are less than optimal for the blind.
“The blind have to maintain every single piece of information in short-term memory when they are listening to an audiobook,” Quek said. “The cognitive load imposed upon a blind person in academia, in learning, is far greater than you and I could possibly entertain.”
Additionally, audiobooks for the blind, which use the latest optical character recognition and speech rendering technologies, do not provide the kind of flexibility afforded the sighted.
With Quek’s SDF technology, consisting of software and a user interface, the blind can download and read any PDF on an iPad. The software, designed with the help of graduate student Niloofar Zarei, renders audio for every word and formats the document, while a plastic overlay provides a tactile landmark grid on the iPad screen that allows the readers to keep their place spatially on the page.
“The PDF has become the standard way we post information for visual consumption, and as a scholar, I can download any PDF and read it immediately at any time,” Quek said. “We have built a system that allows blind students and scholars at any university to read the same documents and assignments at a much smoother pace with more control than is currently available to them.”
Braille for the 21st century
With the overlay applied to the iPad screen, the blind readers scan the text left to right with their fingers, and as they glide over the words, the system announces them audibly. The sound of crinkling paper alerts readers when they stray outside the boundary of the sentence they are reading; a clicking noise alerts them when they move too quickly and miss a word; and on old-fashioned typewriter “ding” lets them know when they have reached the end of a line.
Braille, a long-used system of raised dots that also can be read with the fingers, allows the blind to read letter by letter, rather than word by word, which takes more time, and the books tend to be cumbersome.
“For example, ‘Harry Potter and the Order of the Phoenix,’ is 896 pages, while the Braille version is 13 volumes, 1,215 pages, and two-foot high in a stack,” Quek said. “Braille was invented in 1823 before iPads with touchscreen technology existed.”
Quek and Zarei are in the process of refining their SDF system to give blind readers the ability to highlight text and make notes, just as sighted readers studying often do in their books or manuscripts.
“Studies show that when you highlight, you remember better, even when you don’t look back at the highlights again,” Quek said. “We suspect that this is because reflection is an important part of learning, and when you decide what to highlight, you are using this process.”
Ultimately, blind readers will have those same advantages. SDF will allow them to select snippets of text they want highlighted, which a different audio “voice” will distinguish. They also will be able to dictate notes as they read for later reference.
###
Media contact: Dr. Francis Quek, professor of visualization, at quek@tamu.edu; or Elena Watts at 979-458-8412 or elenaw@tamu.edu.
