|
[August 1, 2001]
Atomistic Parallel Listening: The Next
Step In Speech Recognition
BY WALTER ROLANDI, Ph.D.
Everyone agrees that speech recognition technologies have advanced
significantly over the past ten years. Under favorable conditions,
recognition rates for both grammar-based and dictation systems approach a
human-like ability to recognize words and phrases. The dramatic growth of
various speech recognition companies attests to the stability of the
technology.
But if the technology is so advanced and sound, why isn't everyone
everywhere already talking to computers? The answer depends on what is
meant by "talking to computers."
There are indeed a great many people who interact with voice portals on
a daily basis. These applications are typically "hidden menu"
systems, or voice response units (VRUs), where users are prompted in some
way or another to say one of a set of available choices. In these
circumstances, the number of things a user can say is limited and the
possibilities are usually reduced to set of relatively short but
phonetically distinct utterances. Recognition of one item either leads the
user to content or to other menu choices that are required before content
can be delivered to the user. "Talking to computers," in this
respect, is well supported by current technologies. When a grammar is
limited and its elements are sufficiently distinct, commercial recognizers
will understand user utterances at rates close to 100 percent.
But human conversation is not limited to proffering and selecting among
menu choices. Conversational dialog is highly variable and indeterminate.
Being able to recognize utterances, while certainly an important
component, is merely one of many capacities that are called into play when
humans converse with each other. Thus, if "talking to computers"
is to convey any significant similarity to "talking among
humans," existing recognition technology is obviously inadequate.
What's Missing From Today's Speech Recognition
Speech recognizers do not currently account for semantic information
conveyed by intonation, volume, pitch, pace, and other prosodic
characteristics of conversational speech acts. Nor do they provide any
means to model user intentions.
Additionally, and paraphrasing speech industry guru Bruce Balentine,
speech recognizers have no memory: they have no way to relate what was
recognized last to what is being recognized now. Each and every utterance
is treated as an independent and unrelated event.
Commercial vendors of applications using conversational dialog
interfaces typically compensate for these shortcomings through skillful
design of their dialogs. They know that speech recognizers are designed to
be the "ears" of their applications, and that they are
responsible for designing the "brains."
Speech recognizers will probably advance to account for factors such as
intonation, volume, and pitch in the not too distant future. On the other
hand, recognizers are not likely to incorporate user models or any
significant sense of memory anytime soon.
Additionally, there are limitations to what can be overcome by creative
dialog designers. Not even the world's most gifted dialog designer could
create an interactive system capable of continuously compensating for the
absence of "brains." What is still missing in speech recognition
today is its ability to adapt. In order to progress towards human-like
conversational interaction, speech systems will have to be able to
dynamically learn from their "experiences," and adapt their
behavior appropriately.
A New Approach: Atomistic Parallel Listening
Atomistic parallel listening is a promising new approach in speech
recognition. Basically, parallel listening entails using both a
conventional primary grammar in conjunction with another secondary grammar
that looks for "atomistic" components of speech. Think of an
atom as essentially a smaller unit of speech. An individual word, a
syllable, or even a phoneme serve as examples. In contrast to the primary
grammar that anticipates entire utterances, the secondary grammar looks
for constituent atoms within the utterances.
The basic idea is to use the secondary grammar to extract atoms from a
user's utterances that are predictive of his intent. Here is how it works:
Each time the user says something that is recognized on the primary
grammar, the recognizer returns a natural language tag. These tags
represent the "meaning" that the user intended to convey.
Behaviorally, tags lead to the consequences of the user's utterance: they
cause the system to take some appropriate action as a consequence of the
recognition result.
As such, each correct recognition result on the primary grammar
represents an opportunity for the system to learn by example. While the
primary grammar is processing the utterance, the secondary grammar is
doing so as well. Learning takes places by correlating what the secondary
grammar "hears" with the natural language tags returned by the
primary grammar. Correlations are made for each atom that is returned by
the secondary grammar, both individually and collectively.
Consider an example in which syllables are used as atoms, and the
following recognition results are obtained on the primary and secondary
grammars:
Primary:
Schedule an appointment for tomorrow
Secondary:
Sked joule an up point ment for to mar row
Again, each time the user says something that is recognized on the
primary grammar, the system has an opportunity to learn. If the user
accepts the subsequent action taken by the system (and the user does not
cancel the action), the atoms detected by the secondary grammar can be
assumed to be predictors of that action. The atoms returned by the
secondary grammar, both individually and collectively, provide cumulative
evidence of what the user intends the system to do.
Thus, whatever actions the application might take as a consequence of
recognizing Schedule an appointment for tomorrow become predicted
by the individual atoms, Sked, joule, an, up, point,
ment, for, to, mar and row.
Furthermore, since atoms such Sked and joule are strongly
correlated, the presence of both of them provides additional evidence that
the Schedule an appointment for tomorrow tags are indicated.
Advantages Of Parallel Listening
There is a facetious saw in the speech industry: Grammar-based systems
never fail, as long as you know what the user is saying. The problem, of
course, is that one can never anticipate everything a user might say, or
even the highly variable ways he might say the same thing. Speech
recognizers do a very good job under sound studio-like conditions and when
speakers only say things that are represented in the grammars. But in the
real world, particularly in telephony applications where cell phones come
into play, recognizers are doing well if they reach 80 percent recognition
rates. Even if recognition accuracy rates were at 95 percent, one out of
every twenty utterances would fail. And this sort of behavior, over
repeated interactions, makes for very tiresome conversation.
The greatest advantage of this parallel listening approach, therefore,
is that it provides a way to compensate for recognition failures, at least
when they occur on the primary grammar. Each time there is a failure on
the primary grammar, the application can use the atoms detected by the
secondary grammar in order to formulate a guess as to what the user
intended by his utterance. The best guess is experientially indicated by
whatever natural language tags these atoms have reliably predicted in the
past.
Initially, a parallel listening application has no atomistic knowledge.
It must acquire this knowledge by observing the user. It relies on the
user to say things that are in the primary grammar in order to return
actionable tags and perform some task. But every time there is a
recognition success in the primary grammar, the parallel listening
application compiles additional atomistic evidence that can later be used
to infer the user's intent.
Another desirable advantage of this approach is that it allows users to
change the subject more naturally, even within a directed dialog. Failures
on the primary grammar will always defer to the results of the secondary
grammar. Thus, a user can change the subject of conversation and still
expect the application to "follow along." While the
application's primary grammar was listening for some particular,
predefined input, the atoms obtained from the secondary grammar will
always point to an action that has been appropriate in the past.
Another example:
User: Schedule a meeting for September 15th.
App: At what time? (Application must obtain the time
of day to proceed with the task; directed dialog primary grammar is
listening only for expressions of time.)
User: Give me the weather report for San Jose. (User
changes subject and primary grammar fails; application determines the
tags historically predicted by atoms of the utterance obtained, and
proceeds.)
Is "Talking To Computers" The Next Step?
So, what is the next breakthrough in speech recognition? The answer may be
surprising because this is actually a trick question.
Speech recognition engines will continue to evolve and improve. Their
developers are likely to expand their recognition abilities to more
prosodic components of verbal behavior like those mentioned above.
However, advancements in the field of speech recognition technology alone
are unlikely to promote breakthroughs in conversational dialog systems.
This can only result from advancements in adaptive systems.
When it comes to "talking to computers," expectations have
been set for many by C-3PO and HAL. These are our visions of true
man-machine conversational dialog. But conversational dialog is not a
speech recognition problem -- it is a machine learning problem. And while
methods such as parallel listening are important advances in the right
direction, the next breakthrough in speech recognition cannot truly begin
until people both in and out of the speech industry come to accept that
recognizing utterances is just a single step.
Walter Rolandi, Ph.D. is
director of Applied Research for Conita
Technologies, Inc. Conita (pronounced kah-night-ah) is a provider of
voice-driven technology for mobile productivity. Conita’s Personal
Virtual Assistant (PVA) software provides just-in-time knowledge,
awareness and transactions for mobile professionals by voice from any
telephone, enabling access to business application software, enterprise
messaging and communication systems.
|
