Constance Johnson ABC NEWS

Wally Brill has always loved robots. As a child he would talk with his toys and was disappointed that they never spoke back to him. Eventually, he started talking to robots and teaching them to talk back to him. He worked for several years as a music producer before getting to fulfill his childhood dream. Brill has been teaching robots to talk for 20 years. At the moment, he’s focusing on training others entering the popular field of conversation design. Here he explains how a a career in music led him back to his childhood passion. 

You’ve said that you have always been interested in talking with robots. How did you end up at the New School and how does that background inform your work in conversational design? It’s not really a technical school.

You know, I grew up with Robbie the Robot, B9 from “Lost In Space”, the Computer in “Star Trek” and even Hal 9000 of COURSE I’d want to talk to robots! As for The New School, I went there to study electronic music composition and production.

I heard about a company called Nuance that had brought speech recognition to market and I got hooked.

Do you mind sharing your trajectory, and how you came into this position?

So I mentioned electronic music. In those days we were making synthesizers from oscillators and filter banks, recording snippets of sound and combining it with “found” audio samples. So my great love was audio. Over the years, I became a music producer, working mostly in the U.K. with artists like Thomas Dolby, The Sound, and Ofra Haza. Somehow that all led me to wanting to make an interactive opera where you could steer the action like a “choose your own adventure” story. I heard about a company called NUANCE that had brought speech recognition technology to market and I got hooked. Instead of the opera (which remains unfinished) I started designing ways in which people could achieve tasks through conversation. This was in 1999.

What is the position exactly? How are you advocating for conversation design? What does that mean exactly?

In my current role at Google, I run a lot of workshops and train people entering the field. The discipline of conversation design is exploding, and there’s a real demand for talent.

It was time for a consultancy to design VUIs.

VoicePartners sounds amazing. Was that working on the telephone at that time?

Yeah, it was. The head of user research at Nuance and I decided in 2002 that it was time for a consultancy to design VUIs (Voice User Interfaces). We set up VoicePartners with a small team of the best designers in the business. Several of those designers are currently colleagues at Google. 

To answer the second part of the question, yes, most of the work at the time was in IVR (telephone systems). Though people love to hate them, if they’re well designed, they can be really useful, effective, and even elegant.

How has VUI design changed? I am sure it has since people use the voice assistants for just about everything now instead of telephone interactions.

As the technology has accelerated in the last few years, the interactions are more natural, more open- ended, more intuitive, and more successful. One big change is the huge leap in TTS (Text to Speech) or computer generated voices. They’ve become nearly indistinguishable from human voices. And they just keep getting better.

Last question: The future of this as it relates to corporates. Where is it headed? Why isn’t it there yet? It seems like everyone is still sort of figuring it out.

Great question. I spend a lot of time talking to enterprises large and small and giving workshops on how to design for this technology. Look for more use of intelligent assistants in the contact center. While some have been trying to figure it out, others are pushing forward. Some of the most creative are advertising agencies who see this as a completely new interactive medium. I also have the pleasure of working with a number of game design companies who are really pushing the technology in fascinating ways. Watch this space. 

Some of the most creative are advertising agencies who see this as a completely new interactive medium.