Published: 4 February 2018
Slack's Amir Shevat talks about bots, and why you can't just port existing services to a bot, and about the power of conversational interfaces.
Gerry Gaffney
Gerry Gaffney: This is Gerry Gaffney with the User Experience podcast.
My guest today is Director of Developer Relations at Slack. Previously he managed Google’s Startup Outreach program, helping developers around the world design and build better products.
He’s author of the book Designing Bots: Creating Conversational Experiences Amir Shevat, welcome to the User Experience podcast.
Amir
Thank you, thank you for having me.
Gerry
I guess bots are certainly one of the current hot topics, although I noticed an article in Wired magazine the other day saying bots are dead. Along with machine learning and AI, I guess, how did you get into bots or chatbots and why?
Amir
I started with bots and chatbots a while back, before 2000, I think in 1999. I was working on an SMS system in Europe and I was bored because I was the only one on that SMS system.
So I started creating bots to be my friends. I used to write bots that would answer me automatically so I had, one of the first bots that I used was a “yes man” and everything I sent to that bot, it agreed with me, which made it a little bit more delightful not to be alone in that system. So I’ve been writing bots for a very long time and recently at Slack I’ve been helping developers build work fills and integrations that are actually useful bots for our platform.Gerry
You sounded a bit freakily like J.F Sebastian in Blade Runner there, you know where he builds those toys to talk to him when he gets home from work.
Amir
It was definitely that experience, yeah it was definitely that experience. I was very lonely. They shipped me to Romania. It was cold in winter and lonely and I created all these bots to be my friends.
Gerry
I guess one of the things that’s interesting about bots is that the barrier to entry, certainly for writing basic chatbots noadays is very, very low, isn’t it? It’s very, very easy to begin writing chatbots.
Amir
Yes, like the “Hello World” version of a chatbot is very simple. But once you want to build a comprehensive product using a conversational interface, it becomes much more complex and I managed to write 150 pages on the topic. So if it was easy it would be a pamphlet rather than a book.
Gerry
Yeah the book is really interesting. Coincidentally there is also a book called Zero to Bot which I looked at at the same time and it’s a very, very short, I don’t know 15 or 20 page walk-through of how to get something up and running. Did you come across that on your travels?
Amir
No but if it’s better then you should probably get that. [Laughter.]
Gerry
No it’s shorter but no it’s a very, very basic, I mean I actually myself when I got interested in bots I got both books, yours and the very, very brief one and they both have their place and I enjoyed yours very much of course which is why I’ve asked you to come along today.
First of all I guess, what is a bot? Can you give us some sort of definition of a bot and a chatbot; are they the same thing or are they different?
Amir
So, yeah I consider bots and chatbots the same. I think at the basic level a bot is a user experience, a new user experience. So we moved from using pencils to using computers and from using computers to the Web and from the Web to mobile. This is a new way to expose services and products in hopefully a delightful way and the key reason here is that users have moved into chat. If we look at our phones, most of the popular apps we have there are chat or communication apps. So it makes sense to bring software into where we are already. So if we’re talking to our mates over Messenger or Slack it makes sense for us to expose and consume the tools and services that we use every day in that manner and this is what bots are.
Gerry
Yeah, I saw some interesting stuff the other day that in Australia the number of telephone calls made in Christmas just past is the lowest in many, many years. It’s been trending upwards for a long time but it’s actually fallen and of course a lot of that is because of people using different, not using the traditional telephone to contact oversea family and friends but using various other types of interface and I guess chat of one sort is part of that. I’ve certainly noticed in my own life that I have much more asynchronous conversations with overseas family. Do you find that that’s a general trend?
Amir
Yes, definitely and I think with millennials it’s even truer. If I call one of my colleagues here, they will be surprised. I would need to apologise for calling them; why don’t I send them a text or Slack or anything else of that sort? Like, it seems to be coming intrusive to call someone.
Gerry
Yeah, so what are chatbots good for?
Amir
So I think this is the great question. Chatbots are, because they’re a new user experience they provide an opportunity to disrupt the market. So if we look at past revolutions in user experience, for example when we moved from desktop to the Web, there was a small company that wanted to do customer relationship management over the Web and a lot of people thought that that was a bad idea; who would want to consume secret information like my list of clients over the Web? That doesn’t make sense. But now the biggest building in San Francisco is actually Salesforce which based their entire business on moving CRM and customer relationship management systems into the Cloud. So I think the key here is that we should look at new experiences and see if we can transform them if they perform better in conversational interfaces.
And that’s the key thing that I want to make, it’s like look at a process that currently suck and see if you can make it better with a conversational interface. And if it’s better, if it’s performing better than mobile or Web or previous interfaces then it makes sense. Things that we’ve seen that work well are things like expense reports, like notifications, like DevOps, a lot of engagement with e-commerce has been moved into conversations. And not just textual conversations, with Alexa you can actually, Google Home, you can actually talk verbally to a bot and do your commerce there. What we’ve found is that, at least for our household, 20% of our shopping has moved into Alexa and Google Home because it’s much easier and more convenient to use this conversational interface.
Gerry
Yeah it’s a whole new world, isn’t it, the conversational stuff. One thing that I’ve found quite scary was the fact that I can go back over all of my search history on my Google Home device. Have you done that on your own Alexa or Google Home?
Amir
Yes, but I used to work at Google so I know that’s part of the deal.
Gerry
Yeah there’s a lot of swearing in our household, you know people will say something to Google and it doesn’t understand it and then it’s repeated in more and more basic language until it degenerates into swearing.
Amir
That’s actually a great point. I talk to a lot of bots founders and they say that they test the Mean Time To “I love you” or the Mean Time To “I hate you.” So how fast can a user actually express emotions and what they found is that users tend to express emotions to bot in a much faster way than apps or websites.
Gerry
And I guess this even goes back to ELIZA, I mean would you consider ELIZA from 1964 to be a bot? The psychology thing?
Amir
Yes, of course, one of the first bots out there. Definitely. I played with later versions of that and even tried to break it, because I’m an engineer so I like to break things. I definitely think that that is a good use case for a bot.
Gerry
And that was very simple. I mean it was only a bunch of pre-written answers that responded to certain key words, right?
Amir
Yes. I think I wrote an article about that that I think… So there’s a Turing test. The Turing test states, by Alan Turing, one of the forefathers of computer science, tests that a true intelligence will be...
Gerry
Indistinguishable.
Amir
The test for true intelligence, yeah the test for true intelligence is having a conversation with software and not being able to distinguish between software and human. I think that bots should not pass the Turing test. They should actually pass the beer test and the beer test is much simpler. Do I want to take that bot to a beer after I get my service, right? Is it a delightful bot rather than is it a human-like bots? So I think we should strive for delightful experience rather than human-like experiences.
Gerry
Yeah, I must admit I’m a bit cynical when it comes to delight and people who know me will know that whenever people in conversation say “delight” I kind of shudder a little bit because I often think are we really delighting people or are we just trying to be you know can we just be utilitarian, isn’t that a really good start?
Amir
So I’m from San Francisco, we say delight many, many times. [Laughter.] But I think we actually mean it and the key here is that you can be utilitarian but if you add utilitarian and a little bit of empathy you can get to a better service. And I think for the rest of the world that’s true as well, like if you pour a little bit of niceness and if we just be nicer to each other we could provide better services.
Gerry
Okay, niceness I like.
Amir
Okay, perfect. I think that this is a good ethical question, like how do we treat our software? I think that if we treat our software right we’ll treat each other right. So if we swear less at Alexa and Google Home we can possibly be nicer to each other at the end of the day.
Gerry
And what’s an example of a bot badly used or, you know, trying to do the wrong thing with bots. I mean I’m not asking you to pick a particular company perhaps but can you give us generic examples?
Amir
So generic examples are, and I’ve seen that when I worked at an Android as well, is trying to port an old user interface into the new user interface without thinking about it. So we’ve seen that when we tried to move into mobile and there were a lot of developers that tried to cramp web pages into small screens. So they saw the mobile as a small browser. But that’s not true. True Web pages are different, right? They need to be designed differently. They need to interact differently. The same thing goes with bots. I think one of the key failures that I’ve seen is trying to cramp the functionality that you previously had into a conversation and that turns into a very cumbersome conversation. So if I had a form with 30 questions it would be a nightmare for me to ask you these 30 questions in person, right? It would be very cumbersome, so think about how can you optimise for this new user experience? Just like you optimise for desktop.
Gerry
I think sometimes bot interactions can let one down. I mean, they seem to be quite clever and sophisticated and I’ve used an example before of talking to an airline’s bot and looking for something or other that comes up and, you know it offered me a weekend trip to Dublin as if that was some sort of thing that one would do from Melbourne, you know? And it was sort of, everything it had done up to that point was reasonable and then suddenly it’s offering me this insanely packed, you know, 26 hour each way journey for a weekend break. It’s very disappointing, isn’t it, when we come up against I guess the limitations of a bot?
Amir
Again, I think it’s the limitations of us as humans like coming back to the empathy and being a little bit more empathic, that should have been probably programmed into the bots, right? Humans don’t like to travel more than eight hours in each direction. And I used to live in New Zealand for two years so I definitely relate to what you’re saying, like travelling around the world so far away is probably not the best choice that the bot could have made but at the end of the day a bot is only as delightful as the service it exposes. So think of, again, the bot is a user experience. The search functionality is broken in your example.
Gerry
Should a bot try to emulate or pretend to be a human?
Amir
No. I think it shouldn’t and I think it’s counterproductive. It’s counterproductive because it’s really hard and then it creates a bad experience. It’s counterproductive because I think it’s questionable from an ethical perspective. And it’s counterproductive because the service you want to expose might not require that. I think the key is to look at the funnel; what is the task that you want to achieve and what is the least amount of steps that are needed to be taken to achieve that task? People don’t want to speak to human-like bots, they want to get shit done. So I think that’s the key aspect of a good bot.
Gerry
Yeah. I guess, you know, related to that somewhat is the idea of handover points. It’s always interesting when you have to get handed over from one agent to another, whether that be a human to human or human to bot or bot to human or whatever. Is it particularly difficult to design that part of a conversation?
Amir
It’s not trivial but it’s pretty useful. So I think the key here is to understand that bots cannot answer everything but they can answer a lot. So, for example, if you’re building a support use case bot and you give the bot, using AI, you give it a dataset of a lot of frequently asked questions. It’s very easy for the bot to address common questions but once it comes to uncommon questions it’s extremely hard for the bot to address those. So handing off to a human makes a lot of sense in these use cases. So I think it’s the 80/20 rule; have the bot do the what’s easy and what humans don’t like to do and then refer to humans for the complex tasks that are actually interesting.
Gerry
Now, I was talking to somebody recently who’s building bots for their particular organisation and they told me that the people writing the content for the bots were copywriters and that they’d specifically chosen copywriters to write the scripts and so on. Is that a sensible approach or should that be done by technical writers or by marketing people or by devs, or who should be doing that?
Amir
Definitely not devs. I’m an engineer and I don’t know how to write copy. I think we’re seeing the emergence of a new profession which is like designer, visual designer and words designer for bots, and I think we’re going to see a lot of people with an English major joining the Silicon Valley and the rest of the world, tech world, in creating nice experience. I’m not going to use the word delightful again in this conversation.
Gerry
That’s okay, you can say it! [Laughter.]
Amir
…nice experiences for humans using bots.
Gerry
I guess from a UX or a UI designer’s point of view the rules, if you like, or best practice for designing a bot are very similar in many ways to the rules for designing a good UI?
Amir
Yes, I think yes, using user research, A/B testing, getting feedback. There are certain things that you need to take into account when thinking about this user experience. It’s short and contextual, meaning that humans are already talking to each other using this medium of conversational interface. So the software when you’re designing software that is coming into the conversation it’s already joining a well-known and well-formed mental model that the users have. So I would actually try tactics like the Wizard of Oz in order to design a good bot. If you, for those of you who don’t know what the Wizard of Oz technique is it’s having a person mimic the bot for the test, for the design test in order to get a lot of feedback from the users.
Gerry
Yeah, I’ve personally found that very powerful. I mean you can mock something up very, very quickly and you could look at things like conversational pauses and, you know, whether the bot is using the appropriate language or whether it gets negative reactions and so on and so it’s a great testing mechanism.
Amir
Exactly.
Gerry
You talked earlier about the mistake of porting, if you like, something form the Web to something to mobile or something from mobile to an app. What about, and it’s something that I’ve often struggled with in the apps, in the bots that I use, the ones that support either text or voice are trying to sort of juggle two modalities as if they were the same thing; do you have any thoughts on that?
Amir
I think that’s a big challenge and it’s still unsolved. I think talking to software and texting with software is very different. I think texting and conversational interfaces that happen in messaging apps are much simpler currently to implement. So, let’s say I need to choose from N items, it’s much easier to add a drop-down in the conversation which is actually text than actually listing a list of items with voice.
So I think at least in Slack for the last year and a half we’ve been adding more and more controls that help you manage the conversation; things like buttons, like drop-downs, like forms. So we’re actually calling these bots slack apps, so you can, these are like small mini apps that live inside the conversational interface with rich interactions that let you facilitate workflows.
Gerry
To change topics a little, one of the issues with chatbots is that unlike apps there’s no real central repository, like if you’re writing apps you can launch them on you know the Google Play Store or Apple’s Apple Store or whatever it is, there is no such repository for apps, I mean maybe there shouldn’t be but it means people don’t necessarily have an easy way to get their, to insert their bot into a market. I mean is that the wrong model to be trying to do that in any case and should it be just a part of an overall service? Sorry, that question’s probably a lot longer than it should be.
Amir
No, no, I have managed to process your question and I’m not going to answer with “I’m sorry, I don’t understand” like a bot. I think that is true to some extent. So if you’re building a bot there’s no one single place to put your bot but there’s multiple places. For example, we have an app directory. We actually have a bot directory if you go to slack.com/apps you’ll see the list of bots that we have there. Kik has a list of, has a bot store. Facebook is experimenting with discovery. I think at the end of the day, discovery is still not optimal in our market but all the big players, including us, are thinking about it. And I can give you an example; we just added the ability to add bots from within the client because we think a directory might not be the right thing that people don’t necessarily go to a directory to look for bots. Maybe they just want to add the bot from within the Slack client. At Kik I’ve seen a really nice interaction where you @ mention the bot and even if you haven’t had the bot installed, you immediately start a conversation with it so discovery actually happens in the way of starting a conversation.
Gerry
You just reminded me there of, I mean I guess a truly useful bot might just be in the background until such time as you needed it. But then the example that sprung into mind was Microsoft’s Clippy, you know, “It looks like you’re writing a letter.”
Amir
I won’t comment on Clippy.
Gerry
Okay.
Amir
I used to work at Microsoft so I didn’t work on Clippy but yeah Clippy was an interesting… it generated a lot of feelings from a lot of people.
Gerry
Yeah there were a lot of memes going around with various alternative versions of the wording but it was kind of the right idea, wasn’t it? I mean you’re doing something and an agent unobtrusively notices that it might be of some assistance and just gently offers you that assistance.
Amir
Yes definitely. I’ll give an example, we have a health finance channel so if you have any questions internally in Slack about finance, about your 401(k) or retirement you just post those questions there and there’s a box that if it knows the answer it immediately answers with a comment. And if it doesn’t then the humans who monitor that channel answer it. So I think augmenting human conversation with bots is definitely something that we need to explore as an industry.
Gerry
One day will we connect all the bots and eliminate the humans?
Amir
Oh! Great question. I don’t think so. I think that we’re very, very, very far away from general intelligence AI, very far away, and may the gods of the future forgive me but I think we’re very far away from overlords managing our every task in our life. I think we’re, I think AI should be called artificial smartness because that’s a better definition of what these algorithms are. They’re smart, they’re not intelligence. There is no consciousness.
Gerry
Is it necessary these days to have AI or machine learning behind a bot? I mean something like ELIZA clearly didn’t, it was purely a pick list, if you like, of response but is it fair to say that most bots or all bots these days have got some AI behind them?
Amir
No, no. Most bots do not.
Gerry
Right.
Amir
I think most bots are now domain specific bots that help you realise simple and short workflows and that’s why you don’t really need AI. So, for your example, like talking to the United bot or to Qantas bot you would just need to post your flight number and get notifications if your flight is delayed. There’s no AI that is required during that process. So for short and useful workflows that are not verbose you probably don’t need AI.
You need AI when you need to have a complex conversation and you need to extract the intents and entities from the conversation.Gerry
You mention intents and entities and, you know our conversation has been and I guess will be on this podcast anyway not an engineering one but you do talk in the book about the specific details of how to create entities and how to extract the objects that are being discussed and so on. There is quite a lot of, I mean I guess engineering detail behind the construction of a bot, is there not?
Amir
So again depending on the use case, simple use cases like an FAQ bot, you probably don’t need a lot of intelligence. The bot is going to be as complex as the service that you want to expose. So if it’s a coffee bot it’s probably going to be easy, if it’s a performance review bot, it might be a little bit more complicated. So I think the bot is only, will serve as the complexity of your product and hopefully will simplify it. But at the technical level, it’s not hard to get started which is kind of fun and democratising this type of engagement.
Gerry
One thing that struck me about the book and you did allude to it somewhere, I don’t have the quote to hand, but you did talk about the fact that the information you’re providing in some cases would almost by definition be obsolete by the time the book was in somebody’s hands; how did you juggle with that? I mean I think the book is still a highly relevant and will be for a number of years but obviously things where you talk about this specific service and that specific bot and this particular API and so on… Did you struggle with that, with the decision as to what to include in the book?
Amir
So the first tip is always put in the contract the Version 2 of the book, so you’ll always be relevant. [Laughter.] I’m joking. But I think the key is that we’ve been having conversations for I think 160,000 years more or less. So a lot of the knowledge around building the good conversations is not going to change in the next ten years. Things like context, things like memory, things like how do we interact and prompt the user to give the right decision? So if I’m saying to the user “When would you like to meet?” versus “Which date would you like to meet?” I’m prompting the user to actually give me the right type of information. So things like that will never be deprecated hopefully until we learn a new form of conversation between humans.
Gerry
For people who are listening who might be interested in this space, you know they’re working in UX or web design or whatever, and they think this chatbot thing sounds interesting, how do I get started? What would you recommend they do?
Amir
So first of all build a useful use case. I can’t say it enough; a bot is only as good as the service it exposes. So start thinking about something that is useful that you want, something short and contextual that you want to expose using this bot. And then there’s a lot of ways to do that. If you want to do it on Facebook there’s simple tools like Chatfuel that could help you do that. On Kik the API is pretty simple if you want to address teen use cases. If you want to do bots for work and make your work life better then Slack is probably, and we have an API, api.slack.com, and most of these APIs are pretty simple to start with and once you’ve built that use case, start using it. And once you start using it you’ll learn a lot and then you’ll change it and iterate it and then you can make it into a product.
Gerry
And one of the things that occurred to me when I had a play with building a little bot, I didn’t have a specific use case and I thought I’ll put something around uxpod for just exploring what the content is, and what it really exposed to me was how lousy my metadata and well my metadata in the podcast was and it was kind of, it was one of those unanticipated consequences of starting to put the bot together.
Amir
Yeah. I think that’s true for every service. When I built Android applications I had the same thing like my data structure is not in the right way, right? So I think the user experience exposes the fallacies of the back end usually.
Gerry
I’ll just remind listeners that Amir’s book is called Designing Bots: Creating Conversational Experiences and I would recommend for anybody interested in this space, who’s already in it or looking to get in to it that that book is a very, very good starting point. It’s very comprehensive and thorough and clear.
Amir Shevat thanks for joining me today on the User Experience podcast.Amir
Thank you for having me.