Published: 1 February 2008
Gerry Gaffney:
This is Gerry Gaffney with the User Experience Podcast. My guest today managed ethnographic and laboratory-based user studies in Europe, Asia, Australia, North and South America. He’s worked for a variety of high-profile companies, including a stint as International Design Manager for the Platform Design Group at Yahoo! in USA, Hong Kong and London. He’s been a research director at ISER, a not-for-profit think-tank in Rio de Janeiro. He has a PhD in Psychology from the University of Chicago, where he specialised in ethnographic methods, and he is currently User Experience Research Manager at Google.
Patrick Larvie, welcome to the User Experience Podcast.
Patrick Larvie:
Thank you, Gerry, it’s a pleasure to be here.
Gerry:
You’ve told me that you got interested in research and design by accident. Can you give us a bit of background about how that all came about?
Patrick:
It was entirely by accident. I was working in Rio de Janeiro on my PhD. I had done an ethnography in the Ministry of Health, trying to understand how social scientists were understanding and operationalising ideas of illness. And I was very interested in the way they were using technology. But what really made the shift was the fact that I had done institutional ethnography. I was at an institute there that was approached by the police in the state of Rio de Janeiro, strangely enough, who were interested in understanding how police data were produced. What they wanted to understand was the gap between the day-to-day activity of police officers and the kinds of statistics they saw in the readouts from machines. It’s interesting to note here that the police data analysts had no real access to the data inside the machines and certainly didn’t understand what the police officers were doing.
So it turned out to be a long experiment in which we had to follow police officers around. I spent about nine months working with beat police officers in Copacabana, in Brazil (in Rio) and we ended up kind of re-designing a lot of the information management system in the police force. And that meant experimenting with data analysis’ and data collection tools, which meant putting a lot of people in touch with technology in a way that made some people very uncomfortable.
And therein lies my interest in technology, its power to change institutions, its power to I think ultimately change or effect change on broad areas of our culture.
Gerry:
Did that work that you did with the police force result in positive outcomes in terms of a better understanding by the police of what the crime stats were telling them?
Patrick:
It did, in some really unexpected ways. It had some very positive outcomes in that I think we were able to… As a little bit of background information here, most people probably are aware that Brazil for many years had a military regime. And the purpose of police forces in authoritarian regimes is basically to protect the state, not to protect the citizen. And that ends up setting up very inefficient systems, where it’s not really clear what counts as good information, good information is information that leads to routing out people who are politically suspect, it doesn’t do a lot for figuring out the day to day problems of cargo theft and muggings.
And one of the big problems in Rio was exactly that. Rio de Janeiro was having an epidemic of cargo theft. And we were able to figure out that one of the problems that the police were having was that they could understand where cargo theft was occurring, they could understand when it was occurring, but they couldn’t understand the two things at the same time. And that’s because of legacy data systems that were put in place in the mid 1970′s.
So, we were able to help them unpack a lot of what they didn’t know and ask more intelligent questions of the data that the police were producing. So in that sense it was very positive.
Gerry:
From a user experience perspective, Google is I think a very interesting phenomenon. On the one hand I think a lot of usability people point to the Google search engine in particular and say here is the epitome of user experience and good usability design. But in fact Google is very much an engineering company. Is there a conflict between the engineering focus and the user experience focus?
Patrick:
I don’t think there is, and what I would like to do is actually point out is… Google is different from many other companies. It’s different from many of the consumer-facing companies that people are accustomed to seeing on the internet. Google’s mission is to organise information and make that information accessible and useful to people everywhere. And one of the interesting characteristics of Google, and one of its distinctive characteristics, is that it really understands itself as, in part, a technology company. So it’s experimenting with new ways to organise information and to make that information accessible to people.
About the search box, I’d like to comment a little bit about the search box. A lot of people point to it as an example of the epitome of simple design, and a very powerful, simple interface. What people don’t recognise, and I think we have very short memories in this respect, is that search on the internet is a relatively recent phenomenon. Most search engines were not very powerful, as I think a lot of people who have tried to use the search box on proprietary sites have discovered. The search function is often the worst, and it’s the last resort. If you can’t find what you’re looking for if you’re going to Qantas or to United, you may try the search box, but it’s not likely to be that great.
So search is a relatively new technology. Today that we regard that simple interface as the epitome of a usable, elegant design really speaks to the fact that we’ve become accustomed to a radically different approach to organising information, and that’s what Google is really about. Search is a new way of thinking, not just about how to organise things, but about how to retrieve things, how to remember what they are.
Gerry:
I think Google has had… this may sound very conservative, it’s had a very interesting affect on the teenagers I encounter. You know my older son, Brian. He doesn’t bother spelling anything, he just types it in, in some random quasi-spelled fashion and the assumption is that, oh, Google will fix it.
Patrick:
Right. Well, I hope that that’s true. I think that points to an interesting user problem. And one of the things that I’ve learned in my career is that you must pay sometimes to the most basic problems or the most basic challenges that we encounter.
One of the problems with a search interface, and this is very different from, say, directory structures, is that people have to be proficient at input. And English shares, I think in common with many other languages, that its spelling is not intuitive. So spelling is a real barrier. For search to be effective, one of the things that has to happen is that we have to account for the fact that lots of words aren’t very easy to spell, and there are lots of variants for the same word. So [laughs] we will continue to try to make sure that Brian doesn’t have to know how to spell everything on his own.
Gerry:
That’s good. That’s going to look bad on his résumé, isn’t it?
Patrick:
Well, he should… Google offers some tools that might help with that. [Laughter.]
Gerry:
When we met late last year, you spoke about Google focusing on a layer of user experience below the user interface, and I guess your previous comments have alluded to that somewhat, but can you tell me a little bit about what’s below the level of user experience, and how can you focus on that?
Patrick:
Sure. I think that search is actually one of the greatest and most powerful examples of how Google’s approach is both to demonstrate a new strategy for managing, for organising and managing information, and for showing that that strategy can apply to more than one domain. So, to illustrate that: Five or six years ago, lots of search engines weren’t that great because there wasn’t a whole lot to search through and it was probably in many easier to look through a directory or to look some kind of editorialised version of what there is to do in the realm of travel, or what there is to know about Australia or the United States, than it was to do a search on something like Google.
Today that’s no longer true. We’re applying that same metaphor to other things. For example, email, the idea that people don’t have to delete their email. What we would like is for people to think about email much as they think about other things. So, they can retrieve the message they want by entering some keywords that might remind them of the conversation or the person or the theme, but they don’t necessarily have to keep track of exactly where that message is. That’s no longer necessary. So, that’s the idea of what’s slightly underneath the interface.
And Google’s pushing that in some interesting ways. If you think about things like Google Docs, or the desktop search function, what’s important there is the idea that you don’t have to remember that you stored a particular document in a particular folder and it’s on hard drive, or remember which email account you were in when you wrote a message about a reservation that you wanted at a restaurant. The idea is that you should be able to use a simple search function to organise all of that.
And once you have dislodged place as a key aspect of… organising and finding information, it’s also possible to push at other aspects of finding and organising information, such as who created it, or to whom does it belong. And that’s something that we’re working on with Google Docs. I don’t know if you’ve had a chance to use Docs or Spreadsheets, but I think one of the most interesting aspects of those applications is the fact that you can work on them, they encourage collaboration. So not only do I not have to remember what’s on my hard drive, I don’t necessarily have to remember if I created the document, because if it’s shared with me I might be able to find it through some other way.
So, that’s kind of what I meant by looking at what’s underneath… the layer underneath the interface.
Gerry:
You ask whether I used Docs, I use them fairly extensively and in fact I now do all my accounting on Google Spreadsheets.
Patrick:
Oh really?
Gerry:
My only worry is that if Google’s going to go bust over the next couple of years and all my financial data will be gone.
Patrick:
[Laughter.] I’m hoping that doesn’t happen.
Gerry:
That’s very interesting what you say about Google Mail. I switched to Gmail when it first came out on a trial basis, and I guess like everybody the absence of a delete button was sufficient to drive me away for a while. And now I’ve been using it for a year or so and it’s exactly as you say, I don’t… I’ve had to sort of let go of my own need to organise my mail, and being to sort of trust that the application is clever enough to do the organisation for me, and let me retrieve it more naturally. So it’s a very different paradigm.
Patrick:
It is, and I would actually… I had a similar experience; I initially reacted very negatively to Gmail. And I think this is a very interesting point for people in the HCI [Human-Computer Interaction] community, that a lot of people’s initial reaction to the application is maybe negative. And it’s negative because what Google’s trying to change is the basic, the underling metaphor of organisation. So if, before, in my email applications I had to remember which file I created, or… you know, in email messages, the content description is absolutely key – that message line, what this message is about – that’s an artefact from 1950′s and 60′s era information management, information science. There’s no real need for us to maintain that. And in trying to shift away from these older paradigms of information management, I think it’s reasonable to expect that initial user reaction isn’t 100% positive, and we need to be able to look a little deeper than that. So the typical paradigm of run a usability test and if people don’t like it then declare it a problem may or may not apply so easily there.
Gerry:
I often think of the example of SMS messaging. If you’d looked at text or SMS messaging before it was launched in a usability lab you would surely have concluded that it would never take off.
Patrick:
It’s a usability disaster, and it’s an amazing success.
Gerry:
When you talk about not having to know where things are it also implies a level of intelligence that’s not quite there yet. For example, today… somebody from User Experience magazine editorial board sent me a pointer a couple of weeks back to some clipart online, and I can’t remember what the email actually said. It didn’t actually say “clipart” so what I want to be able to do is look for a piece of email that’s got “something to do with” online photographic resources, but none of those words may exist in the email, so I couldn’t find it.
Patrick:
That’s true. This technology is not perfect, but I don’t really want to go back to the series of drop-down menus where I had to remember title, author, date, or those kinds of things to retrieve email messages. And I remember applications like Eudora quite affectionately. But I remember searching around for messages in my Eudora that just used to drive me crazy. So I think that we’ve made a shift. The shift isn’t 100% successful all the time, but it’s a pretty sizeable difference.
Gerry:
Google has so many properties, partly because of its acquisitions program, I guess. It’s got Analytics, Picassa, Gmail, Google Docs, Google spreadsheets and so on. If you look at someone like Microsoft, they’ve been able to impose a reasonably standard look and feel right across the product range, but as you move from Google product to Google product they are often starkly different or strikingly different. Is this something that Google wants to address?
Patrick:
I think this is a really good observation, and I think it’s an observation that’s related to your earlier question about what kind of company Google is, if it’s an engineering focused company and is that in some way at odds with creating an excellent user experience. I’m not sure that it is, I think Google’s a different kind of company, it’s one that encourages experimentation, and the experimentation that Google does is often available to the public. We welcome feedback from users and we make lots of things that we would consider experiments available to the public.
We have this property called Google Labs which is full of the kinds of things you’re talking about, full of all kinds of new and often very quirky and very idiosyncratic applications, and I would encourage your listeners to give it a try and to have zero expectation that one thing will look and feel the same as the next, because that’s really not the objective.
We also take user experience quite seriously. So, if inconsistencies in look and feel is something that in a particular instance keeps users from understanding how to interact with the interface or diminishes their understanding of what it can do for them, that’s obviously a problem that we try to address.
And we try to address it, you know, there are better than 50 user researchers at Google worldwide, and we are examining our products very closely for exactly this kind of issue. But one of the things that makes Google an interesting place to work, and I think one of the things that makes some of the Google products that have come out recently very interesting to end users is the fact that we actively encourage experimentation with new kinds of metaphors for design and new kinds of technology.
Gerry:
You mentioned Google Labs there; have you played with the… I think it’s called the image labeller?
Patrick:
I have not. [Laughs.] How have you found that?
Gerry:
Actually, Brian pointed it out to me, I think it’s one of those incredibly clever ideas. It’s for tagging images. It’s actually a game, so you log on and somebody else is picked at random who’s also logged on. An image is shown to you and you have, I don’t know, 20 seconds or something to provide labels for it and you can get a score depending on how closely your labels match somebody else’s. But it’s being used to improve the tagging, within the Google search, of images.
Patrick:
Oh, that’s great. That sounds like something that could keep me from doing my work for a significant part of the day.
Gerry:
That’s exactly what it’s for – great for displacement activities.
Patrick, I think you have a fairly critical view about some of the recent talk about cultural differences and the relevance… the significance of culture in the fields of design. I remember you took a bit of a swipe at cultural probes recently, and Apala Lahiri Chavan’s Bollywood method when we chatted before. What underlies this critical view?
Patrick:
Well, I should be honest about where I’m coming from. My background… my PhD is in psychology but I studied ethnographic techniques precisely because I was studying psychologists as a group. And my background really is in cultural anthropology. That’s where I come from as a researcher.
My problem isn’t really so much with the Bollywood method. I think the Bollywood method is a really great opportunity for people to think about how they can get people who might be quite different from them to provide the kind of feedback they need to make, say, an interface better. My problem, or my criticism is really one of the HCI community more broadly, and it really has to do with the fact that the HCI community has not engaged the concept of culture with a lot of seriousness. A lot of ideas about culture that have been rejected in other disciplines and have been cast away years, decades ago, are often found at HCI conferences today.
So, we insist on engaging with culture as something that mystifies rather than as something that clarifies. Often you can look at papers or research in the HCI community that engage the concept of culture and in many many instances you can substitute “people not like me” for “culture”, in almost every instance of that word within a study, and not really be terribly confused. It’s not really clear what the word “culture” means, it’s not really clear what work it’s doing, expect to describe some amorphous nebulous other that is posited as something fundamentally different from the world of computer science, the world of HCI, and as we all know the world of HCI is a very limited and culturally very specific one.
So my criticism really has to do with the fact that I don’t see the field engaging with culture in a very productive or sophisticated way.
Gerry:
And do you think it’s something that the field should be doing?
Patrick:
I think it’s something that’s absolutely critical to the field. I think that we should learn from some of the adjacent disciplines in the social sciences, including sociology and anthropology… And to fair, there are lots of people in the HCI community who are doing very interesting work on culture. But we need to see the fact that we use computing machinery and networks as a part of the material and social culture of which we’re a part, and in many ways these objects, these machines, the metaphors that are behind their use, are part of a culture that in many ways brings people closer together than they might be otherwise apart…
On the one hand I want to examine the kind of connectedness that computing offers people of radically different circumstance. On the other hand I don’t want to erase the fact that there are very different forms of social organisation that surround the use of computing machinery, the computing infrastructure. There are radically different material, social and political circumstances around the availability of that infrastructure and what it means…
I would like to see more critical and more meticulous engagement of what culture actually means.
Gerry:
So, Patrick, for somebody out there listening, like myself, who does not have any background in this area at all and would think Jeez, I better bone up on this area [laughs], what’s a good primer? What would you point people at and say “read this book”, “visit this website” or whatever.
Patrick:
Actually, I’m going to look up the reference. There’s a fantastic book, written by an anthropologist on… I’m going to provide you with the exact reference. It’s on karaoke, and karaoke is a fantastic technology when you think about it. It’s a very simple technology, it involves some soundtracks that are divided up into pieces so that you can manipulate certain parts of the soundtrack. You can change the beat, you can change all kinds of… I don’t have much of a background in music but there are lots of things you can manipulate. And one of the most important things you can manipulate is the voice track. So that enables people to take the microphone and engage in what everyone I think around the world would understand is what’s key to karaoke, which is the act of performance.
But we can see how what’s really important about karaoke is that it’s organised socially in radically different ways. In Hong Kong it’s fairly typical to see housewives gathering at karaoke houses at 8 or 9 o’clock in the morning. That would be almost unthinkable in the United States where karaoke without alcohol is almost unthinkable. [Laughter.] Right?
The meaning associated with the act of performance is very very different from one location to the next. So, I’m using that example to really point out how we can take something that’s a technological artefact, that is a karaoke machine, and we can look at all the wonderful differences that we can see in the way that that technology is used. But we’re not really baffled or confused by the fact that some of the basic purposes of karaoke across the world are really the same. Karaoke is a social circumstance, in every place that it’s used it’s an opportunity for people to be together.
The idea of performance is central, even though the meanings of performance vary tremendously. In the US, when people pick up a microphone [and] they start singing karaoke there is some really important element of star imitation that’s happening. In other parts of the world that may be very different. It may be an expression of emotion, it may be all kinds of things that are not necessarily related to star quality. So I would look to that as a really excellent example of how we could look at a technology and at once understand how this technology brings radically different people together, and at the same time have an appreciation for how it is very distinct and how it has generated cultures and sub-cultures that are very very different from each other around the world.
Gerry:
That’s a fantastic example.
Patrick:
I’m just trying to find the reference for you here, and you know… I’m finding search technology to be a bit weak! [Laughter.] I’ve just had to remember the title, and I think that’s how I’m going to find it. Yeah, here it is… It’s called Karaoke Nights, an Ethnographic Rhapsody by an anthropologist named Rob Drew. And I think it’s a great opportunity. It’s one of those ethnographies, I don’t know how familiar your listeners might be with the genre of ethnography. I would remind them that the genre of ethnography is really related to the genre of travel adventure in the 19th century. So for me a good ethnography is one that enables the reader to go on a bit of a trip. And this book definitely allows that to happen. So you really understand what it’s like to be a karaoke… he interviews people who organise karaoke nights, and it’s quite a good introduction. It’s a little light on the theory without being dismissive, so it enables you to understand the theory behind the ethnography without being inundated with anthropological theory if that’s not what your interest is.
Gerry:
You talk about travel… I’m sure you won’t like my mentioning this but you were nominated as a super-commuter on CNN.com last year, is that right?
Patrick:
Yes, that’s very funny. People keep seeing that, it keeps popping up and haunting me. [Laughter.]
Gerry:
This is the trouble when you go online within anything. It lives forever.
Patrick, we often hear, and I heard you talk about this a few years ago, we often hear people say that Chinese users want sites with lots of flashing text and animation and so on. As if that was the key differentiator between the way that, if you like, Chinese people and American people perhaps, North American people, use the web. And you hear it from Chinese people as well, they say that things have to be “re nao” which means really hot and flashing and busy. Is it true that that difference exists?
Patrick:
Well, I would say this, it is absolutely true that you hear that all the time. [Laughs.] And in many ways… Because I was the International Design Manager at another big media company this is kind of a familiar problem for me. You know, it’s very tempting to sort of roll your eyes and say “Oh no, not this again”. But I think that the fact that this problem is so persistent speaks to a few really important things for the HCI community.
The first is the fact that the concept of culture continues to mystify us as a group. You know, in that idea that Chinese users like flashing animations or that there’s an Eastern idea of design and a Western idea of design, we’re really kind of baffled by these really simplistic notions that there’s an “east” or a “west” or that somehow they are so distinct that we could say something like “flashing animation appeals to people that we call eastern” as if there were any coherence to the word eastern or to the term western for that matter. So I think that our lack of sophistication for understanding culture has come to haunt us with problems like this.
But I think it also speaks to some other very practical and very important problems. The first is, I think that kind of assertion has credence in the web or in the internet community because a lot of the technologies that underlie the internet really did limit the kinds of expression, limited culturally the kinds of expression that are possible. And I’m thinking of things that are not terribly complicated. I’m thinking of the fact that for lots and lots of years only ASCII characters could be used in browsers or in other kinds of internet applications.
This of course meant that most of the people around the world who use scripts that aren’t simple Latin-based alphabet scripts were rendered less literate, or somehow had to come up with other fixes for expressing themselves. Even most Western languages use a lot of diacritics, and those were excluded for years and years. I was living in Brazil at the time and I remember there was a bit of a crisis in figuring out how we were going to express things that had accents, when those kinds of things just weren’t available to people speaking Spanish or Portuguese or French or German or any other language with diacritics.
And it excluded entirely people from writing in Chinese or Japanese or Hindi or any of the other complex scripts that were simply written out of the first generations of the internet.
So I think we have to recognise that as a technological culture, ours is one that has traditionally excluded anything but simple Latin alphabets, and I think that that is in itself a fact that’s worthy of note. And I think that as an HCI community we should take care to realise that… and I think that this is a value, this is certainly a value at Google… and I think that this is a value shared by lots of other people in lots of other places… that we want to enable people to express themselves in the language that they’re most comfortable in. If that’s a language that doesn’t happen to be the one that the application was originally authored in, then we have a challenge of making sure that people can write a blog in Hindi just as easily as they can write it in English.
And that’s, I think, both a substantial challenge for us, and I think it’s in many ways an obligation. If we want to see the universalising aspect of internet culture, and if Google is going to make good on its promise to organise the information in the world and make it useful, we can’t simply pretend that English is the only language that information exists in – that’s obviously not true. And Google has, we have a substantial commitment to making sure that our products are fully functional in many languages.
And I think that as a community… that’s another reason why people are able to say to this day, well, visual design in China or visual design in the East is fundamentally different. It’s interesting to note that people don’t make those claims about South America or about Africa. Mostly, I think, because the scripts of South America and Africa are largely, well not entirely, but they’re more commonly based on Latin alphabets.
That of course excludes Arabic and Amharic and lots of other languages that are very poorly represented on the internet. But it’s interesting to note that people in, say, Brazil, are not claiming that the internet excludes them because it has a visual design that’s very different. That’s because, I think, the accommodation has been somewhat easier.
I think, lastly, it’s an obligation of the HCI community first to recognise that there’s a historical grounding for such claims, and secondly to engage rigorously with the concept of culture. That means understanding that there are many rich histories of visual representation, there are many rich traditions of visual representation, including the representation of writing in the world, and the internet does not support all those equally.
But it also means that as, for example in my case as a researcher, I have to dig a little deeper and say well, okay, am I really going to accept this person’s claim that Chinese users like flashing animations, or what tools can I bring to bear to evaluate that claim? And I think that some of the tools that the HCI discipline offers us are actually quite good.
So we can ask really basic questions about the design. They like it for what? Does that mean we should put flashing animations on ATMs in China? Probably not. I think that we need to make some distinctions about what the intention of the design is and what constitutes success. I think if we’re a little more rigorous in applying those criteria, and a little more inclusive in demanding, in requiring that the things that we produce as producers of technology are more inclusive, I think that we would see those claims about eastern exceptionalism fade away.
Gerry:
Google’s at the big end of town, if you like, with both the budget and the need to spend on user research. Do you have any advice for people how need to do some user research but they’re on a shoestring budget? They might be a lowly web developer in a small organisation or whatever.
Patrick:
Yes, I do. And some of these assertions are a bit controversial. One of the things that being at Google, and it’s true, Google’s a big company… I spent years in the not-for-profit sector. I know what it’s like to live on the shoestring budget, and there have been a few circumstances at Google where I’ve just marvelled at the resources that we have. We have amazing laboratory facilities. We have fantastic recruitment. We have tremendous infrastructure. These are luxuries that lots of people around the world don’t have at all.
But there are many times when having an enormous, expensive and very sophisticated research plan isn’t necessarily the best thing to do. There is often no substitute for going out and getting rapid and frequent feedback from users, using iterative… The one bit of advice that I would give to anybody out there who’s working in a one-person or two-person shop is to become very proficient at iterative testing… getting feedback from users, understanding how to do that, understanding the right way to get feedback from users early and often. I can say that even at Google, even at a place with lots and lots of resources and very fancy and sophisticated infrastructure, that’s the number one technique we use. And it’s amazingly powerful.
Gerry:
Well, Patrick Larvie, as always it’s been a pleasure to talk to you and thank you so much for joining me today on the User Experience podcast.
Patrick:
Thank you very much, Gerry, it’s been a pleasure.
Audio first published: February 2008