Published: July 2006
Gerry Gaffney:
I’m very happy to have John Murphy from a company called Design For Use here today. Welcome to the User Experience Podcast John.
John:
Thanks Gerry.
Gerry:
And the subject we want to cover today is cultural probes. Can you start off John by telling me what a cultural probe actually is?
John:
A cultural probe is a very mysterious term for something that is simply a technique for collecting data about a user or about a user’s behaviour.
Gerry:
And is that then the same thing as a diary study?
John:
No, a diary study is a subset of a cultural probe if you like. Cultural probes are characterised by the user collecting data themselves, collecting data over a long period of time and collecting data in a relatively free format. Now a diary’s a very good way of collecting that sort of data. You give the user a diary and they hopefully write in it regularly and collect data around a particular topic. But you could also set the user up with some self-addressed postcards, you might use a scrapbook as well, you might use a voice recorder, there are all sorts of different types of media you can use to collect data in a cultural probe, so whilst the diary study can describe it, it’s actually a bit broader than that.
Gerry:
Okay, so “cultural probe” is not just a term invented by usability people to charge more money.
John:
[Laughs]. No it’s not, it’s actually a technique invented by academics about five or six years ago, and it has been used successfully in academic circles and research projects and it’s starting to be adopted by industry. We’re seeing some of the companies, some of the larger companies using it to try and inspire design and discover things about users and user behaviour.
Gerry:
What sort of companies are using it, John, and in what sort of circumstances?
John:
Look I’m seeing case studies reported by companies like Intel and they’re using it in circumstances where they want to study behaviour what I would call non-traditional work behaviour. So for example leisure behaviour or working behaviour where the work is carried out in a more ad-hoc manner. So these companies are looking to provide technology and devices to support people in these other behaviours or in these other areas of behaviour and the traditional techniques are just not effective or not as effective at gathering data about the users’ behaviour.
Gerry:
So when you’re talking about traditional techniques you’re thinking probably in terms of contextual enquiry?
John:
Contextual enquiry, observation, that type of thing, where the actual presence of a researcher or of a practitioner gathering data disturbs the quality of the activity. So for example if you’re talking about leisure, just the fact of having someone there is very likely to disturb the quality of the leisure.
Gerry:
Right. Now you talked about cultural probes or diary studies, having people fill in diaries or provide information over a long period of time. Doesn’t this mean then that cultural probes are a very time consuming activity to undertake?
John:
They are time consuming but the interesting thing about them is that because all of the data gathering is conducted by the user themselves that great amount of time, or that great whack of time is spent by the user rather than the researcher. The challenge as far as time is concerned comes in, in focusing the study well so that that the analysis, which can be fairly extensive, is somewhat contained. If the focus of the study is not set up properly, there can be an awful lot of irrelevant free form type data and hence the time consuming part comes in analysing and weeding out the valuable information from that data.
Gerry:
Okay, so what steps do you go through John when you’re actually conducting a cultural probe?
John:
The steps you go through are planning and setting up objectives, setting up this focus that I’ve spoken of, is very important; recruiting participants that are happy and willing to keep a diary over an extended period of time.
Gerry:
Doesn’t that mean, John, sorry to cut across you there, but this is the thing that I’ve been concerned about when I’ve been involved in this sort of study is you know, you’re saying let’s pick participants who are willing to participate and willing to gather all this data for you, doesn’t that almost mean by definition that you’re risking having a very unrepresentative set of users getting your data?
John:
Yeah it’s a good point. You need a user that’s prepared over a regular period of time to contribute to the data. Now sorting out how you reward the user for that and sorting out whether you employ a professional recruiting company or whether you use networks of friends of friends, that type of thing, is most definitely a challenge and I’ve seen it done both ways. It’s hard to say one way or another which is the most effective method without knowing the focus of the study but certainly it’s ideal to get a user that’s prepared to fill in a diary or diary media regularly and yet not be biased by knowing the researchers or anything else.
Gerry:
Okay, sorry I interrupted you there, you were saying planning, getting participants, what other steps are involved?
John:
Getting participants, preparing the pack, and the pack to me often looks like something out of a primary school social studies project. They might well contain as well as the diary, scrapbooks, scissors, Post-it notes, glue, sticky tape, pieces of string, that type of thing.
Gerry:
Pieces of string [laughs].
John:
They can contain all sorts of things. The pieces of string might be for use in the scrapbook to allow a person to stick in a picture and then use a piece of string to connect that picture with something they’ve written somewhere else on an opposite page.
Gerry:
So it’s making everything very tactile I guess and inviting to use.
John:
Exactly right. Exactly right. We’re trying to do everything we can here to encourage the users to work with these diaries and scrapbooks and to enter data into them and to make it a fun and interesting exercise rather then a mundane thing that they have to do.
Gerry:
Now you mentioned also putting in pictures. One of the things that we’ve seen in probes and I know you have is the use of cameras. One I was involved in had digital cameras and printers but you were saying people were using other camera types as well.
John:
There’s a couple of different ways you can go with cameras. We’ve used cameras that print out the picture straight away so a user can, or a subject can take a photograph and then stick it in the diary straight away.
Gerry:
So these are the old-fashioned, well I don’t know if they’re old-fashioned but Polaroids is that right?
John:
Exactly right, the Polaroid cameras. Users can also have digital cameras, there have been occasions where we’ve issued them with digital cameras and printers so they can take a whole load of photos and then print them off at a later time.
Gerry:
And then is that part of the remuneration? Is that a reward that they get to keep the camera at the end of it?
John:
Again, this is all about organising your users that are going to contribute well and remunerating them is certainly part of that, and so rather then just paying them money some of the more innovative and interesting ways of remunerating them are to give them the camera and the printer, and this has been done quite successfully. So a user has an incentive to learn to use the camera and the printer and to use it a lot because it’s part of their reward and they get to keep it and use it for their own devices as well.
Gerry:
Okay, so again I’ve interrupted your flow because you did talk about planning, getting participants, preparing these packs and we sidetracked a bit into packs. What else happens during the process of conducting the cultural probe?
John:
The next thing really is to plan initial interviews, follow up interviews. A follow up interview might be conducted part way during the study, planning a time for the data collection, the data might be collected over a period of days or weeks and then to finally have a debriefing interview. Just to come back to that follow-up interview that is conducted within the study, it’s actually a very important part of the study because in our experience many of the users or subjects forget what the study’s about or lose focus or misunderstand the focus and it’s not until they’ve actually had a go at working with the diaries and the scrapbooks and the various other media that they realise what they don’t know and then this follow-up interview serves very well to re-focus them or bring them back on track or narrow down their focus perhaps and hopefully insure that the data that’s being collected then is more useful.
Gerry:
I guess that emphasises too the importance of that initial interview. I guess by initial interview you’re describing a briefing session of some sort.
John:
Yes, I am, and that may well be done with all the users together or each one individually.
Gerry:
It’s funny too how you talk about making sure that people actually engage in the activity. My experience has been that you get some people who are highly engaged and really get totally involved and then you’ve got others that you almost need a cattle prod to get them to bring back any information to you.
John:
Yes that’s right and again that comes back to the point of how they’re recruited and also how they’re remunerated.
<
Gerry:
So it sounds like, and I know from my limited experience of it, it sounds like a very exciting technique and you’ve been lucky enough to be involved in your academic 20% of your time as well as your commercial work in this area.
John:
Yeah, I have, and I think the future for the technique is very exciting in that it’s not just about validating designs or making hypotheses and seeing if they’re validated, but it’s about inspiring design and inspiring innovation. It provides a real opportunity to gather new data very early on in the design about the way people behave or think about a topic or a subject related to a technology being developed and very often the HCI practitioner is able to discover new things that they didn’t know previously and use those positively for the designers as some sort of artefact.
Gerry:
I found that information from cultural probes was very useful for creating personas and scenarios. Can you tell me a little bit about the data analysis I guess because the risk with something like this as you’ve already mentioned is that you’re going to end up with heaps and heaps and heaps of data.
John:
The data analysis part of it, I think is probably the least developed aspect of it considering it’s a new technique. There are all sorts of ways of analysing it there’s nothing emerged as standard yet, and simply having several people look over the data and brainstorm it, to creating some sort of affinity diagram around certain topics or subjects within the data is another way of reviewing the data. But yes there is the risk or the effort is that there’s so much data and it takes a very long time to look through it thoroughly and it can be looked through many, many times with many different focuses or techniques.
Gerry:
Which I guess brings me to a question that’s probably going to be very difficult to answer. If you compare a cultural probe to say a contextual enquiry with the same number of people, say half a dozen people, which of those two activities is going to take more time?
John:
I think a cultural probe will always have a risk of taking more time simply because of the big difference that the data’s being collected by the subjects themselves. In a contextual study, no matter how extensive the data, the data is actually collected by the practitioner and so there’s always this filter being applied as to what’s collected and what might be relevant. Whereas when the user’s collecting the data there’s no such filter. Now whilst that is dangerous or difficult in that there’s a lot more data, it’s also very exciting in that it really does allow for the possibility of the designer discovering more things and new things that they previously might have filtered or discounted. But yes, I think the cultural probe will always be at risk of having more extensive data than a contextual enquiry.
Gerry:
Okay well John it’s a very exciting technique and obviously we could talk about it at some length But just to finish off very, very briefly do you think this is something that the typical usability practitioner is going to have in his or her toolkit in the next couple of years?
John:
Yes. Most definitely, as we see the boundaries between working life and leisure life blurring and it becoming more important to design artefacts and devices to support both leisure and working time we need these types of effective techniques to inspire design and to allow us to work effectively in the leisure world and I think this will definitely be one of the techniques that’s in there, in the pack of the designer to help support them.
Gerry:
John Murphy, thanks very much for joining me today on the User Experience Podcast.
John:
It’s been a pleasure Gerry.
Published: July 2006
You may also be interested in the interview with Bill Gaver, who invented the cultural probe technique, and in Gerry Gaffney’s slideshare of a presentation on cultural probes (including a brief case study) at the UPA conference in Turin in 2008.
We make verbatim transcripts of the User Experience podcast. We then edit the transcripts to remove speech-specific elements that interfere with meaning in print (primarily space-fillers such as “you know…”, “um…”).