Published: 7 July 2014
Donna Tedesco and Fiona Tranquada, authors of "The Moderator's Survival Guide" talk to Gerry Gaffney about moderation. They discuss styles like Friendly Face, Down to Business, Inquisitive Mind and By-the-Book, the risks and pitfalls, Gerry's labcoat, and advice to moderators of all levels.
Gerry Gaffney:
This is Gerry Gaffney with the User Experience podcast. Today’s guests each has over 10 years’ experience in user research and each has a Masters in Human Factors and Information Design from Bentley University. One is a senior user experience researcher at eBay and one is a senior user experience specialist at the The MathWorks. One is in San Jose and one in Boston.
They are co-authors of the recently published and very entertaining and interesting book The Moderators Survival Guide: Handling Common, Sticky Situations in User Research.
Donna Tedesco and Fiona Tranquada welcome to the User Experience podcast.
Donna Tedesco:
Thank you Gerry. Thanks for having us.
Gerry:
Well the book is aimed at moderators of various user research activities although I guess it’s probably fair to say that its focused mainly on one-one-one type activities like usability testing.
Donna:
Yes, so we have three one-on-one methods we focus on in the book and they are usability testing, interviews and contextual inquiry or field research. Basically we did start a little broader when we were early in the process of writing the book and we were thinking about including situations with multiple participants but the more we thought about it we realised that there are just a lot of aspects and situations that come with that. For instance, do you have a focus group or co-discovery? There’s a whole layer on top of regular moderating because you have to deal with session management and group-think and different personalities and that type of thing and we just realised it was kind of opening Pandora’s box a little bit and we wanted to keep this really narrow and focused so it didn’t become a 500 page book or 800 page book.
So along the same lines we actually did focus on what happens during a session rather than before in the preparing phase or in the reporting phase after, for the same reasons. It just would be you know those could be entire books in themselves so we really wanted to keep the focus specific and go into detail in a deep dive on that.
Gerry:
Yeah I guess you can look to other books for a sequel.
Fiona Tranquada:
Exactly, and this is Fiona, just to add something to what Donna’s said, we felt like there were a lot of existing resources that covered some of those other aspects, particularly in preparing for research and getting all the appropriate materials together as well as doing the analysis portion of it and we really wanted to, as Donna said, keep the focus narrow as possible.
Gerry:
Now you start with an overview in the book of moderating itself and I found that the discussion of the four different styles that you identified being very interesting. Can you tell us a little bit about, you know, where those styles came from, is it something that you sort of invented or researched or basically how they were derived?
Fiona:
We came up with the styles based on our experiences watching a number of different moderators at all levels of experience and we knew that we wanted to find some kind of categorisation that would let us talk about some of the differences that different moderators have that’s basically independent of the role that they’re playing in the session. So, something that was kind of reflecting whatever that person’s core personality traits were and kind of their natural disposition.
And based on that we came up with those four different styles and realised that it kind of fell into these different spectrums based on how engaged or disengaged the moderator was trying to be with the participant at the time, as well as based on how fluid or how rigid they were being compared to whatever they had planned back on when they had put together their research script.Gerry:
Do you want to tell us about each of the four types, I guess relatively briefly and what the implications are and the characteristics of each style?
Donna:
Yeah, this is Donna. I’ll talk about that a little.
So the four styles that we put in the book are Friendly Face, Down to Business, Inquisitive Mind and By-the-Book and we kind of came up with those names to categorise them. So just to describe a little bit about each:
Friendly Face is really about approaching the session with the participant’s comfort in mind, like being comforting and kind for the participant.
Down To Business is really a serious tone, projecting a sense of control. It’s really more about the research and the quality of the research, focusing more on that piece of things than the participant themself and the comfort piece.
Inquisitive Mind is really being like a student; eager to learn, really there to get as much information as you can; you know, these styles tend to have a lot of follow-up questions and maybe you’re not as loosely or as tightly following the study plan. It’s really about understanding what’s happening in the experience and you see a lot that style happening in formative research.
And then By The Book is really kind of the opposite. It’s following a plan very closely, sticking with a certain flow. And so basically each of these have different strengths and different pitfalls depending on the situation you’re in and what the participant’s like and what their disposition is like.
So, for instance, with Friendly Face, that really can be a great style to take on when the participant’s especially timid or intimidated or self-blaming. But if you get a participant in there who’s quite gregarious and you start being overly friendly with them it starts to become a very informal conversational fraternising type session and that can lead to a lot of problems like social desirability bias on the participant’s part. They might be, you know, trying to say what will please you and it becomes more of a friendly thing than a professional setting.
So, really keep in mind that there are different levels of each of these things. It’s not mutually exclusive and, as Fiona said, we just have different styles across sessions or within sessions really to adapt to the participant’s personality and what’s happening in the sessions. And not in a Jekyll and Hyde kind of way but more in just a subtle way that works with your personality and trying to adjust it to what’s happening and steer the session gently as the moderator.
Gerry:
So you mentioned some of the risks, if you like, of the Friendly Face approach and that it degenerates potentially into having a chat and the whole social desirability factor kicking in. Do you want to tell us about the pitfalls of each of the other three styles as well?
Donna:
Sure so for something like Inquisitive Mind what happens is you are like a student, you’re eager to learn but that often can have implications about the session going off track. You might not be able to manage time as well if you’re kind of following tangents a lot and really probing too much. So that could be a problem and also if you’re doing more summative research, it’s not really the right time for Inquisitive Mind in a lot of cases because you’re really trying to follow more of a structure in which case you might want to fall into more of a By The Book type attitude.
Similarly, Down to Business was the serious kind of style and, you know, you don’t want to necessarily be abrasive off the bat. So you know if you’re very overly serious and kind of not very warm and somebody is especially nervous they might get even more nervous or it might really start to affect them and the session. But it’s like I said earlier there’s a gregarious participant who is talking it up or maybe even like flirting with you or going a little too far, that’s the time to start focusing more towards a very down to business type attitude so they know you’re here to be in control of the session, to really like get through this research and get their feedback and that’s about it.
So, they’re very kind of subtle shifts in some cases and it’s really around adaptation to what’s happening.
Gerry:
Yeah I think it’s really difficult to get the mix right, isn’t it? I often say when I’m working with people in a sort of mentoring situation and they’re starting to do moderation for the first time to be friendly but imagine that you’ve got a lab coat on you so that you can sort of try and, you know, I guess straddle the two approaches, the two key approaches.
Donna:
Yeah that’s really great advice. I like that one.
Gerry:
Now the main part of the book is the survival guide and I must admit this brought back to me when I was first moderating many, many years ago. You’ve got a whole bunch of things that can, well not necessarily go wrong, but situations. Some of them for example, “Participant gives vague responses to questions”. Another is “Something personal, inappropriate or confidential is visible.” And it struck me that these scenarios could well you know serve to scare the daylights out of a novice moderator who might not have imagined the full range of possible catastrophes.
Donna:
[Laughs.] Right, our goal is always to give people a good scare. No, I’m just kidding. I’ll let Fiona talk about that one.
Fiona:
We did actually talk about that a lot, especially when we looked at the full list of situations that we identified and realised just how many there were and just what an incredibly broad range of circumstances we were covering. But at the same time we felt like there was a lot of value in that.
Donna and I have both done a lot of work with really new user researchers and have seen them moderate their first sessions with users and it doesn’t happen often but sometimes your very first session ends up being a disaster for one of the situations that we describe in the book and we wanted to be able to compile these and have these all available so that someone who is getting into moderating starts to understand the scope of all the different kinds of things they might eventually encounter.
We wanted to have the book be almost someone’s smart friend or experienced friend who’s been through all these different kinds of things and is going to share some advice about it so that you don’t have to be the one figuring it out on your own in the middle of the session.
Gerry:
I guess one of the things that often occurs to me too is with newcomers, is that an important thing to remember is that you are in control as moderator and you do actually have more leeway than you would in a regular conversation.
Fiona:
Absolutely, one thing that we talk about is that the participant really has no idea what’s going to be happening. The moderator sets expectations at the beginning of the session about how the session is going to work but the moderator’s really the only one who knows what tasks they want the participant to do or knows what questions they were planning to ask.
So if something does happen and you as the moderator need to change your approach for whatever reason or maybe skip a task because you realise that the participant doesn’t have the criteria that’s really necessary to give you the feedback that you need on a task, you as the moderator can just skip that and the participant doesn’t even need to know that anything changed. Because the moderator is the person who’s in control and there’s a lot of responsibility with that as well as a lot of power that lets you accommodate all kinds of things that might happen.
Gerry:
And the list that you’ve got is extensive. How did you come up with it? Was this from your own experiences or talking to other researchers or where did the list of scenarios come from?
Donna:
Yeah, that was actually a little bit of everything. So we started off by brainstorming on everything that we’ve encountered ourselves in our careers in moderating, everything we’ve seen others encounter in their moderating, like we were working with other moderators on a study, and then also we did elicit stories and different anecdotes from various researchers that we either have worked closely with or just kind of in the field and heard it first hand from them things that happened.
So we did have a little mix of everything and it was quite fun to actually make this list and we had a kind of running list for a while and there would be an occasion at work where I would just come across something that either I had forgotten to add to the list or just was a new thing that hit me like I’ve never realised or had before and then I would go add it to the list and kind of email Fiona and say, “Oh gosh I had this angry ranting participant today, like this is definitely something we should cover because that was a tricky situation.” And then we would proceed to kind of talk about “What would you do? What would you do?” And sort of reconcile what our thoughts were and it was actually pretty easy for us. I guess we think very alike in a lot of ways and you would imagine we’d have a lot of difference in opinion but we really didn’t. We really had a similar thought along the lines for all of these different situations.
Gerry:
Where did the idea for the book come from? Was the list in existence prior to deciding to write the book?
Donna:
It was in existence in our heads basically. It’s something that Fiona and I had talked about many times and kind of discussed different stories; maybe we had little lists here and there on our own. I know that Fiona always kept kind of a list of things when she would mentor newer moderators and would talk to them about different situations.
So I think they were piecemeal but we didn’t have anything formal and we realised, wow, there’s so much here and there’s so much to talk about with moderating that we see day to day, we experience day to day and you know why don’t we do a deep dive? There are some really great resources out there for like Intro to Moderating but you know there’s so much tacit knowledge out there that we can pull out of us and others. So that’s really where it was born.
Fiona:
And the other thing that started happening is as we started realising that we really wanted to put together a book about moderating and we’d mention it to our colleagues and other people in the User Experience field they started sharing stories with us and you’ve probably encountered this if you go to almost any UX event, someone’s going to be telling a story about something that they encountered while running a research session and it will probably be funny, it might be interesting, it might be terrifying depending on what actually happened. But we started really paying attention to a lot of those stories and using some of those as basically inputs into the list that we had created so we’d be able to really have a pretty extensive spectrum of different kinds of things that we knew actually did happen.
Gerry:
What’s your favourite scenario?
Donna:
Ah, favourite. We always get asked questions like this and I actually I feel like I have a bit of a boring answer because the boring situations are really my favourite. So what I mean is, you know, things like the participant didn’t complete a necessary task or was looking for affirmation or is nervous, like, all those little things don’t make for really great stories but they are, they represent the small wins day to day. They come up a lot and I really feel like as I work through them more and more I feel good about it. It’s really satisfying when a scared or angry participant comes into my session to begin with and then by the end they’re walking out and they say, “Wow, I really enjoyed this”. Or they open up over the course of the session because something I did with my moderating, or hearing from other observers like “Wow, you’ve really, like, you’re great at this.” You know so that’s really satisfying for me and I can’t pin it down to any one favourite because it’s like, you know, like your children you can’t have favourites, to me.
Gerry:
Fiona?
Fiona:
I think my answer’s similar to Donna. I mean I think there are some scenarios that were really fun to write about and try to come up with really specific advice on how you handle them so that things like a participant who’s flirting with you or a participant who maybe gets interrupted by their significant other when you’re running a session with them and gets distracted by something like that. But I think the scenarios I’m proudest of are actually some of the ones that we have that talk about some of the technical obstacles that might come up during a research study. So, this might be anything from problems with your remote technology, if you’re running a session with someone who’s in a different location than you, or your prototype breaking or not working in the way that you’d expect, and I feel like we were able to put together some advice that we hadn’t really seen anyone else talk about. So, for example, a lot of people when they’re running remote sessions encounter problems when they try to get their participant into the remote session and that’s something that can be very frustrating. It can take up a lot of time and if you only have an hour worth of time with a participant that can feel like a real waste. And even if you try to set the thing up with them beforehand and get all the technologies sorted out, I like to call them gremlins, sometimes show up between when you do a test run and when the actual session happens and one of our recommendations that we actually put into the book was that in order to accommodate these gremlins that might break everything between these two times to put together essentially a cheat sheet for yourself of exactly what the participant experience looks like for joining a remote session. So essentially screen shots of exactly what they should be seeing at each point in time.
So if you’ve ever experienced a participant who’s trying to join WebEx, for example, and says “Oh I’m seeing a screen that just says that I’m joining the WebEx meeting centre but nothing is happening.” If you have a couple of screen shots in front of you of what they should be seeing and what else you can maybe reference to get them successfully into the session, that’s something that’s going to be a huge win for you as a moderator and will hopefully get you to a position where you can start the session and be successful more quickly.
Donna:
Yeah I was just going to add to that for a moment you know we started seeing a lot of situations kind of really share a lot of the same techniques for trying to prepare and mitigate them or things that happen in the sessions and so we actually realised later in the game that, wow, this is a lot of content. So we actually have Chapter 15 in the book which is all about preparing for and either avoiding or mitigating situations and that really came from all the stuff that Fiona’s talking about like technical challenges is a big part of it, you know how can you prepare ahead of time so that these situations don’t happen to the same extent or at all and I think that was a really valuable add on for us.
Gerry:
And I guess to some extent you’ve addressed another question that I had. At times the book reminded me a little bit of an aircraft checklist, you know engine one is out, smoke is coming in under the cockpit door and you’re desperately flicking through the binders looking for the appropriate checklist to follow, to find out what to do next. I guess you envisage people reading the book more as a preparatory activity or do you see it as a retrospective activity?
Donna:
Yeah well it’s definitely not meant to be read during a session. That could actually provide a very awkward situation if you’re sitting there and saying “Hold on a second, let me reference my book here.” But just aside actually if you do take a break in the middle of the session, I can imagine using it sort of in a back room or control room if you need to go take a look. But really that wasn’t the intention. It was yes kind of both of what you’re talking about is for preparation as well as retrospectively. So if we look at part one of the book, it’s actually meant to be read sort of sequentially like any regular book. It has an introduction and the first three chapters which really set a lot of context for the rest of the book, get you in the framework of understanding, like we were talking about the styles of the moderating and sort of our philosophy as the authors so that you can put that in context when you’re reading later into the situation.
But Part Two is more about just all the different situations. So for the situations themselves, you can really read them ahead of time. I mean you can maybe read it through or you could cherry pick or kind of browse, you know. I think we suggested you could do any of those things but I do recommend reading it ahead of time as much as you can because it really, not to be a UX geek here, but it helps you form a mental model, right? Like it really helps you create a framework for understanding what can happen in these sessions and how to approach it and in fact we talk about this thing called moderation patterns in the book and that’s really different techniques that you can use to apply to a variety of situations.
So one is actually taking a break. Another might be engaging more with a participant, another is engaging less with a participant. So there are these patterns that you could apply across the board. So once you get a sense of sort of these patterns and the different things that can happen it really helps you to be prepared to let anything come your way even if you face a situation that’s not in the book. So I think it’s truly helpful to get that mindset ahead of time. And then of course after something’s happened, when your moderating you could always go back and check the book and say OK did they have a situation or something like it? What was their recommendation? Does that really apply in my particular situation? You know, could I have done something differently? And it’s really about, the book is as much about advising as it is about encouraging introspection and basically reflecting on your moderating skills. That’s really half the battle and just trying to continuously improve and think about the way you affect participants and the way you affect the study. So we think it’s useful in both contexts.
Gerry:
It was interesting that you mentioned patterns of moderator behaviour and I guess when I see books or websites on patterns there’s always a risk that there’ll descend into being mechanistic and formulaic and boring but I didn’t find that with the book. The patterns worked quite well.
But let me throw a couple of questions without notice at you: tell me how you would deal with this situation. I had somebody some years ago come into a useability test session I was running relatively early in the morning, I think around 8am, and this chap arrived and he clearly had been night clubbing and he’d obviously equally clearly been imbibing some sort of illicit substances and he was coming down from them and he sort of sat there staring at the screen in this monosyllabic trance with, I can just imagine the duff music still going through his head. What should I have done in that circumstance?
Donna:
That’s a good one. Fiona? We’ve talked about this kind of one before. For me personally I think you know that’s when you have to assess the situation because we’ve heard stories and actually seen situations where people were so kind of you know gone, so to speak, that really having them stay there would not yield very useful information. There are situations where someone may be kind of under the influence of something but they are otherwise pretty, you know, they’re pretty lucid about their thoughts and you don’t necessarily want to dismiss them just because of that or maybe you feel like they really need to sober up, you know, in that situation.
So it really depends on like kind of how far gone they are and what their goal is and through the professional situation. If you have you know a room of observers in suits in the back and what they’re going to think about kind of your recruiters or you representing your organisation. So I think that’s all part of it but also there’s the safety piece. So, like I said, you don’t want them to walk away and drive if you see that they’re in a bad situation so that’s where you might want to get kind of your local security involved or something else to really care for their comfort because they are the participant at the end of the day, even if they went clubbing the night before and they’re a little gone there, you want to definitely protect them as your participant. You have an ethical responsibility to do so.
Gerry:
I’ll give you another one. I’m normally fairly relaxed about moderating because I’ve been doing it for some time but this coming week in fact I’m going out and I’m doing usability testing with a number of visually impaired users doing one-on-one sessions but I’m going to their homes or their place of work because we want to see the, you know, the website perform in the wild. So I’ve got no idea what the technology’s going to be like, I’ve got no idea how it’s all going to work and I’ve got a degree of trepidation. What should I do?
Fiona:
That’s a great situation. So given that you have no visibility into what the situation is actually going to be like, I would definitely recommend spending as much time essentially brainstorming every single possible thing that you might encounter and come up with some kind of game plan for how you might address it. So if you show up and for whatever reason the internet isn’t working, is there something you can have in your back pocket that would let you still be able to get some kind of feedback, whether it’s something on paper, which may not be appropriate, depending on the level of visual impairment, or can you bring a laptop with you that has some kind of mobile wireless access on it so you can at least have some, give you feedback on the website even if it’s not actually on the computer?
So I think going through those steps and trying to figure out all those different things you might encounter is going to be incredibly helpful. The interesting thing about going into people’s homes and watching them in their own environment is that first of all it’s the kind of research that I think almost anyone in user research loves being able to do, being able to get out of the lab and actually see something the way that users actually encounter it.
But that does mean that you have to be coming in with an approach where you’re really just willing to just roll with whatever is going, whatever the situation on the ground is when you get into that participant’s home. Because you have no idea if they’re going to have a shiny new MacBook Air or if they’re going to be using an incredibly old PC. There’s going to be many, many things that you might encounter.
Depending on how you found these people and what kind of contact information you have with them you may be able to reach out to them ahead of time and ask them questions that might give you a little bit of insight into what the environment might be like and what you might need to prepare for. However, we always take feedback like that with a grain of salt because we find that sometimes users and, gosh, anyone doesn’t always really know what they’re using to access the internet or to do anything in terms of technology. But it might give you a way to nudge yourself in the right direction. Donna, do you have any initial thoughts?
Donna:
Yeah, I was just going to add something around the point that Fiona made about maybe having some screen shots like just in a more general sense. That’s such a great thing that’s really saved me a couple of times, like having screen shots of a website or your product or even having a survey or something as a backup that you can bring on paper or on your own laptop because you never know when something’s going to go wrong and you can’t proceed with the session. That’s kind of, you know, your last resort or you could just change the context I guess. One of our patterns in the book was around shifting the focus of the study so you could shift the focus from observation to more of an interview if you want to and get a lot of great information that way and not need the product at all.
So, you know, that’s definitely a last resort because you don’t want to have to do that in your study, but those are great things to have in any situation, either field work or even in your own lab if something happens. So that’s an example of what I meant by there are themes that come up in a lot of these different situations and they’re just some more overarching ways you could be prepared for a lot of them.
Gerry:
You have a series of videos that you mentioned in the book also and they’re posted at http://www.modsurvivalguide.org and they illustrate some of the situations and how to handle them. It looks like they were great fun to make.
Donna:
Yes.
Fiona:
They were. They were a lot of fun to make and we actually spent a lot of time as we were writing the book thinking about what would be a really helpful video accompaniment to go along with what we were trying to portray and we ended up with a total of seven videos where each video is showing a few snippets in time of some kind of session and across those snippets of time you’re able to see essentially the progression of a session and you’re able to see the moderator encountering some kind of situation; whether it’s something caused by something the participant is doing or the participant maybe not having the criteria that you were really looking for or dealing with a stakeholder who’s observing the session and who doesn’t want you to continue and through those videos you’re able to see the moderator try to handle the situation using some of the patterns addressed in the book.
Our goal was really to both have something that was fun to watch and engaging and something that would hopefully generate discussion, discussion among other moderators about, gosh, was that the right approach? What are some variables that might have changed how the moderator should have handled this? Is there anything else that we really should be thinking about when we’re dealing with situations like this?
Donna:
Yeah, I’d add that the third motive was to provide our own entertainment for sure because we’ve spent a lot of time kind of holed up writing or meeting together to write it and this really helped us kind of get out of that and do something a little more fun and a break from all that writing that would still provide value and add to the book. And I should mention here that we need to give a little shout out because Chris Portal is the person who actually was our DP and editor and director. He’s the one who really was the mastermind behind creating the videos aside from the writing it, which is what Fiona and I did, but he really took it to the next level of making it really great and professional and they were long days. We spent many weekend days across various weekends and each shoot took, you know, at least six hours, sometimes a lot longer. So it was a lot of work but I think it was really worth it because it’s really good to see, we talk a lot in the book about how to handle things and what to say but a lot of it’s about intonation and it’s kind of that para-language and the videos really help with a lot of that.
Gerry:
Indeed and I guess on that parallel track, one of the things that we mentioned in communication prior to this interview and coincidentally I contributed something to it also was Steve Portigal’s war stories. Have you guys paid much attention to them?
Donna:
Yeah. Actually we had discovered that when we were earlier in the writing process I think in 2012 maybe. We had come across them and we were like this is really great. They’re really engaging stories and it’s a great way to bring different things alive and help people learn from their experiences. So that actually helped us really confirm that we were on the right track because actually we were doing something similar. If you look in the book there are what we call “survival stories” and it’s not quite as focused on just field work as the war stories are but it’s along the same lines and I think it really helps to have other voices besides our own come in here and say “Hey, like, these things really do happen.” These are a variety of different situations and really bring in their stories and how they learn from it, what they would have done differently. So I think it’s really added a lot to the book but I really love those war stories and just as Steve’s doing on his site we are also collecting stories as well. So I’ll sort of plug in here if anybody has any survival stories for moderating you’re free to contact us. You can go to our website which is http://www.modsurvivalguide.org and you can definitely contact us from there and we can submit your story on our site as well.
Gerry:
But guys what’s your best advice to the novice moderator?
Fiona:
So for the novice moderator I’d actually reinforce two things: so first is that just acknowledging that it can be a little bit scary to get started and you’re not going to be perfect the first time you get out there and interact with a user. I used the analogy the other day that it’s almost like when you’re making pancakes. The first pancake is never the best pancake but as you keep making pancakes you’re going to get better. The heat’s going to work itself out and you just have to prepared that the first few sessions that you run may not go very smoothly. But if you’re prepared for that, you can leave each session, talk to anyone else who observed and figure out, OK what could I have done better? What can I learn from this? What can I then apply to everything else that I’m going to be doing throughout the rest of my hopefully long career as a user researcher.
And then the other piece of advice for the novice moderator would be that when you’re sitting in the session it can take a while to get used to the role that you’re playing so that you’re not just someone who’s having a casual chat with another person like you would in a coffee shop or in a bar and one of the things to kind of help get a moderator used to that is reinforcing you don’t have to respond right away. You don’t have to react immediately. If the participant says something that you don’t fully understand or is potentially a little bit controversial, don’t feel like you have to have an immediate appropriate reaction to it. Give yourself a little bit of time to think about it, you don’t have to respond immediately and that ties back to one of the patterns that we talked about which is just taking a moment, so giving yourself the time to come up with a reaction that is appropriate for whatever you’re dealing with, and if you’re a novice that might include actually taking a moment to go next door and talk to your observers or talk to someone else and have them help you figure out what to do, or it may just be something that you take a moment and think about it and maybe you can come up with a solution yourself.
Donna:
One thing we also talk about in the book is kind of like the three second rule, if you will; so basically taking three seconds. It can seem like a lifetime, especially when you’re a new moderator, you want to kind of respond the way you would in conversations or something but if somebody asks a question it helps if you take a few seconds and that will sometimes resolve what’s happening, like sometimes the participant asks a question like, “Oh did I do that right?” or “Is that it?” and that type of thing and you don’t want to necessarily answer them so if you give it a little space they might just go on or kind of realise the answer for themselves.
But it also helps at the beginning of the session to actually tell participants, and this is more in the context of a usability session, like you can tell them, “Hey, I may not really answer you” or “I may be a little vague about my answer and if I do that I’m just trying to be neutral and trying not to lead you.” I’m always very upfront about that and it helps to I find participants not to really look to you for help along the way because you’ve set that context for them. But anyway basically I’m saying that the three second rule is really a saviour because you don’t feel the need to just jump straight in and help them and it’s really helpful and I’ve seen it really help new moderators a lot as well.
Gerry:
Yeah it’s a great and simple rule I think. You do notice that difficulty that people have in realising that they’re not in a regular conversation and that you know slightly longer than conversational pauses are appropriate.
So, what’s your best advice to the experienced moderator?
Donna:
I would say for the experienced moderator that, kind of along the same lines like you can make mistakes, it’s OK. Just because you have been a moderator for a while doesn’t mean that you shouldn’t or don’t make mistakes. I’ve definitely had many times where I’ve had my foot in my mouth, even just recently when I’m moderating you always kind of slip out and say something leading or that type of thing.
So we’re always continuously learning, even if you wrote a book on the subject, like, these are experiences that you’re having all the time. You might be having a bad day or you might be just thinking of a lot of things at once and little mistakes happen and that’s what we learn from.
So, you know, basically we all have this great tacit knowledge. You may have the right answers and know how to handle a situation but you also have to think about the factor of thinking on your feet and that takes practice as well. It’s kind of like a separate swim lane, if you will, of becoming a better moderator. So you’re always going to hit new situations, you’re always going to make mistakes and that’s okay. The best thing to do is learn from that, reflect on it, talk to colleagues, have them watch you no matter how long you’ve been in the business. I’m always pleased to hear people who have been moderating for like 30 years and say, “Oh, yeah like I just messed that one up the other day.” And they’re just honest about it because that’s what happens day to day.
So don’t be cocky, it’s OK. That’s my answer.
Gerry:
The book is called The Moderators Survival Guide: Handling Common, Sticky Situations in User Research. You can use the discount code PBTY14 at Elseviers online store for 25% of this and other books.
Donna Tedesco and Fiona Tranquada, thanks for joining me today on the User Experience podcast.
Fiona:
Thank you for having us.
Donna:
Thanks Gerry. It was great fun.