Published: 19 December 2020
What to do when no-one can attend, and how to design a user-centred conference experience
Gerry Gaffney
This is Gerry Gaffney with user experience podcast.
My guest today has worked as a program manager of Ryanair and as a mobile program lead at Aer Lingus.
In 2006, he co-founded UXDX as an annual conference. Like many conferences at switching from an in-person to an online event. And I wanted to talk to him today about managing that process.
Rory Madden, fáilte, welcome to the User Experience podcast.
Rory Madden
Thanks for having me.
Gerry
I should start by asking what UXDX actually is.
Rory
So what it stands for is User eXperience/Developer eXperience, and the idea where it's, it's kind of becoming a lot more commonplace now, but strangely for a lot of companies, back in 2016 the concept of user experience, particularly in an enterprise setting was quite foreign to a lot of companies. Not to everybody obviously. So the idea was to try help companies. It was the biggest gap that I was seeing as every company was going Agile. But it was still just a way to try and make developers work faster. So the concept is bring UX into the product development process.
Gerry
I guess, moving from in-person to online is a big step. I presume that COVID-19 was the primary reason for doing that?
Rory
Yes. We have been looking at a kind of a kind of a… So as a conference, a lot of conferences, what their goals are different, and our goal is to enact change. So that that's how we kind of rate ourselves on whether we're succeeding or not. And so are companies making the shift over to more product teams, product development, style way of working? And we always have the goal of moving past a conference because a conference is fantastic for motivation, you get people there, it's a great kind of feeling, everybody's riled up, you're hearing these great success stories and you leave kind of really pumped up. And then we kind of did some follow-up research with people a week, two weeks afterwards, things are starting to wane and people have forgotten, the deadlines have come back and all of those great ideas and ambitions seem to kind of fallen to the wayside. So we always had been thinking about, we have to do more than just a conference and make it more of a continuous thing to help people avoid that lull and that drop-off after the conference. So it was something we were looking at, but it wasn't going to happen immediately. Our hand got forced a little bit with COVID to figure out how we were going to run an event that would deliver on that mission of making change happen online. So it, it was definitely influenced by COVID.
Gerry
And I presume the first three conferences were in Ireland where they on an annual basis?
Rory
Yeah. So in Dublin there's a venue called the RDS. We've held it there for the first four years. So 2016 to '19, and it was kind of, the RDS is a very large venue. We were kind of taking chunks of it and just growing and expanding how much of the venue we were taking. And it was looking in, February was our best month ever for ticket sales. So things were looking fantastic for, for 2020, and then in March things took a different turn.
Gerry
And your intent is to to continue with an in-person one after this period of emergency is over or do a combination or what?
Rory
So I think it's going to be hybrid forever. The in-person to me, I don't think any conference had cracked this yet, or none of the, nobody that I've talked to… The networking effects of a conference in person are amazing. The fact that you're in a room with all of these other people, just the serendipitous, you bump into somebody and you get chatting over lunch or something like that. It's hard to replicate that online. And I haven't seen anybody who's properly succeeded in doing that yet. I'd love it if somebody out there can send me some links or anything about how has somebody who's done that successfully. I'd love to hear about it, but what I've found, particularly with the online ones is in a physical in-person event, people are there, they have to go for food. They have to go sit down beside somebody at a talk. Whereas on an online event, people are at home, they're in the office, they can check their emails, they can walk away, they can turn off the feed. So there's too many other options. So while everybody does offer and we offer as well kind of networking activities, the take-up is relatively low, unfortunately. So it's something we're definitely going to still experiment, but that's why for us getting back to in-person is critical because I think there is something that online can offer the physical can't, but physical can offer that online can't as well.
Gerry
Just on that issue of getting together one thing that I've been involved in a few conferences you know, over the years, and a lot of conferences, organizers talk about wanting to do something about their carbon footprint and, you know, arguably being a virtual conference stops all those people flying in because that's, you know, like putting out keep cups or, you know, switching to hessian bags instead of, you know, kit bags or whatever, I mean, they're pretty tokenistic. I mean, a big part of the whole thing is the travel. Isn't it. Have you put any thought into that? And I know this is a question without notice Rory…
Rory
We massively have been thinking about the impact. It is a little bit… like me personally, we don't buy plastic bags in the shop, with my daughter we use reusable nappies. Like I'm trying very hard myself personally, to cut down on my impact. So when you're looking at a conference, yes, there is the travel for everybody, there's also a huge amount of, you're building this,. you go into an empty warehouse, almost any build this entire stage structure and entire kind of exhibition space for two days, three days, and then you rip it all down. And so we're trying to figure out ways that we can work around that. The kind of the simple ones that we've started doing in the physical ones is the reusable glass bottles, the reusable coffee cups. So we're trying to avoid just that disposable kind of plastics at a minimum, but yeah, for physical events, we need to do a bit more experimentation on how we can reduce the carbon footprint.
And yes, that's where hybrid, I think is going to be a huge thing for people because there are some people who, and it comes back to the goals of what a conference is… I think there's kind of like five different goals.
There's the knowledge and inspiration. People are just want to learn something new or get inspired.
A subset of that would be the kind of people who are just intrigued about how another company is working. So they're my competitors, how are they working? Which is slightly different to that just general knowledge and inspiration.
Learning new skills would be the third one where people want to learn how to adopt a new methodology or pick up a new tool.
And networking is a separate forth kind of goal. Not everybody has that goal. So I think that that one is much better served in person.
But some people might just want to know what their competitor is doing and might not want to fly halfway around the world just to satisfy that goal. So that's where that kind of hybrid model can come in. If somebody just purely wants to see the curated content, then they can apply for the virtual kind of ticket for that. Whereas the people who want to get more of that atmosphere, that networking that kind of, I don't know, I personally come out on a high with that many people around, it's just really inspiring and you don't get the same kind of effect from a virtual one.
So that's why I think that hybrid model is going to be where we're going to go.
Gerry
Do you think there are any conferences that have been successful in going online that you've experienced so far? Or is it all too new to say?
Rory
I've done quite a bit of kind of stalking at other conferences just to see how they're working. There's been a few different conferences that have been quite open in terms of how things have been going. I can't remember the guy's name now. He was just commenting on CubeCon and, and just some of the challenges and… his view actually was that he's going to stop doing virtual ones. He's just going to wait until he can do physical ones again.
I have a different view. I think that the hybrid model is, is the way to go. And, but there's other ones like SaaStock and Web Summit have been very good with their promotion, they're very good marketers. So been kind of just watching it, how they've been working, there was a couple of different models.
There's the, kind of just that to try to turn it into a training kind of model, let's try replicate exactly where we've seen some of the use kind of graphics to make it look like you're at a virtual event and there's little people walking around on screen. So there's different styles in terms of, I don't think I'm, I haven't seen their internals and their reporting and stuff, so I don't know which are successful and which aren't, but I think what's really interesting is that a lot of people are experimenting. And I think over the next few years, what we're going to see is that something is going to be established as the right way of doing things.
Gerry
Yeah. I guess just the way we've gotten used to running workshops, using video conferencing tool and, you know, Miro or Mural or whatever, and you know, working with, with the new constraints. And I guess that's one of the things is that some of the constraints associated with an in-person conference disappear when you go online don't they or are modified by going online.
Rory
Yeah. Massively. So the, the biggest constraints of a physical event are time and space. You're limited in time and you're limited in space because these venues cost an arm and a leg to rent. And then the, the kind of the setup and tear down…
Gerry
You've gone very up-market with the RDS, haven't you?
Rory
[Laughs.] Well, in Dublin, there's no, at the scale that we're at, it's the RDS or the Convention Centre. So there is very limited choice in Ireland or in Dublin at that scale and the Convention Centre is a multi-level building. I don't like that, people are splitting up across levels, and I much preferred to be, the RDS is a single level kind of flat building where I think it just leads to a lot more interaction and mingling. So we're kind of limited in where we can host it, but yeah, so, sorry, just back to the question about the limitations it's time and space.
Both of those make an impact. So space means that you're kind of trying to design around the limitations of how much space you have. So you're, you're kind of putting things on top of each other that don't necessarily need to be that close and in the past, and this is the problem every year that as our numbers change, we're, we're redesigning our space. And sometimes we put things too close to each other. And as much as we've tried to get the sounds not to bleed into the different areas, it hasn't been as successful as we would've liked. So we just, every year we're learning, okay let's not make that mistake again. But there's definitely problems with space and sound. That goes away. There's no risk of things bleeding over into each other. Or you underestimate, because you always have to make estimations of how interesting is this talk going to be versus this one, because the other bit of space and time is that you're limited in time.
So you, you tend to have to run sessions in parallel, and then you're estimating, okay, I'm going to put that session in this room, which is slightly smaller, and you get it wrong sometimes. And there's people standing all over in that room, whereas the other room is relatively empty. So those are some of the constraints for physical events that you're trying to work around. With a virtual event, those constraints go away. And so what we've done actually is we decided, we normally do a two stream kind of two sessions on in parallel. Whereas we said, let's, let's re remove that, there's no need for that. So after we experimented a lot, so as I was saying in February was when things started to change for us, February 2020, sorry, as coronavirus was kind of in the Western world's coming to, to the floor.
We were actually in, in the States doing community events. We run community events around the world, and we were in San Francisco and New York running events in February. And there was this news of some virus starting to pop up. And then, so for all of our March events, we haD to pivot, but it was the best thing because we just, we were forced into doing it lean and doing it quickly. So all of our March, April, may, June events, every time we would change something, tweak something. And it was a huge amount of learning because our assumptions that I started with in March of kind of almost replicating what we would do changed dramatically by the time we got to the event. So the big ones where people can't stand sitting eight hours in front of a screen, so we can't do a full eight hour kind of schedule like we would. And the second one was, people when they're watching videos, they are going to be a bit distracted.
So it's better to kind of give it to them in their own time. So what we do there is we do 10 recordings, or for the main event, what we did was we did 10 recordings released on the night before. And the reason the night before was because we're based in Dublin. So that would mean that people in the US would be able to access them before they went to bed or, or get them first thing in the morning. And then we started the conference from 12 and did a single track and our track then, it wasn't the talks because the talks had already been released, but we got all of those speakers together for panels. And the idea was, you've watched all their talks. Now you can get them debating against each other with their slightly different points of view, and then the audience can join in with questions.
So that was that was the result of a lot of experimentation over the summer to try and figure out what format people liked best. And again, it was a huge amount of learning, running that conference because our community events are in the 100 to 150 people. We got a lot of questions this year because that, that would be the style that we would do it, except this year it was online. So people were going, why are you calling it the Stockholm event? Or why are you calling it the Helsinki, the Copenhagen event when it's not happening there? But the idea of our community events is to kind of help stoke up the local kind of scene. So it would always be local speakers, tended to be when it was physical, local attendees. And that was the kind of the thinking there that the community events, they're free events, but the community events for us, it's not just purely a… it is definitely about giving back, but it's not a hundred percent altruistic.
We use them for experimentation in two kind of main ways. One is format. So we introduced kind of debates into our London community event just to try test it out as a format, whether people would really like it. So that, that actually worked really well. And we were able to bring that format to the main conference. And then the other thing is the speakers. So we have a lot of people applied to speak, and it's really disappointing to have to say no to somebody that unfortunately we can't get you onto stage because sometimes it's really hard to tell from the blurb that's sent in whether the person is going to be good or not. And we use our community events going, listen, we're going to take a chance. Hopefully this person is good. If they are, then we can get, ask them, invite them to the main event. And what it does is it increases the quality of the speakers at our main event, while also giving people more of an opportunity to speak and improve.
Gerry
My next question, I'm not sure if it still makes sense, cause we've talked a little bit about it. And I've said, of course, going online raises a new set of constraints. What are the main ones and how do you manage them?
Rory
Yeah, so I was saying the time and space have gone away with moving online, but the new constraints are that you don't have people's attention or you're competing a lot more for people's attention. So in a physical event, they're there, they're, it's almost difficult to step out and kind of avoid attending. Whereas in a virtual event, you're competing with email, you're competing with just walking away, you're competing with somebody tapping them on the shoulder, asking for some something at work. And so, so that becomes a lot more difficult. How do you work in a situation where you're competing for attention and the kind of the thing there that that's what led to a lot of well let's give people, the videos that they can watch in their own time and people, the biggest feedback about that was that people love being able to watch them in double-speed.
It's funny because I watch all of them in double speed myself when I'm just making sure that okay. But it is, I don't think it's necessarily a thing of not paying enough attention. I think often what I find is that I'll play it and then there'll be a bit that I really liked. So I'll, I'll rewind a little bit, slow it down and then really take some notes. So there's, there's times in a physical event where I'd be, Oh, I wish what, what did they just say there a second ago? Or whatever. So it does give you a bit more control there. As long as you don't focus on something else while listening to it would be my thing, but that is a big challenge that people are trying to multitask. And so that's one that we're really going to work on as well. So the good news was that people were really liking that pre-recorded approach, coupled with the fact that the people would come live and would answer questions live. So that was, that was the big thing. It wasn't, if it was just pre-records people were saying, well, what's the value in coming? So the value is that all of the people will turn up later on. You can log in live and ask them questions based on what they've kind of talked about.
Gerry
I can see that working well with a smaller group of paper, up to, well, I did, I don't want to nominate a number, but when you talk about 2,000 people, I mean, it seems, does it scale up to those sorts of numbers?
Rory
Yeah, it's kind of the, I can't remember who came up with it, but it's the 1:9:90 kind of percentage of all social networks seem to follow a 1:9:90, which is 1%, very heavy engagement, 9%, somewhat, every so often might engage and 90% just listen. And it seems to be following back kind of an approach that, cause I was wondering, I asked a few people who kept logging into the live sessions, but never asked anything. And they were saying, no, they were really interested in the conversation that was happening. So they just wanted to see what other people were asking and the conversation. So it wasn't, even though they weren't asking questions, they didn't feel that they, they weren't getting the value. So it is that kind of 1%, we definitely noticed that there were a couple of people who were asking questions all the time. But yeah, it was just the, it, it did seem to scale well.
Gerry
Hmm. That's interesting. And that leads me pretty nicely to my next question actually. You told me in an email, I think that you were user centred in recasting UXDX as an online conference. Can you describe the process you went through? Like, did you do any form of user research or what, how did you do it?
Rory
So it kind of started last year. When we weren't thinking about an online conference, we were just every year after the conference, we ask for kind of feedback. And so we get hundreds of kind of submissions of kind of things we could do better. And what we did was a process called DIBS. I'm not sure if you've come across that, but it's data and it's Data, Insights, Big leaps, Bets. I stole this from Spotify. Basically, you get, so let's say you have a lot of quantitative and qualitative data, so you have lots of data. And so what we did was we spent a bit of time just massaging that data, putting it into a, we use Airtable and as a system, just to kind of get all of the information in. And then we started kind of quantifying some of the qualitative data kind of booking it up.
So we had all this data come out and then the reports and what we did then was an insights kind of piece. So we started looking at the data, started trying to clump it up. So an example would be, we do 30 minutes for our talks. And some of the feedback was you can't get into detail in 30 minutes. It feels like things are just getting warmed up and then they move on. And then other feedback was saying, I love the format. 30 minutes is fantastic and don't change it. It's great. I get to see that broad kind of spectrum. And you're kind of, it can be very frustrating when you get that conflicting feedback, but what it allowed us to do, because we had put all that data into one place is we linked that up to the goals that the people had said. So whenever everybody buys a ticket, they outline what goals they have.
And that kind of tied to those goals that I was saying, at the start, are you looking more for the knowledge and inspiration, or are you looking more for the skills? And we found that looking at the feedback, the people who loved the 30 minutes, we're looking for knowledge and inspiration, and they just wanted to get that kind of broad spectrum, lots of information. The people who wanted the new skills were kind of saying, okay, yes, you told me about this really interesting new topic. I'm very interested in this, but now moving on to another topic and they don't get the time to get into the detail. So that was kind of the feedback that we were seeing. And that led us to kind of move into the, that was insight then was the the different goals or having different approaches?
The belief was that we need to do more workshops for the people who want the skills, because I don't believe personally that a talk format is the right way to learn a skill. Somebody telling you how to do something versus you actually hands-on trying it yourself. And so that led us to our bet was keep the 30 minute format, but we're going to massively increase the number of workshops. And, and then one of our speakers, actually, he just, on the way home on flying back to the States, Alan Klement, he's the author of the book When Coffee and Kale Collide [actually Compete] he just said, we really should market it differently and make it explicit, and he said aggressively explicit that there is a difference so that you don't have that mismatched expectations amongst the audience. If you want your knowledge and inspiration, go to the talks, if you want your skills go to the workshops.
So that was his kind of feedback as well. So that was just an example of how we use that DIBS process. We came up with a lot of different bets and, but that was really useful because come February, what that meant was we had a really good understanding of the goals that people wanted from the conference and how we were delivering or not delivering very well on those. So when it came to kind of looking at a virtual conference, it wasn't just starting from scratch and going, okay, well we do stages and we do talks and we do workshops. And let's just, what's the quickest way of making those online? We were able to go back to the user needs, the user goals and be able to then figure out what's the best way of trying to satisfy those.
Gerry
Was there anything about the findings that really surprised you?
Rory
One of the things that has really surprised me about the last few months of experimenting is, and it's that age-old user research problem of what people tell you and then their behaviour. And it's the networking element of it that everybody says networking is like one of the most important things. And even the people, when we were advertising online events, there were still saying networking was a huge goal of the conference, but then the amount of people who participated in the networking pieces… So it wasn't, I'm probably painting it a little bit too bad, but my bar for success is very high. But like, so our networking was probably 10% of people and got involved in the networking. And I would like that obviously to be a hell of a lot higher
Gerry
To reflect back to a word you used earlier, you talked about serendipity. I mean, a lot of the networking is serendipitous, right? … It's not necessarily going to the form of arranged drinks. It's bumping into somebody at the buffet or whatever.
Rory
Exactly. and it is, and we were talking to some people after the conference, because what we had done was a for want of a better word, speed-dating type of a networking approach where you just randomly, you clicked the button and you randomly got paired with another attendee. And there was a countdown timer of two minutes at a question to talk about. So it was a product-related question, but just to skip the small talk and what people were just saying, it's quite scary because it's like, imagine in a physical sense, you were standing outside a room with a closed door and you didn't really know what was in there and people, and somebody said, Oh, just go on in there and network. So it, it just seemed like, that physical analogy was quite good for me that it was, well, I'm not just going to randomly open the door and walk in. And so it's something that we need to experiment with more.
Gerry
I was a bit confused when you said the next conference or the last conference is October. Cause I thought the 2021 one is scheduled for March, is that right?
Rory
October is our European conference and APAC is in March.Gerry
And where is APAC? Oh, it's online. Of course. Yeah.
Rory
This year it's online. But we're looking to host it in Sydney. It was intended to be hosted in Sydney. But obviously for obvious reasons, we can't do that this year, but hopefully come 2022. I really hope this, this virus, the vaccines will have kind of solve this problem for us. But we'll see, hopefully come 2022, we'll be physically in Sydney.
Gerry
I often wonder if the the capacity for conferences. I mean, we've got as you know, UX Australia is a very big conference here and there's, there's a bunch of UX- CX- whatever-X related conferences happening worldwide. And there's been an explosion in the last few years. I mean, how much demand is there?
Rory
Ours isn't a pure UX conference. And if what you're looking for is just pure UX and two days of UX content and UX, latest methodologies, and things like that, ours actually isn't the best conference for you. But if you're looking for a way of improving how you work as a product team. So UX is an incredible, incredibly important aspect of a product team, but is often in a lot of companies I talk to it's either not consulted or it doesn't exist, or it's kind of given a bit of lip service. So I've seen a lot of companies where they go, okay, we'll, we'll set up a UX kind of team and you'll go sit off in the corner and you'll give us reports every so often, but they're not embedded. Our concept isn't to get that team off in the corner, getting them to work on the best methodologies if it's not having an impact. So it comes back to our goal is making the change happen in the teams. It's not about just ensuring that kind of people have the latest practices. It definitely is about the latest practices and how, the best ways of working, but it's a very heavy influence on the product team as a whole,
Gerry
If people are interested in that topic particularly are in indeed in online conferencing, they can go to uxdx.com I presume. Is that right?
Rory
Yeah.
Gerry
Okay, great. And as usual, you can get a transcript of this episode@uxpod.com Rory Madden. Thanks for joining me today on the user experience podcast.
Rory
Thanks for having me. Thank you.