Published: 17 July 2019
Gerry Gaffney
This is Gerry Gaffney with the User Experience podcast.
My guest today is a designer based in Aotearoa, or New Zealand. He's currently working at the Government Service Innovation Lab, which is part of the Department of Internal Affairs in Wellington, I presume.
Previously he's worked at the Australian Centre for Social Innovation. He has a Bachelor Degree of Innovation from Victoria University of Wellington and a Master's in Design and Engineering from Estonia's Tallinna Tehnikaülikool.
Matt McCallum. Welcome to the User Experience podcast.
Matthew McCallum
Thank you.
Gerry
Can you start by telling us what the service innovation lab is all about?Matt
Yeah, so I guess I'm a couple of years back it came about to really transform how the government delivers services and the idea was to really test through doing and, and the team really uses an informed evidence based approach, along with a lot of design methodologies to do a whole bunch of experiments on how to offer services in different ways.Gerry
And this is a whole of government initiative, is it?
Matt
Yes. So the Service Innovation Lab is funded by a bunch of government agencies and so a lot of our projects are focused on looking at services that span across multiple agencies at one time. So users would be required to bring things or access a whole bunch of government agencies.Gerry
And I guess for people outside of the region that we're in, they may not be aware, but New Zealand is considered a quite the leader in many ways in user cantered design. Would that be a fair characterization?
Matt
Yeah, I think in certain areas there's a really big drive to kind of incorporate the user at the centre of what they're doing. But yeah, I think certain government agencies are finding that journey a bit easier than others.Gerry
Yeah, I can imagine. Now a blog post of yours caught my attention. It was entitled Service Innovation Lab: Shaping government services around entry to adulthood. What was the driver of that project?
Matt
I guess the driver was really to understand young adults first interactions with government and it's an important opportunity to set a positive tone as the relationship that these young adults will have with government will last their entire lifetime. So this kind of first interaction can really kind of make or break how they perceive government in other interactions. So they might need to have, especially at this time when they're transforming their lives quite a lot, finishing school and then they've got a lot of big life decisions to make.
Gerry
Was there a particular government department that was interested in this or was this the Service Innovation Lab driving its own agenda?
Matt
No, this was I guess the main sponsor for this project came from the Department of Internal Affairs and that department is responsible for a whole bunch of identity-related attributes and services. So they do all the births, deaths and marriages, passports and things like that. So they are really a big identity holder across government. And so they were really keen to try and understand this. But we also knew at that time that there's a whole lot of agencies that were interested in developing services, particularly around young people and how they are experiencing government for the first time as well. And there was a whole bunch of assumptions that had been kind of pulled together through the different governance groups that help the lab operate and that we wanted to try and unpack and explore during this research phase.
Gerry
Do you want to give us an idea of what sorts of assumptions they were?
Matt
Yes. I think one of the big ones was around what age people were accessing the government for the first time. And also what types of things and reasons were leading to that kind of trigger event going into a government agency or whether they even understood what a government agency was...
Gerry
Yeah, I guess entering adulthood is a very fluid period and it may cover a wide spectrum.
Matt
Yeah. So that was a big challenge. We had a reasonable kind of timeframe to do the initial research in, but that can cover so many different events. And even though it was quite a broad topic, through talking to even just a small amount of individuals, we could already start to identify certain patterns or triggers and factors and the childhoods that maybe helped them change how they go about making decisions in their lives.
Gerry
Now you said the project had to do with identity management. What does that mean?
Matt
Yeah, so we tried to, I guess, use the perspective of the young adults to try and articulate things like this rather than the technical definitions. So this would kind of cover how people access services, especially online using logins such as Facebook and Google, but there's also one called RealMe, which is a government-run verification service here in New Zealand. And we also wanted to explore the type of identity related items that people carried around with in their wallet or bag and things like that and where they keep them at home. And just to try and understand the types of behaviour associated with certain identities and whether people kind of treated them differently that would help kind of inform maybe some different designs of products the various agencies were looking at.
Gerry
Now essentially, and forgive me if I'm oversimplifying, but the exploration of young adults' identity management relied very lightly on a series of in depth interviews. Is that correct?
Matt
Yes.
Gerry
And you had a, you had a sample size of 10 people. Did you run into any criticism about this being too few?
Matt
No. So I guess we had a lot of freedom to structure this work and we worked really hard to find those ten young adults that we thought would give us a pretty good set of perspectives around how they were approaching certain decisions in their lives. And we also spend an hour-and-a-half with each of these young adults using a semi-structured kind of interview approach to understand themselves as individuals and then how that were using the identity in those various kind of platforms and ways.
Gerry
And these were one on one sessions. Did you bring them into your location or did you go out to their environment?
Matt
Yeah, so we meet them. So there's two interviewers in every interview and we met in kind of public places, but we often went to them as much as we could. So for instance, we went, interviewed some boarders, up in Whanganui and so we travelled up there for the day and interviewed them at the school and things like that. So we'd kind of get a bit of a bit of idea around the time, what was going on in their lives. But yeah, given that they were young adults, we took some extra precautions to make sure we kept everyone involved safe during this research process.
Gerry
What was their age range?
Matt
The age range was 16 to 21.
Gerry
And I guess there were some requirements for guardianship permissions were there?
Matt
Yeah, so I guess over 16, they could kind of sign up to the interviews themselves, but we always went over and above that and made sure that we had sign off of teachers and parents wherever possible and made sure that everyone was really, really comfortable with catching up with us and knowing what we were going to be talking about.
Gerry
Of course a big challenge with working with a relatively small group of people is trying to, I mean, you can't really get a representative sample as such, but trying to get you know, a range of people, presumably you looked at some Māori people and some Pākehā, how did you go about all of that?
Matt
Yeah. We didn't go to too much lengths to make sure we kind of ticked all those boxes, but it was more just that what are some really different perspectives that we can get. We knew that there was some understanding within the organization about particular groups already. And that there was some other research happening that was going to tap into people with disabilities and people going through the justice system and things like that. So we could avoid targeting those particular groups and that way we kind of just focused on ones that we thought would be really interesting or going through maybe some interesting times in their life. And that proved to be quite lucky for us. And so again, it was just the start of a much longer piece of work. So we didn't feel like we needed to tick all those boxes right at the start. It was more just about, let's start some conversations with some different people from different parts of New Zealand, both rural and urban, experiencing different types of schooling and things like that. And then let's see if that helps us make some better decisions on who we might want to talk to next or who might be wanting to talk to some more. And that was kind of our approach to the project rather than really going out and overwhelming ourselves with data effectively.
Gerry
That's so true, isn't it? Having a starting point is, you know, so valuable in getting some emerging themes and then I guess validating them or expanding upon them later on.
Matt
That's right. Often people try and go out and yeah, kind of tick all those boxes from the demographics and make it as representative of possible and they just overwhelm themselves when quite quickly you can narrow down on things that you find more interesting or unexpected and you only need to talk to a very small amount of people to start discovering those things.
Gerry
How did you actually do the recruitment? How did you find the 10 participants?
Matt
We were really lucky. A member of our team a has done a lot of work with Volunteering New Zealand, so he managed to tap on a whole lot of people that he knew within that kind of network. And then we also have a few people at the lab that are from the school system. We also kind of wind via friends of family and things like that to really kind of just pick up some people quite quickly. We did have a reasonably short timeframe to get those initial interviews done to see how close we were towards, I guess, ticking some of the objectives for the project.
Gerry
So did you have a, like a specification of what sort of people you wanted going into it or did you just go out and say, we want young people entering adulthood?
Matt
Yeah, we had a bit of a chat. There was some other research that had been done by the lab around entry to tertiary. So moving in from a high school through to university types of education and we kind of knew from that certain events that are happening in people's lives that were quite interesting and where identity was going to be quite an important factor. Things like signing up for a student loan, getting a driver's license, moving out of home, those types of events. So we tried to give some examples of those types of things and that kind of helped our network filter out people for us. So we were quite lucky in that regard.
Gerry
Now you wrote that one of the goals of the project, I guess it was a kind of a side benefit, was to test the measurement framework and there seems to be quite a focus over there on what things we should measure. So what were you measuring? Can you tell us what the implications are for the lab?
Matt
Yeah, so I guess, measurement can be quite a big task to take on. And we were really focused on not overwhelming us with too much extra work, it was really about trying to identify what good work looks like to us. And so really reflecting on what we were doing in our week to week practice and going, okay, this is what we want to achieve. Did we achieve it? How well did we think we achieved it? So really just measuring what we could really quickly and easily to give us a bit of a barometer on whether we think our hypothesis, if we do this type of work in this type of way and therefore this should be a positive impact on society. And so rather than kind of going through some elaborate evaluation methodology, we can kind of just really quickly go, well we think this process doing quite well. So therefore we should think about buildings some measurement and to these solutions as they start coming about and therefore later on, this project would be worth measuring and evaluating to a fuller extinct once it's kind of live out in the world. And so there's the kind of things that we're kind of just trying to break down and do as we go. So we get a little bit of a, baseline kind of data around how long it takes to do certain types of projects, viven that we're quite a small team and to what kind of level of evidence can we get that helps inform kind of bigger departmental decisions, and at what time frames that kind of looks like. So really just measuring some quite small and obvious things, the team would just do a bit of a survey, individually each week and then they would all get kind of collated at the end based on the outcomes of the project.
Gerry
How big was the team?
Matt
So the team for this project was a, I guess, a bit bigger than usual. So there was, I think four designers on it. And then we also had some, help from a scrum master and things like that as well.
Gerry
In the in depth interviews, how many people were involved in conducting those interviews?
Matt
So it would be a lead interviewer and then a secondary interviewer, often taking notes or photographs and things like that, helping run the activities. And so in the room there'd be about three people at a time
Gerry
Plus the participant or three in total?
Matt
Three in total.
Gerry
You mentioned that you designed a series of activities and tools, but your blog post was kind of mysterious in not giving us any detail about those tools and activities. Can you tell me a little bit about them?
Matt
Yes. And I was planning on doing a follow-up blog about the tools, but yeah, due to some publishing constraints it can be a little bit difficult to kind of guarantee when and how that might happen. And so we used cad sorting was that club primary tool and it's a little bit different to maybe how card sorting looks in other areas of the design process. So the types of cards we were using were picture cards and they've been particularly designed for the types of interviews that we were doing. These are something I've picked up throughout my career. So the company that designed them is based in Singapore but I don't think are available anymore. But basically they've got really emotive pictures or just scenes of everyday life and objects and things. And so we've got people to kind of sort through them, based on certain questions like, choose three pictures that represents what becoming an adult meant to you and was one of the questions we'd use to open up the interviews.
And so that way it kind of gives them some independence, some time to really just figure out how they want to conduct themselves in this interview. And it gives us time to kind of compose ourselves and, and get ready, lining up our questions based cards they select. And so I was a nice kind of an icebreaker kind of thing but at the same time the cards ended up, it was a deck of about 60 cards and which is quite a big bunch to give to someone. But even from that 60, we actually had quite a lot of patterns, which wass a big surprise to the team across even the different age groups in that as well. So even if there was different pictures, often the answers were quite similar. So it helped us really zoom in really quickly on some of the kind of key events happening in people's lives.
Gerry
It's funny having artefacts of some sort, be it cards or whatever really opens up the possibilities, doesn't it in those sorts of interviews?
Matt
Yeah, I think it made it easier to, to get started, especially for these younger cohorts. We noticed that the, the boys are a little bit quieter than the females. So that really helped them to kind of feel like they could kind of express themselves in a different way. Also the tools that we designed were to help recording when we got back to the lab as well. So we could quickly take some photos during the interview and I'd help capture a whole bunch of information that we didn't need to take so many notes down and then we could kind of turn that into quantitative data later on. So I was using just a really simple axes written on a piece of A3 paper and then getting people to place things in certain places and it helped us really quickly record that.
And then I could just turn it into a grid when I got back to the lab and basically turn that into numbers and we can make graphs out of it. And it just really quickly starts to articulate a whole bunch of data. And then we had that all up on the wall during our synthesis process. So it really helped us be back in that interview when we're going through the transcripts and audio and seeing why they saw that. And we could dispute certain people's experiences 'cause we had their different job activities up on the wall, then go, it's interesting where they put these, these two different things and, and have a bit of a chat about that. So it was a really good process I think, and everyone enjoyed it and that's being reused for subsequent research after this project.
Gerry
What sort of labels did you have on those axes?
Matt
Things like, when people were acquiring IDs, so how old or, or new, how often they use them? So very often to not very often. So it was in regards to a whole bunch of IDs. We also had a whole bunch of activities such as managing health, managing my money, how I get around on little cards. And then we got people to put them on axes of in control to least in control. And that was to try and understand people's independence and, and control over their lives. 'Cause we had a bit of a hypothesis that, that might shift through the different age groups that we were interviewing and, and it did prove to be true. And that really helped us quickly identify maybe some anomalies so we could kind question further.
And kind of also gave a point away from face-to-face kind of conversation as well. So we could kind of point at things on the table, which I think made it just a little bit easier to kind of take the formality of the interview away a little bit. Having things on the table and keeping things a bit interactive. So it meant that when discussion slowed down, we could kind of segue into a different type of activities to lift it up again and then keep it going. 'Cause one-and-a-half hours with young adults is a reasonably long time when, when they don't really know us and we don't know them. So we're very aware of that.
Gerry
It is a long time isn't it? You alluded to it there, but you found out some things that you expected and other things that were surprising, which is of course typical of this sort of research. Can you give us perhaps a snapshot of your findings?
Matt
Yeah, indeed. We were really surprised at the lack of knowledge around interaction with different types of services. People kind of had a sense that certain IDs were more important or more serious. They recognized, I think there was maybe two out of our ten that could like clearly identify a government entity. Which was a big debate within our own team around whether we needed to do some particular kind of activities or questions to articulate that. But that kind of came out on its own based on how people use the language around things like the IRD company, which is our tax department here. That's not kind of the correct name, but they kind of knew that it was someone that you sent certain types of information... So that was one thing that was quite interesting.
The other was the amount of help that these young adults who are receiving from guardians and friends and family and friends of family friends and things like that to achieve the things that they were trying to achieve, such as like signing up for IDs, getting their driver's license and things like that. So even our older interviewees, they was still having a lot of help, which was quite surprising because it meant that some of the pain points were expecting to hear about, might not have been experienced by the young adults themselves and potentially were experienced by their parents. So that maybe opened up a big avenue for doing some further research, either with people who are a little bit older or with that the parents to really try and understand if the solutions that we were thinking about or maybe testing need to be tested with someone else and in betweenGerry
That can be really challenging when what you think is a one-on-one interaction actually turns out to be a intermediated or mediated in another way.
Matt
Yeah. And also the use of different platforms. A bit of research was being done online, certain forms were being used and interfaces, but then also a lot of face-to-face resources, so a citizens advice bureau was being used by one of our interviewees quite often. And that was maybe through her interaction with them as being a migrant to New Zealand as well. So it was kind of a trusted service, even though she knew how to do everything online, was very capable, she'd still often prefer to do things face-to-face. And so yeah, it was quite an unexpected, I think interaction for this particular age group and just their, I guess awareness of technology and the impact it was having on their lives. They weren't maybe as keen to use it as many people involved in a stakeholder group where we're thinking they would be. So that kind of helped shift some of the thinking around the types of products that could be developed from research and different types of research that needed to be done to really start to articulate whether an app's the right approach and, and things like that.
Gerry
I guess that leads us into how will your findings be applied?
Matt
Yes. I think that being shared almost as we got them, were being shared across different teams and that we're kind of exploring some less spices to us. Digital identity is a bit of a hot topic over here at the moment and especially within Department of Internal Affairs. And so making sure that as we were going were sharing that very quickly. And there's also a few other organizations that have been doing some similar research with young adults but from a different angle in relation to their particular service. So we're kind of collaborating with them and making sure that we could get them their findings. Um, but I think the next step because of this interaction that guardians were having in a whole lot of these processes, there could be some really quick wins that could be explored to really maybe make it a bit more official that, that there's a lot of interactions happening when it comes to signing up the, and identity by the parent or something like that. And so how do we kind of make that a bit more official? And then when a parent calls up about a passport application, it's a bit easier for them to kind of get the information that they need. And at the same time, when the young adult is ready to take control of that or doesn't want the garden to be able to interact in those processes, how do they manage that? So that's maybe a little bit of a longer process than maybe what was first anticipated.
Gerry
What would you do differently if you were doing this research again?
Matt
I think definitely trying to talk to a few more people in the older age bracket and even extending it out a little bit. As, yeah, we didn't really capture that many interactions that were done completely independently by the young adults. I think all of them were having help of some description to, to get things done. So it'd be really interesting to understand kind of the, at what point does that shift and where the majority of interactions that they're having with different types of services are they doing on their own. That was even true for people that were away from living away from home. So I think, yeah, some of our research that we're doing back at the lab, just some desk research was showing that there's an international trend, across western countries called emerging adulthood, which stresses that some of the interactions people by kind of 20 would have done in the past are now kind of happening between 25 and 30, so maybe about five years out even in our cohort. So I think, yeah, that would be one of the big things for me,
Gerry
Is it cool to working for the government or is it a Kafkaesque nightmare?
Matt
I think there's some real potential to make an impact on people's lives from working in government, that's what drew me here. And for many people this is the only place that you can do certain things. And I think especially some of the other work that the lab's been involved in around entitlements where they're directly helping people in need get what they're entitled to, I think that's really important. And a lot of that's quite quickly done using the design-led with a multidisciplinary team. We've got developers in the lab and things, so we can really quickly build and teach things and get them live, on a really small scale to test whether something's going to work and things like that. So I think, there's definitely pockets of great innovation happening and people trying to do some really cool things, but at the same time there's a lot of convoluted processes which can go against many of the kind of modern ways of working. As more and more successes happen from these small experiments I think yeah especially when we're making that things faster, cheaper, and making the services more effective and efficient, I think that's when there'll be a bit more of a shift and they will be able to streamline some of the internal processes as well...
Gerry
Matt's blog post was called Service Innovation Lab: Shaping government services around entry to adulthood.
Matthew McCallum, thanks so much for joining me today on the user experience podcast.
Matt
No problem. Thanks for having me.