Published: 7 August 2020
An exploration of how we understand
Gerry Gaffney
This is Gerry Gaffney with the user experience podcast.
I have two guests today. The first is author of Seductive Interaction Design and creator of the Mental Notes card deck. He's particularly interested in workforce learning and organizational development.
My second guest is director of information architecture at Normative in Canada, and a former professor of user experience design at Kent State University.
I've asked them who today, because they've recently published the book Figure It Out: Getting From Information to Understanding.
Stephen P. Anderson and Karl Fast, welcome to the User Experience podcast.
Stephen P Anderson
Great to be here.
Karl Fast
Thanks for inviting us.
Gerry
Well, Stephen we might kick off with you. Other than fame and fortune, can you tell us a little bit about why you wrote this book?
Stephen
Yeah, I think, gosh, it's one of those things that's hard to pinpoint the exact origin, but if we go back 10 or 15 years, I've always been fascinated with visual communications and, and infographics and concept models and the like, and I think the teacher in me wanted to scratch at that interest and figure out why is it that these things work? Why is it that as UX professionals, for example, why do things like our artifacts and our models and our wireframes and site maps and all these things we create, why do they work? Why do these visual things work? So I think that's probably the, the centre where the book started from, but of course, you know, as you get into why these things work it blossomed and bloomed into something much more until, I think the visual part is only three of the 15 chapters in the book. So there's a lot more that that grew into,
Gerry
Okay. And we might jump over to Karl. Karl, thinking happens in the brain, right? Let's throw you in the deep end with this question.
Karl
So part of the book… The answer is that yes, thinking does happen in the brain and that's really what we've been taught, but that is not the only place that it happens. And so a lot of the book is sort of meant as this corrective to the way that we have looked at the relationship between information and how we use that information, the way we make sense of it, the way we reason and problem solve. And what we've been told is that we take information into the brain, we perceive that information, the brain then does all of these, all of this thinking work. And then after the brain does it’s thing, that is when we act on the world. And there's this whole area of research in cognitive science over the last 20 or 30 years that has gained a lot of steam, which broadly goes under the note, the term embodiment or embodied cognition. And the argument there is that a lot of our thinking depends on our bodies, depends on how we use our bodies. It depends on how we use the tools around us. It depends on the physical space that we live in and all that is not merely some sort of extra piece of the, of our cognitive apparatus, but an intricate part of it. So yeah, thinking does happen in the rain, but it happens in a of other places as well. And we need to tell a richer story about that.
Yeah. And you could see this back in the 1980s. And well, even earlier than that, where people were trying in the scientific world to develop models of human cognition, and you would get these beautiful little pictures, which described the human being basically without a body, like your hands are called manual motor processors, your eyes are, you know, like visual processors. Quite often they don't even have feet. So you literally are like this person that just sits there and doesn't, doesn't move or interact with the world in any meaningful way. And it's just simply information in and then cognition. And that's it. And it’s a very stripped down experience. I mean, if you look at those models and then you look at the reality of using a smartphone and you're like, there's an obvious and huge disconnect.
Gerry
Okay. Talking of disconnects, Stephen, near the start of the book, you have an image of a horribly familiar kind of information sheet from a hospital. And the purpose of the information sheet is to provide guidance on treating the diabetes that your four year old child has. And it's one of those, as soon as I saw it, I thought, Oh my God, that's so horribly familiar. Why is it that certain industries or professions or organizations seem to be able to churn out such horribly inappropriate infographics or information sheets?
Stephen
Yeah, in that case I you know, I did the redesign that I talk about in the book and show the before and after. And it was largely just for our family, for my son who was four at the time. But I did at a later date, bring that back to the hospital and share it back to them in the hopes that that would improve, or they could adopt some of the ideas I had modelled there, which, which they did later on, but the way that hospital was set up and the way I think many hospitals are set up is that particular document was a product of the legal group. And so while we were there, there were probably a dozen or so documents we received. Most of them, you know, you just read and sign and things like that, but this particular document why it was so frustrating was this is the thing you're supposed to put on your refrigerator to help you figure out what to do throughout the day for the next many years.
And so understanding the information there was really, really vital. And so that's why I think I put that one under the spotlight and said, why was this particular document not given more attention or given different treatment. And again, the simple answer there is, it was the product of a legal group whose primary goal was not conveying understanding, but to get the information out there in a way that could be signed off on and documented. And I think part of the aim of the book is to say, okay, regardless of whether we're teaching legal groups to make better forms or talking about other groups that need be involved, this idea of treating information as a resource that could be made more understandable or could be made easier more easily understood. I think that's, that's kind of the theme or the arc of the book.
And we want to show that this is something anyone could do. But I think we have to start more fundamentally with recognizing that there is a problem in the first place that there was a problem with this document, which I don't know how many years and how many parents have gone through the hospital saying, yep, this is the form. This is the way it is. So I think step number one, first chapter in the book is to acknowledge that there is a problem when we accept these legal forms or sign that terms of service that none of us read or understand, or you know, make guesses about whether we can park in this parking spot right now or not. And, you know, saying this is, this is a case where the information is there. Yes. But it has not been formatted or managed or given or shown in the way to us that is easy to understand is a problem.
Gerry
It's funny, isn't it? Because it's almost like as a society in general, there's a lack of competence. And, you know, we've had people like Dave Grey and Alberto Cairo and others talk on, on UX pod, you know, about this sort of I guess ability. And it's just, it seems absent. I mean don't be teach this at school?
Stephen
Not that I'm aware of, not in the States. Karl?
Karl
Well, it's not always that we don't teach it. I mean, part of, part of it is, but we can't be experts in everything. And also in a lot of the work that we have, and it's important to recognize that, you know, work is structured around incentives. And a lot of us, a lot of people have creating information as part of their job, but their fundamental incentive is not to create super understandable information. That's why we have design as a profession. Right. So the nature of, so it's not surprising that we have lots of information that is understandable. There's different kinds of incentives that drive a lot of that.
Gerry
Yeah. Gerry McGovern who’s been on the show also talks about the fact that we should pay people to remove information and to delete information rather than to create it.
Karl
Yeah. Or you take like, you know, like take, say privacy examples. Those are really, you know, we think of them as a user centred thing or something that is information provided to the end user would, of course they're terrible, they're complicated, but they're not really written for the average person. You know, they're written for lawyers for stuff that they can do in court, and they're not written for like everyday kinds of understanding at all. So that's another example.
Stephen
Well I would say one of the exciting things as we were doing our research for the book is particularly with legal and legal documents, privacy policy, terms of service was I think discovering the number of alternative ways to present the information that are out there from comic books to baseball cards, to just the simple annotations in the margins that you can turn off or on. There are a number of ways to make the information more easily understood without altering the core legal purpose that some of these documents serve.
Gerry
To change tack slightly here. Karl, what do we mean by, and the reason I ask this that obviously a considerable chunk of the book is dedicated to understanding. What do we mean by understanding and what perhaps are the pathways to understanding.
Karl
So understanding, of course, is a pretty complex word. It's like, you know, It's like what do we mean by information? I remember when I was in graduate school, I took a class and, you know, I think lecture number two, was that topic of what is information and kind of what the professor did was walk you through this idea of you think, you know, what information is, and by the time you come out of the class, you had no idea what information is and you realize that it's a super, super complicated word.
And so we have, and that is true of almost everything like you talked to a physicist, there's a couple of fundamental concepts at the heart of physics space, matter time, energy. And each of those, we have sort of an everyday, you know, understanding of what all of those words mean.
But you talk to a physicist like, those are actually really complicated words there. And the reason that you have a field is because you have those words and nobody truly knows, and you can argue about it, and there's a lot of complexities. So in terms of this book, in terms of the word, understanding, we approached it from that perspective of this is, there is an everyday meaning of the word, understanding. I have some information in front of me. I want to use it. I want to make sense of it. And if I can feel like I have done that, if it allows me, if I'm not confused or frustrated or overwhelmed, if I have able to use that information to accomplish a goal that I feel like I get it, I may not be perfectly accurate, but it's good enough.
I can move forward with my tasks. Then the information in that sense is sufficiently understandable. Now with the example that Stephen gives at the beginning of the book about the diabetes chart, that information was all there. All the information is present, but it was not in a really understandable form without a huge amount of effort. And even then you weren't completely confident that you would use that information without making an error that could have serious health consequences. And so what he tried to do was make the information more understandable to reduce the chance of making a mistake and something that everyone in the family, including his four year old son could make sense of. So we tend to use the word understanding and more of that everyday kind of sense.
Gerry
And you talk about understanding by association, by external representation and by interactions. I mean, without giving you the lion's share of the discussion here, can you run us very briefly through them or is that, is that too big an ask at this time of the day,
Karl
Stephen, do you want to start because you started with the stuff on associations and that was really where Steven did a bulk of the work here.
Stephen
Yeah, absolutely. So, and that is, that is the lion's share of the book right there, the associations, the external representations, the interactions, and then of course the coordination of all of those things. And then coordination of course opens a whole new world. You talk about coordinating a system of cognitive resources, which is the phrase we use in the book. On the associations that I think that was that one, I'll say that was a really hard part to write. I think Carl and I were aligned on the embodied cognition, distributed cognition orientation. But then when I started writing all this stuff about the brain as a perceptual organ, it was really counter to those bigger things that we opened the book with. And so I had to figure out how to talk about that. I think we are very careful not to say things like internal representations, which suggest a particular branch of cognitive science, but associations is how we talk about that section and the idea being very simply that what we think of as understanding or knowledge or beliefs, or what have you, are this web of associations that start building from when we're in the womb, right, when we're, you know, proprioception in the womb and moving around when we're born and reach out and what happens.
And, you know, we have millions, billions of these interactions over our lives that fire these neurons together that form memories, beliefs, predictions about the future, all these things. And so that was the core idea of the association section. And once you, once I grokked that and understood it, and I think it was a, it was a lecture by Douglas Hofstadter that, who, who said, you know, it's analogies or analogical thinking all the way down.
Once I got that, then everything else fell into place. So when we talk about something like a metaphor, and I know in the business world, an iceberg metaphor is quite common. And we talk about the idea of the stuff that seen above and the stuff that's unseen below, the culture and so on. And it's not the iceberg and the drawing of the iceberg, the visual representation, it's the associations that the iceberg suggest or connotes that there's this stuff below the surface that we've never seen.
So once I grokked that, the concept of an iceberg that, that made sense. So then you could talk about things like when I draw on the board and I draw a thicker line versus a thinner line that suggests something, and that suggests something that is consistent with the world, where a thicker a branch or tree trunk is going to be harder to break than say a thin twig or, or limb or something, something much more narrow. And we have these affordances, these patterns that we build up through our interactions with the world, from that early age. And so that's, that's kind of the association section, then the visual representation of the external representations part. And then finally the, the interactions which is really the part that Karl opened my eyes to and brought to my attention. It was one of those things like, yes, we interact with the world, but I've never thought about it intentionally or consciously until we started chatting back in when was it like 2013 or so>
Karl
Something like that.
Gerry
There's been a lot of work to about embodied consciousness in recent years, too. So not just, not just understanding and thinking, but you know, our whole consciousness… let's not go down that sidetrack anyway.
I want to, Karl, maybe this is one for you. A lot of our work today tends to be in the area of incremental change. But incremental change of course, can blind us to the need for well less incremental, but certainly radical or first principle change, can’t it, how can we, how can we avoid that trap?
Karl
Well, that, that question of incremental change should also be balanced with the idea that we have what Thomas Kuhn described as paradigm shifts. Right? So he wrote that famous book in the 1950s, The Structure of Scientific Revolutions and what he argued there basically is that you have a certain way in which we think, and we make a certain set of assumptions about the world and how the world is constructed. And we put pieces together, put pieces of knowledge together. We give them different weights and we operate within that particular framework. And, but then eventually what happens is we have a paradigm shift. We realize that while there may be some truth to that, that is an important way of looking at it, there is other things that we have discounted that we have overlooked. Another way it’s sometimes described in the scientific world is a hill climbing versus valley crossing.
You get very invested in a particular idea where you climb a hill and you're like, okay, the scientists and the researchers are like, okay, we're just going to do this study, which is going to get a little bit closer up that a little closer to the top of the hill. And we get a little closer and a little higher and a little higher, but then what happens when you get to the top of the hill or near the top of the hill, you look around and you realize, oh-oh, this is not the biggest, hill there's other hills. And so you have people in the scientific world, the knowledge building world who are going out, and they're like very invested in keep continuing up that hill. And that's what we've really seen in a lot of cognitive science, right? For the last 50 years, starting in the late 1950s and early sixties, we had this idea, which was a reaction initially to behaviourism and Skinner, BF Skinner, and that notion of, of conditioning and said, we really, the problem with behaviourism was that it viewed the brain as a total black box. There's nothing that we can know about what's happening inside the brain. And then the cognitives come along. And their reaction is like, no, we actually can figure something out about what's going on inside the brain. And so they overturn, right, and throw away the incremental ism that was happening in behaviourism, and they move it in new paradigm. And that new paradigm was let's focus on the mechanisms of mind by which they meant what is happening in the brain. And after 50 years of this, there came to be starting in, kind of depends on where you date this. But a lot of it would get dated to say the early 1990s through the 1990s, where people are saying, wait a second, there's a different hill out there. Let's go down from this hill, let's cross the valley. And then let's start finding and looking a different hill.
And that hill is now generally called embodiment, right? And so they're looking at something new. And we think that this is really important for user experience design, because what most people who work in design don't necessarily, I mean, we were all trained in certain ideas of the come out of visual design, graphic design, and especially human computer interaction, human factors. And all of that comes out of classic cognitive science where the mind and the brain are the same basic thing. And thinking happens in that as we talked about earlier. But when we, and that works fine when we're talking about keyboards and mice, and we assume that to use a computer is like to sit down at a desk, but that is increasingly not the world that we live in with our information, with our technologies, with our interactive tools and systems. We have watches and smartphones. We have tablets, we walk with them, we had stuff in our cars. We have stuff on our wrist. We're talking about, you know, we're going to have, we have VR headsets. We're going to have more and more ways in which the world of information and the world of UX design is going to play in this world where we are mobile, where our surrounding environment is part of it. And so we think that it's really important for user experience to recognize some of the deep seated assumptions, the framework we have and how that has led to sort of an incremental approach to the way we do design. We've been operating in a certain view of how understanding happens, how thinking happens, but the science is moving in a different direction, saying there's a whole lot more. And I would say that through the nineties and the two thousands, we could really safely ignore most of that, but we can see a future where our technologies are going to dramatically change. And in order for us to really take advantage of that as UX designers, we need to recognize the limitations of our own thinking.
Gerry
It's funny you describe you know, in the book and in what you're talking about there, you talk about you know, embodiment and a kind of a different way of understanding through interactions and through associations. And it conjures up this world of, you know collaborative design and co-design, and working on, on whiteboards and in groups and with representations and artifacts and various elements. But here we are, you know, in the relatively early stages of COVID-19, as we speak we're in July, 2020 you know, we're, we're all being pushed to work from home, we’re all online, more and more. We're less in the real world in many ways than we were before. Doesn't that militate against this move towards a future that embodies us more in our environment or connects us more with others.
Karl
Yes and no. I mean, I think that we have different kinds of tools, which change the social interaction that we have, and certainly it limits that. But just because that you're working from home, that does not mean that you are necessarily interacting with information just in this, in a very limited kind of way. So for example, I was recently doing some woodworking projects out in in our garage, and that is something that involves a lot of different space, how I arrange that space, how I use the different tools, how I am using the entire environment to solve the problem. It is not merely a case of looking at a sheet that has plans for the furniture that I'm going to design. Right? And then going, okay, now I need to pick up, the saw, pick up the saw. Now I need to go over there and I need to cut the board, right?
You do things where you say like create jigs where you're sawing and you're looking at it and you're adjusting it. Everything is very much a physical kind of experience with that. And this is true I think for a lot of the home offices that people are creating as well. I mean, Stephen, I know you had like whiteboards in your office. I've had like walls filled with post-it notes. Often when I'm on a call, I sometimes have, I have a very long cord on my headset. And so I often not just, if it's a long call, I get up and I walk around, why, why do I do those kinds of things? You know? And like, one of the reasons that scientists have been, have been discovering is we do those things not because they are some sort of weird, strange, personal quirk or idiosyncrasy, but there are ways in which moving our body actually physically like helps and improves our thinking.
An example of this is work that has been done around how and why people talk with their hands. So like, we're on a video call. I mean, the listeners to the podcast can't see us, but we can see each other. And one of the things that you've noticed is I have been gesturing with my hands, Gerry, like you have been gesturing with your hands at certain points. You're nodding your head as you're doing this. Like, why, why are we doing this? We could just sit here completely stock-still. If we only thought in our brains, why would we ever do this? So there's some people who've been doing this research going back 30 years and they’ve asked this question, why do people talk with our hands? And the answer is really quite fascinating. The basic answer is, well, we talk with our hands because it's a way of expressing extra information to people like the speed at which we talk or the volume at which we talk, right?
Speaking loudly or softly, or quickly, or slowly, pauses, intonation, all that communicates extra information. And the idea is that the hands communicate extra stuff as well. And that's, that's very much the case. The science definitely supports that. So then the question is, why do you talk with your hands when you're on the phone? The other person can't see your hands. And you know that you gesture with your hand, right, we all, we all do this. And so the answer that you could give is, well, I do that because it's a learned behaviour. I'm used to talking with people face to face. So when they're not there, I kind of carry it over. That's a good answer. So then another question then becomes, well, why do people who are blind talk with their hands? And not only that, why do two people who are both blind, who have been blind from birth, who know they are talking to someone else who has never seen a hand, why do they talk with their hands?
They've done studies of comparing people who are blind and sighted and how they do this. And they've given them reasoning tasks. So they've given a problem. And then they have to explain their reasoning about it. And not only do both groups use hand gestures, they use very similar hand gestures. And the conclusion of this and many other researches basically comes down to, there is a way in which our hands and our hand gestures and the way we use our body is directed outwards to other people. And we can see to get back to your original question, why that would be limiting in, in video calls, but there is an important dimension in which that those hand gestures are actually directed inward. They aid our thought. They help shape our thoughts. You can see this sometimes if you go to a conference and there's a panel discussion, somebody asks a question and maybe you've seen this over a video with the podcast. You ask a question and somebody pauses. They're not really sure. And then they'll start moving their hands. And that kind of works as like this cognitive grease. And the words begin to tumble out.
Gerry
Indeed. Stephen, it's nice to see somebody defend the humble pie chart [laughter], which has been much maligned by everybody, myself included in recent years.
Stephen
Yeah. Yeah. The example there in the book, where we're actually talking about visual encodings and the four functions that visual encodings have, and that's two of those being, is it to display precise, quantitative information, which is normally where you see the pie chart maligned, because obviously it's not all that great at that. But one of the other visual encodings is to display generic or general qualitative information. And that sense the pie chart’s very good. So in that example I think I cited a, a time tracking tool I use where they use a pie chart for each day of the month. And at a very quick glance, you can get a sense of, wow, I spent a lot of time on this pink wedge, whatever that was, and it's such a general qualitative information.
And I think in that context for that use case, it's a wonderful tool. Like I have trouble imagining a better way to give someone a sense of where their time has been going. Does it tell you exactly how much time you spent per day per week, per month on that? No, but you can then hover and get those additional details when you need them. So it's a nice balance of the qualitative information and the quantitative on demand as you need it.
Gerry
I mean, obviously you've put a lot of thought into visual design and visual thinking. And, you know, in recent years we've seen an explosion of excellent visual design stuff out there. I'm thinking of Tufte and Dave Gray and Alberto Cairo and people like that. How can we as individuals, how can listeners who are thinking, well, I didn't need to up my, up my skillset in the visual design and, and more critical about the visuals that I produce or that I consume. How can, how can we improve?
Stephen
Well, you can start by buying our book. [Laughter.]
Gerry
They don't have to buy the book, they've heard you talk now.
Stephen
I would say one of the, one of the things that you know, so you mentioned Alberta Cairo and David Gray and Sunni Brown and others, they've written these fabulous books that will give you specific ways to, specific tools that you can use to up your visual thinking. Dan Roam also with his back of the napkin stuff. So these are very, very helpful, very useful sources we can go to and whether online or their books or whatever, to up our ability to do visual thinking where our book is a little bit different as we wanted to go to a more fundamental level and say, okay that's good for the tactical, the practical, the pragmatic, let's dig a little bit deeper and understand why this works in the first place.So we can talk about you know, something that the business model canvas and the whole explosion of canvases that we've seen out there, we can talk about it as a good tool and how to use it, but what we do then to say, okay, well, let's talk about adjacent spaces and the role that adjacency plays in holding or conveying information and why you can't just move boxes around arbitrarily, because their placement means something. It says something about the information that says there's a flow from this thing here to the thing on the right or the thing on the left. And so that's what our book gets to is, let's talk about these more fundamental timeless ways of holding information that sit behind these tools. And so then as a follow on, then like I'm working on a workshop right now and I'm getting into the distinctions between what I would call visual primitives, canvases maps models, and these things, but it all sits on this fundamental understanding of how do we use space to hold, meaning how do we use visual encodings to hold or convey meaning?
And that's really where the book, what the book is a foundation for.
Gerry
The book is a, it really is fascinating and very broad ranging. I guess listeners would get an idea of that from the surfaces that we've just scratched in this brief conversation today. And we have run up against our allocated time.
I'll remind listeners that the book is called Figure It Out: Getting From Information to Understanding. And the code UXPOD will get you 20% discount on that or any Rosenfeld book on the Rosenfeld website. And I had some people commenting that the code had expired, but it's been renewed now I believe until 2030. So you've got a couple of years get out there and by that Stephen P. Anderson and Carl fast, thanks so much for joining me today on the User Experience podcast.
Stephen
It's a pleasure to be here.
Karl
Thanks for having us.