Published: April 2011
As new technologies pervade the car cockpit, Dr Michael Regan talks to Gerry Gaffney about safety, inattention, and visual and cognitive distraction. Mike discusses various types of safety technologies and the opportunities and challenges they present.
This is Gerry Gaffney with the User Experience podcast.
My guest today is research director at l’Institut français des sciences et technologies des transports, de l’aménagement et des réseaux [IFSTTAR]. Excuse my schoolboy French. That’s the French Institute of Science and Technology For Transport, Development and Networks. He’s adjunct professor for vehicle safety at Chalmers University of Technology in Goteborg, Sweden and a senior research fellow on secondment from the Monash University Accident and Research Centre here in Melbourne, Australia.
He’s an editor and co-author of Driver Distraction: Theory, Effects and Mitigation. He has a very distinguished academic and research career in human factors, ergonomics and transport technologies.
Dr Mike Regan, welcome to the User Experience podcast.
Thank you very much, Gerry. It’s a pleasure to be here.
Now, Mike, in the materials of yours that I’ve read, you frequently refer to the car cockpit. The word cockpit suggests a complex environment. Is it really descriptive of what today’s driver has to deal with?
Well, I suppose you’ve got to think about driving in totality. People when they think about driving often just think about avoiding accidents, but there are many different things we do when we drive.
If you as a human factors person were to conduct a function analysis, like a guy Ivan Brown did back in the 80s, you’d find that it involves many things; it involves route finding, it involves route following, in other words you know finding your way to where you want to go. It involves velocity control, it involves collision avoidance, it involves complying with the rules (rule compliance) and it involves some vehicle monitoring tasks such as looking at your speed dial and monitoring the fuel levels and looking at any other displays and controls that need to be monitored during the course of a drive.
So when you think about it, they’re just the high level functions and below those functions you’ve got a whole range of other subtasks that are supporting each of those activities and what’s sort of happening at the moment is that… in days gone by the car, you know when you think about the evolution of the car it really hasn’t changed a lot over the last eighty or ninety or even almost a hundred years since the first car was introduced, in the sense that the cockpit itself looks pretty much the same. There’s still a steering wheel and there’s still an accelerator pedal and you’ve still got some basic instrumentation, while at the same time other aspects of the car have changed considerably. There are new structures that are more forgiving in the event of a crash, much of the car now is in fact electronic or what we call drive-by-wire so there aren’t any mechanical linkages, for example, perhaps, between the steering wheel and the tyres.
So many aspects of the cockpit for a long time haven’t changed but things are starting to change, and I’ll start talking about those during this talk and those things are changing are really changing inside the car to support, in various ways, these different functional activities associated with driving.
Okay, and Mike just reminded me, and to deviate from the script without notice, you were talking about the fact that the car cockpit hasn’t changed all that much… I was reading about some, you know these cases where people inadvertently accelerate even though they believe that they are pressing on the brake and I know there have been some recalls suggesting that it might be in part mechanical failure, but there have been suggestions that in at least some accidents people accelerate when they think they’re braking and there have been moves to change the accelerator and brake pedal positions. Is that something that’s likely to happen, do you think?
It’s very interesting to talk about accelerator and braking systems and the design in the car cockpit of brakes and accelerators because, it’s sort of strange, it’s strange that the accelerator and the brake in manually controlled vehicles has remained the same for decades even though there are people who’ve developed, in Sweden for example, quite interesting brake and accelerator pedal designs which involve a pedal that’s more like a rudder pedal in an aircraft which moves forwards and backwards. And if you press on it in the way you, on an accelerator this pedal will accelerate the vehicle and if you push it forward in the way you’d push a rudder pedal forward in an aircraft the vehicle brakes. And so what you have is one pedal that serves two different functions, and to me that sort of makes sense because one of the problems when you’re trying to brake to avoid a crash is that you have time lags.
I mean, people take time to respond, to take their foot off the accelerator and to touch the brake if they’re trying to brake to avoid a collision with an obstacle ahead, and that movement time, from the accelerator to the brake, it can be in the order of milliseconds and it can be the difference between a crash and no crash. And so it’s very interesting that the design of the cockpit is a very, and has been a very conservative thing, as I said before, although other aspects of vehicle design have changed dramatically from a human machine interface design perspective, the interface itself hasn’t really changed a lot. But I think that’s going to change quite rapidly as the so-called intelligent transport systems work their way into the cockpit in ways that I’ll describe during our discussion this morning.
During a lecture in Sydney, I think December the 7th  at the University of New South Wales, you said there are three classes of technology that are beginning to dominate the cockpit; the first was entertainment systems, secondly, communication systems and thirdly intelligent transport systems, which you just mentioned. Perhaps we could focus on the latter two, putting the entertainment systems aside for the moment. Can you describe what’s happening in the realm of in-car communication?
Well, I suppose if you look at the history of communication systems the earliest communication systems in vehicles were probably CB radios for emergency vehicles and for use by truck drivers and these radios are still in existence and still used. And then following that mobile phones sort of came onto the scene, and mobile phones of course enable people to communicate in different ways. Wee can talk through them, we can send text messages through them, we can video text, we can send internet messages through them, we can even send emails via the mobile phone.
As well as these what we call nomadic devices coming into cars, car manufacturers themselves have introduced mobile phones, fax machines and internet facilities into cars as dedicated features in vehicles.
And so this is sort of thing is changing the cockpit because you’ve now got different kinds of interfaces that have been developed to support the introduction of these communication systems into the vehicles in the safest way possible. And to their credit, the vehicle manufacturers have been good citizens in terms of trying to use the best human factors, knowledge and advice that they can in designing the human machine interfaces to support the introduction of these kinds of systems.
And then on top of these communication systems you’ve then got this most important class of systems coming into vehicles… We call these intelligent transport systems and again, they’re coming into the cockpit either directly through the vehicle manufacturers, as what they call OEM products, original manufacturer products that are built in to the vehicles, or some of them in fact are increasingly becoming available on these nomadic devices like mobile phones and other mobile electronic devices that can be brought into the car. And intelligent transport systems are really just a collection of electronic telecommunications and computing technologies that can be combined in different ways to increase driver safety and driver ability and driver comfort and pleasure. So they have different functional purposes, these intelligent transport systems, but as a person who works in road safety I’m really more interested in the ones that can increase safety… From a safety point of view, they can help to prevent crashes, they reduce the severity of crashes and hence the amount of injury in the event of a crash and they can also reduce the amount of trauma post crash. I could give you an example of some of these systems, if you like, and explain what I mean.
Some of the systems interestingly keep the driver in the loop and some systems are capable of automating drivers completely out of the loop. So what’s interesting about these systems, unlike the other ones, is that these are really designed specifically to support the driver, to perform each of these functions that we talked about before: the route finding, the route following, the velocity control, the collision avoidance, to some extent rule compliance and also vehicle monitoring. And so if we’re looking at the category of intelligence transport systems that are designed to help prevent crashes, one of the ones that many people around the world are excited about, including myself, is what we call intelligent speed adaptation.
So this is a clever system in which, which is available by the way in Australia, I think on the iPhone as an app and also on some other smart phones, but also has been built into the vehicle on some vehicles here and in Europe, and this is a system that will basically warn you if you drive over the speed limit, and if it’s a little more clever than that it will actually restrict your top speed to the speed limit, and even cleverer systems will take into account wet weather conditions and perhaps icy roads to reduce your speed limit or give you a warning at a slightly lower speed than the speed limit itself.
So what this system’s doing mostly is taking that function we talked about of vehicle monitoring, in this case the monitoring of speed, and the way it works is interesting too because the car has a GPS system so it knows where it is on the surface of the earth, it has a digital map of all of the roads that the driver’s driving on, and within that general map are the speed limits on the segment of the road, and so together with a little computer the car basically knows where it is, what the speed limit is and how fast the car’s going, and of course the computer calculates whether the car’s exceeding the speed limit and the driver gets a warning. And so it’s a very interesting system… What’s interesting though from a design point of view is that this system has been shown again and again to reduce speed, average speed and peak speeds. I ran a study in Australia while I was with the Monash University Accident and Research Centre to investigate the effects of these systems and we found that they could bring down average speed by three or four kilometres per hour, which doesn’t seem much but it translates into a huge saving in lives, because we know that as speed increases, the ability of the driver to process information is reduced because they have less time to scan for important things in the environment, and if they have to brake it’ll take longer to stop and the energy involved in the crash will be greater.
So speed is a major issue for different reasons in road safety. But interestingly not a lot of work, hardly anything has gone into the design of these systems. And so if designers could come up with a better way of optimising the design of the human machine interface they might be able to bring down speed by even one or two more kilometres per hour and so you could demonstrate that the HMI [Human Machine Interface] design in fact had a huge impact on the likelihood of a crash or severity of injury. So this is one example of a system in which, interestingly, the HMI is not being given a lot of thought.
Other systems that fall under this general category of preventing crashes include forward collision warnings. So here’s a system in the car where you’ve got a radar or a laser sort of sitting in the bonnet of the vehicle looking out for vehicles in front and if the system detects that the car in front is braking very rapidly and that you haven’t responded at that moment in time accordingly then you get a warning and if you ignore the warning, perhaps because you’re distracted, or inattentive for some other reason, the car will actually break for you to avoid the crash. So here you’ve got a system that’s supporting the driver, in this case to support them for this function of collision avoidance, to both warn them but also to avoid the crash for them. So it’s automating the requirement to brake and it will brake much more heavily than people typically will want to brake. Most people are reluctant to brake I think more than 0.3 g, negative 0.3 g so some of these systems will brake you at up to 1 g if you don’t respond. And that rate of braking is something that we can’t sort of bring ourselves to do readily for whatever reason.
Other systems include things like blind spot information systems. Quite a number of crashes are due to the fact that we drive along but we’re not situationally aware of what’s in our blind spot so we, you know, we change lanes and next minute we know we’re hitting the car in the blind spot. And so these systems have radar around the vehicle to detect whether or not there are cars in the blind spot and so if you change lanes and a car’s detected, you’ll get a warning. And again it’s very interesting to know how to design the human machine interface for these systems, so where and how do you convey to the driver the fact that there’s someone in your blind spot? You know, do you put the information or the warning information in the rear view mirror to provide the information? In other words do you put them in places where people normally seek out the information or do you put it somewhere else? Do you have a total rethink about what situational awareness means in terms of blind spots and ignore the way we’ve tried to seek out this information in the past and put it somewhere else? From a design point of view these are the interesting issues to think about.
There are systems now that have been developed that rely on what we call “cooperative driving” and these are categories of intelligent transport systems that we call cooperative intelligent transport systems… You can imagine now that you’ve got two cars approaching an intersection but one car, for whatever reason, can’t see the other car ,and so we now have real time communication between the cars. We’ve got radars and electronic communication links between the cars that will warn one car of the impending collision with another car so that at least one of the drivers will do something to avoid a crash. So this is now a very interesting development because so far the systems that have been developed in vehicles that I’ve been talking about are what we call autonomous systems that are sitting in the vehicle, operating sort of in isolation, it’s just the driver and the system interacting with each other. But now what’s happening is that we’ve got new systems that are being developed to solve problems like this of intersection collisions that really rely on communication between cars, between drivers in fact, and so that requires a new way of thinking about how we convey information and through what modality to drivers when vehicles are now starting to communicate with each other.
Other systems, just to finish off, that are important for crash prevention are things like lane departure warnings so these are systems in vehicles where you’ve got cameras at the front of the vehicle that are looking at the road markings and if they determine [from] the road markings that a car is veering off a road or starting to veer off the road and that it’s likely that there will be a major lane excursion, then they’ll give the driver a warning. And this could happen, for example, if the person is fatigued, if they’re starting to fall asleep or if they’re distracted.
I didn’t want to cut you off there but there’s so many implications for all the things that you talk about. It’s fascinating to think about, for example, the way drivers may adapt their behaviour to increase the risk by offloading some of it onto the systems, for example, if they did have that blind spot avoidance and out of lane marking and so on.
Yes, many different issues arose by these systems. And then another one that has been around for a while is the driver drowsiness warning system, and so from a human factor perspective this is also a very interesting system. It’s using cameras to look at a number of behaviours, in particular eye closure rates and patterns of eye fixation to give the system some idea of a) whether the person is starting to fall asleep and b) when they’re likely to fall asleep. And they look at other behaviours as well that are typical of drivers who are tired, like scratching themselves, tilting their heads forward and having less variation in steering movements and things like that, and then they correlate all this information that, from a human factors perspective the interesting thing is what do you do with all that information? How do you convey to the driver that they’re falling asleep and can you do that in an effective way? And if you do that what’s the driver meant to do with that information? Will they actually pull over at the side of the road and take a break or will they use the system to keep them awake, which is not really how the designer of the system intended the system to be used? So there are many interesting human factor issues that come into play.
Just one other interesting one that’s been developed is what they call a driver distraction prevention and warning system and as you know I’ve done a lot of work on driver distraction and inattention and I’ve written one book and I’m just about to write, I’m writing another book on the topic. And these are quite interesting because these systems are again using cameras predominantly to focus on your patterns of eye movement to determine whether or not you’re what we call visually distracted, in other words you’ve got your eyes off the road, or whether you’re cognitively distracted which means you’re still looking out at the road but you’ve got things going on inside your head, you’re sort of thinking about things and as a consequence of the internal thought may not be attentionally aware of things that are actually happening in front of you. And this notion of cognitive distraction is quite an insidious thing because there’s plenty of evidence to show that when people are cognitively distracted they can be, you know for example, seeing things such as a pedestrian that’s about to walk out in front of a parked car that they’ve only just seen, and so they’re physically looking at them but they’re not attending to them so they either do nothing or they brake too late and they hit the pedestrian.
And so this is the danger of people talking on mobile phones, for example, when they drive because their attention’s actually not on the forward roadway, even though their eyes are on the forward roadway, it’s actually, it’s on the conversation and people don’t realise how important it is to have attention focused through the sensory modality that you need at the time, in this case vision and the eyes in order to avoid the pedestrian. So this gives you a bit of a flavour for… so these systems will warn you basically if you’re distracted through either mechanism and there’s evidence that this particular system’s quite good in calibrating drivers to spend less time looking inside their cars and spending more time looking outside the car.
To shift very slightly from where we’re at, you made some interesting remarks talking about the mismatch between drivers’ mental models and the technology in their cars. In some of the materials that I’ve read that you’ve produced, I think for the Sydney lecture for example, and the example – I’ve done this myself – was speeding downhill because of an incorrect assumption that cruise control will activate the car’s braking mechanism. How can system designers though ensure or even attempt to ensure that drivers’ mental models match the real world device or application that they’re using?
Yeah, this is a tricky area and there are people, I think more expert than I am in this particular area of what they call mode awareness and automation and that sort of thing, especially people who have been working in the aviation industry because they have a lot of crashes in aviation because of people losing awareness of what mode they’re in, whether it’s manual or automated. But I think in terms of driving aids there are probably two fundamental ways in which you can try and support the, or induce the mismatch, and firstly I think drivers need to be aware through education and training of the limitations of the systems because I think if you’re aware of the limitations then you will be more aware of the kinds of problems that you could get yourself into. So, for example, if you were to do a survey of these systems that are pretty common in vehicles now that they call backing aids, where there are ultrasonic sensors at the back of the car and you’re reversing into a car park or doing a parallel park and you get close to the vehicle behind you and you start getting these beep, beep, beep, beeps coming on.
There are a lot of people out there who think that these systems are capable when they back out of their driveway of detecting pedestrians that might be walking behind the car but in fact most of them won’t do that, and so it’s a slightly different issue to the one we’re talking about but it’s all to do with situational awareness of what’s going on around them and understanding the interplay between the system and what is going on around them and in this case if they had the false belief that this system will detect moving pedestrians, they might just simply back out without looking around and wait for the beep and end up hitting a pedestrian.
So one way is making drivers aware of the limitations of systems and then the other way really is to design the HMI to make it quite clear to the driver which mode the system is in. With things like cruise control, I suppose, just conventional cruise control, there are a number of different feedback mechanisms that the driver has to remind them that they are in cruise mode and for other systems similarly there are warning lights and other sources of information needs to remind the driver that they’re in a particular mode. But how effective they are in doing that I’m not sure. I’ve not actually seen a single study that’s looked at mode awareness in relation to these kinds of systems.
Certainly I’ve read one study that’s shown that when something like adaptive cruise control, which is a more advanced form of cruise control in which you set the speed at which you want to travel at, let’s say it’s 100 kilometres an hour, and if you come to a car that’s travelling more slowly than you, this system will actually slow your car down and then adopt a constant following distance behind the car in front. That’s what we call adaptive cruise control and I know there was at least one study conducted in the UK that showed that when that system unexpectedly failed, this was in a simulator study, that I think about 35 per cent of the people ended up crashing into the vehicle ahead because they’d become very complacent using this system, they were totally reliant on the automation and when it failed they weren’t aware of the fact there was a change in status because the system… all of a sudden went back to manual control. So there are sort of two issues; being aware of the state of automation you’re in, but then being aware of the changing status of automation when something like a failure occurs, and these are issues we don’t know a lot about.
Mike, tell us about task difficulty homeostasis and the implications for in-car systems design.
Well, task difficulty homeostasis is a term that was coined I think by Ray Fuller in the UK and simply what this means for example in relation to driving, and I have a particular interest in this phenomenon because I work in the distraction area, but an example of that in the driving domain would be that you’re sort of driving on a road that you’re very familiar with, so navigation, route finding is really not part of the job, you’ve travelled through this area many times so you’re familiar with the layout of the road, and so the workload’s pretty low. So in response to the reduction in workload during this particular segment of your drive you do something else to maintain some sort of level of equilibrium from a task difficulty point of view while you’re driving. So what you might do is pick up the mobile phone and call a friend or something like that, and so essentially what he’s saying is that we have a sort of target level of difficulty when we’re driving that we try to maintain for whatever reason to, I don’t know, gain arousal or to remain satisfied, or for whatever reason, but the mechanism is that we have this motivation to maintain some target level of difficulty.
And so a guy called Peter Hancock in the US, who a lot of your listeners would know because he’s a very significant figure in the human factors community wrote a chapter in this book on distraction I’ve talked about before where he referred to this phenomenon. In fact he called driving a satisficing task, not satisfying but satisficing, so the point he’s making is that, as I made before, that people in conditions of monotony in a car automatically are going to want to keep themselves stimulated, to make life a little more difficult for themselves, so there’s going to be a natural tendency for them to want to distract themselves under such conditions because distraction is one mechanism by which we can increase arousal and increase workload. And so he thinks it’s almost inevitable that as cars become more automated, using the sorts of technologies that we’ve talked about before, that this phenomenon is going to happen. What this could lead to is more and more people engaging in these distracting activities and so again it’s very interesting from a human factors point of view because on the one hand we’ve got all these fantastic systems coming in that are going to save lives and we think we can be fairly categorical in saying that because at least theoretically they target the very breakdowns in performance that we know lead to crashes and support drivers from breaking down in ways we know they do. But on the other hand the safety benefits that they provide and the degree of automation that they would bring about could result in engaging in other behaviours that might undermine the benefits of the systems, and we really don’t know much about that at all because we’re only at the very early stage in the evolution of these systems. So for human factors people like me it’s a very exciting area to be working in.
Indeed, and talking about crashes, you’ve got in your Sydney lecture the figure of 1.2 million people killed in crashes every year around the world. Do you think it’s appropriate to have humans in control of cars at all or should we just devolve all the command and control activities to automated systems?
Well, that’s an interesting point. I used to work in aviation safety and human factors and used to design aircraft cockpit interfaces in fact for a few years, and I know that in commercial aviation now even a system as sophisticated as a Boeing 747 can pretty much take itself off and land itself with all of the route guidance and other systems and a route finding guidance systems that is has on board But when you talk to pilots in that domain and human factors people they will always say that you need the pilot there in case something unexpected happens because you know the designer of the 747 can never anticipate all of the situations in which that aircraft will have to… that aircraft will encounter and will have to react to unexpectedly. And so the human or the pilot in this case then becomes I suppose a part of the risk management system for operation of that aircraft in case anything unexpected happens, because humans as we know are quite creative at handling unexpected events, just like that pilot that who landed on the Hudson River recently with no engines. So sometimes you really need humans there in that case.
In cars the situation’s a little bit different. Certainly cars have been developed that are capable of completely driving themselves, so there’s no doubt such cars have been developed, but they’ve not been developed to be able to predict and anticipate in the way humans do all the things that could potentially go wrong and hence to have the repertoire of responses as we have as humans in being able to react to those unexpected events. So at this stage in the evolution of cars we’re pretty much in the same situation as aircraft. You know, driving a car in many ways is more complicated than flying a plane in terms of the amount of information that’s streaming through to the driver, mainly through the eyes, continuously, and unexpected things that happen, because unexpected things happen all the time on the road because it’s a less regulated environment than up in the air. Automating vehicles to drive on roads and particularly in a place here like France where you tend to drive, because I’m living in France at the moment, and you tend to drive here to expect the unexpected rather than in Australia driving to sort of expect the expected to happen, and then when people violate your expectations in Australia it’s almost as if you’re unprepared for the violation and that can lead to a crash. It’s strange. Here I think you’d expect there to be more crashes but I think people drive very defensively to expect the unexpected. And so there’s a different way they drive… so it may be that designing, I haven’t thought about this before, but it maybe that in designing these support systems you have to design them to support the sort of different driving styles that people have in different countries. But can we get rid of, you know the question is do we still have to have humans in control of cars at all, I think we do at the moment but I don’t think they have to be controlling everything. As I’ve said, one very important system that could be monitoring speed and helping you control speed in fact is intelligent speed adaptation.
I think there’s no doubt that the collision warning systems I’ve talked about are excellent systems and they can be taking over the role of humans having to scan for hazards and avoid them. But the problem is at this stage in the evolution of vehicles where you’ve got maybe some people coming to work and driving these high-tech cars and then going home and driving ordinary cars, you’ve got to have some maintenance of skill so that when you hop into the manual vehicle you can still do the job you have to do in avoiding crashes and saving your life and the lives of others. So it’s very complicated at the moment during this transition from fully manual to fully automated vehicles.
Now Mike I believe you’re based in Lyon, is that right?
Yes, I was seconded by Monash University to come here for three years but then it became four years and it’s going to be a little longer than that now even, and the institute I work at has a laboratory that specialises in research on these advanced vehicle technologies from a human factors perspective. So I work mainly at the level of the European Union on a few big projects that are related to exactly the sorts of issues that we’ve been talking about today.
Now Lyon is noted for transforming itself from a fairly bicycle-hostile city a few years ago to a fairly bicycle friendly one, do you hop on the bike yourself? Do you have any thoughts about that modality in terms of its relationship, I guess, to the research you do and the work you’re involved in?
Well, that’s a really interesting comment. They call them velos here and it was one of the first places in the world, I think, where you can hop on a bike and ride it for thirty minutes and then take it to another docking station and leave it there and the ride’s free. I think what sort of bothers me here is that no-one wears or has to wear a bicycle helmet even though we know that they’re quite effective. But then I think the, you know the other thing about that is if people had to wear helmets all the time they might not bother hopping on a bike because they couldn’t be bothered with the helmet and so I suppose when you look at these things from a societal perspective it might be that in the end more lives are saved because they have less heart attacks because they ride bikes than if they wear helmets and fewer of them ride the bikes but more of them have accidents. It’s hard to get the balance right if you’re a policy maker, I think, in this area.
Now from the technology angle I haven’t really thought too much about this issue. What I do know is that Volvo just, I think late last year, mid last year I think it’s the Volvo S-70 or S-60 came equipped with a pedestrian detection avoidance system so the first vehicle in the world I know of which will detect if you’re about to have a collision with a pedestrian, and it might include cyclists, and brake the car at 1 g to avoid a collision. I think in terms of equipping bicycles with technology at the moment we haven’t really come at it from that perspective. We’ve thought more about equipping the car to predict the presence and the whereabouts of vulnerable road users like pedestrians and bicyclists but probably the next stage of development will be to equip things like bicycles with technologies that will give them more situational awareness of the whereabouts of vehicles and other objects out there on the roads that they could conflict with and which could cause them grief. I think that’s a very interesting development and perspective that hasn’t yet developed as far as I know.
Well Mike obviously you work in a very fascinating area and there’s lots and lots of things that we could talk about and I guess one of the things that surprised me, both in reading your materials and in speaking with you is how much research remains to be done in an industry that you would imagine is relatively mature from the point of view of the car cockpit but as you point out much of the technology is very, very new. Mike Regan thanks so much for joining me today on the User Experience podcast.
That was an absolute pleasure, Gerry, and thank you for your time.
Published: April 2011