Nobody wants to be Average! (Base Rate Neglect)

Tessa: And I’m Tessa. Welcome to season three. In this, our third season of How to Choose, we’re exploring the topic of thinking problems and biases.

Ken: That’s right. In this season, we’re focusing on several common thinking flaws that undermine our ability to judge well and make good decisions. We’ll tell you some ways to identify if you’re succumbing to these errors and how you can reduce the impact of these biases.

Tessa: What’s our topic today, Ken?

Ken: Well, today we’re looking at base rate neglect. But before we talk about what that is, I asked several of our listeners the following question, and you can have a think about it, too. “Tom is introverted, quiet, tidy, and very organized. Is Tom more likely to be a librarian or a tradesman?”

Brian: I would say a librarian. Us tradesmen tend to be more ADHD, more cluster brained.

Dana: More likely to be, in the stereotypical way, a librarian because of his character traits, but certainly they could be very fitting to a tradesman.

Peter: My gut instinct is to say a librarian.

Lucy: I would probably argue he would be more likely to be a tradesman. But that’s coming from someone who, probably… I think I’m a bit of an introvert myself, and my first thought was I kind of pictured myself as a librarian, and then I pictured myself as a tradesman, and I think a tradesman would be a more peaceful life than a librarian.

Erin: I think I’m going to have to claim not enough evidence there. However, based on the description of personality, I would imagine he might be more attracted to a career as a librarian rather than a tradesman. I mean, the tradesmen I know tend to be loud and messy, but I probably know more tradesmen than male librarians. At any rate, I know quite a few female librarians, but I think I’d need to know a little bit more about Tom before I could make a call.

Becca: A tradesman. Every librarian that I’ve ever known (and I’ve known, I want to say, more librarians than the average kind of person) based on no empirical research, just my anecdotal evidence: there’s a chaos element in every single one of them. And so they might be kind of tidy, and they might be organized, but they’re not quiet. Or they might be quiet and they’re organized, but they’re not tidy. They know where their things are, but there doesn’t seem to be any obvious sort of system.

Nick: Librarian.

Isha: I’m trying to think I think my best friend, who’s very extroverted, works at the library, and she loves it. I would still say librarian just because I feel like a trades person would probably need to do a lot more communication. That’s not regular. It would be a lot more maybe negotiation, a lot more bargaining, a lot more of that sort of impromptu conversation so that would be my guess.

Duncan: Say the description again of Tom.

Ken: So he’s introverted, quiet, tidy.

Duncan: Very organized, quiet. Yeah, probably a librarian.

Tessa: Wow. Ken so most people went for librarian, but such different justifications for the reasoning for giving that answer. There was definitely a bit of availability bias on display here, and some people seemed like they were starting to consider the base rate, realizing that they knew a lot more tradespeople than they did male librarians. So do we have a right answer?

Ken: Not necessarily, given that it’s a hypothetical. But let me start with this. I’m going to ask you a different question, Tess. It sounds quite similar to the one that I asked our listeners. ‘I met a guy named Tom today. Is Tom more likely to be a librarian or a tradesman?’ Now, what goes through your head when you hear that question?

Tessa: I guess, first, other Toms that I’ve met before, the fact that most librarians I’ve met are women and most tradespeople are men. All the books, movies and TV show depictions I’ve seen of them along those gender lines, I’ve never seen or heard of Tom the librarian.

Ken: Well, that’s an interesting point. I should say that there certainly are male librarians. I’m sure many of them are listening right now, and I’m pretty confident that at least one of them would probably be named Tom. But it is interesting that your reasoning was quite different to that of our listeners when I asked them a similar but slightly different question. Our thought processes are different because we’re given no information apart from gender. We’re more likely to think how many men are tradesmen and how many are librarians. Now, most of us don’t know the proportion of tradesmen and librarians in our population, but we might at least try to guess what that is. However, what happens when I give you a description of Tom with some attributes?

Tessa: Well, it makes me focus on the description of him to see what kind of profession matches that description. Like trying to match a pattern?

Ken: Yeah, absolutely. We’re focused on using the pattern to create a bit of a mental story about Tom, perhaps filling in some blanks and matching that to our mental picture of a librarian or a tradesman. We tend to ignore the question of prevalence.

Tessa: So it isn’t stupid to guess that Tom is a librarian?

Ken: No, not at all, because we can see characteristics in Tom that match a pattern that we’ve observed in librarians. But the issue is that we’re forgetting a key point, and that is there are far more tradesmen than librarians in Australia. At least, there are 1.6 million out of a population of 25.7 million people who would identify as tradespeople. And now that’s about 6% of the population. Now, to compare some slightly older data suggests that there were around 25,000 librarians or people in related roles, such as archivists in Australia. Now, that’s about 0.1%. So what that means is, then, for every librarian, there are 60 tradesmen. So regardless of Tom’s interest in books, the fact that there are a vastly larger number of tradesmen than librarians should make us think Tom is more likely to be a tradesman.

Tessa: But there are other factors to consider, of course. What percentage of librarians are male compared to the percentage of tradespeople? Does that impact likelihood? What if we found out that Tom was confined to a wheelchair? Might this impact the base rate as well?

Ken: Yes, you’re absolutely right. But the point is more that we can make a quick judgment based on a couple of characteristics such as gender and personality, and then completely forget about the broader issue of base rate or prevalence of tradesmen and librarians in society.

Tessa: Now, we’re moving to the realm of probability and mathematics here, aren’t we?

Ken: Yes, in broad terms. But be reassured, you and I don’t have to know how to calculate the base rate probability in order to be aware that it’s a factor worth thinking about. So whenever we’re making judgments or decisions that rely on the likelihood of something happening or the likelihood that something is true, then we should pause and consider base rate. But having said that, can I bring in a little maths?

Tessa: Look, Ken, I’m always keen. I haven’t done it since high school, but I’ve always loved math.

Ken: Yeah, and look, I’ll put my hand up here. I have a science degree and I have taught junior maths at high school, but I am in no way, shape or form a mathematician either. So with those disclaimers, let’s proceed. Now, when people are calculating base rate, we can obviously have a simple figure like the number of registered tradespeople. It’s a simple matter of finding that figure that someone else has tallied for us during a national census. That’s pretty straightforward. But base rate can also be a bit more complicated. It can involve the calculation of the number of what are called false positives and false negatives. Now, I won’t linger on this too much, but we have provided a link in our show notes to a nice little website called mathsisfun.com that explains it very well. Now, if the name of the website, Maths is Fun causes you to roll your eyes or provokes nasty flashbacks to high school, don’t be alarmed. I will try to give you a simple example. All right. You ready, Tess?

Tessa: I’m ready.

Ken: So let’s imagine someone in your household recently acquired a goldfish. You’ve started getting a rash on your arms, and your first thought is to say, this can’t be a coincidence, it must be the fish! However, you then remember that it is quite common that people notice two things that have happened at the same time and then to assume that one has caused the other. So the technical term is to confuse correlation where two things have occurred at the same time with causation, where one has caused the other. I think we’ve talked about that in a previous episode. If not, we’ll aim to talk about it more another time.

Tessa: Look, we have Ken. We’ll put a link in our show notes but it’s a really important idea. So worth diving into causation and correlation.

Ken: But let’s say for the moment, Tess, that you are very aware of that trap of causation and correlation and instead you want to investigate properly to see if the rash might be caused by the fish. Now when you talk to your doctor you find out that in fact there are tests that you can take that are 80% accurate. Well, what does 80% accurate mean? Well, in the ‘case of the mysterious arm rash and the goldfish’, there are four possible outcomes from the test that you’ve bought from your pharmacy which can be nicely illustrated in a simple two-by-two matrix. And given that this is a podcast, you have to visualize that matrix in your head, I’m afraid.

But if a person is allergic to goldfish, the test will either show that the individual is allergic so that’s a correct result, or it could show that they’re not allergic which would be an incorrect result and that is known as a false negative. So we could say that maybe 10% of the time there are false negatives when you take this test.

Now if a person is not allergic to the goldfish the test is either going to show that they’re not allergic, right, that’s the correct result, but it could also show that they are allergic and that is another incorrect result which is known as a false positive. And we could say that in this case that 10% of the time you will get a false positive result to the test.

Now here is where false positives and false negatives create a lot of confusion because on their own they won’t tell you whether or not you have got an allergy to goldfish. There is another really important figure to consider and that figure is the base rate of goldfish allergies in the broader population. So simply put, if goldfish allergies are extremely rare then it is extremely unlikely that you are allergic to goldfish. It is more likely that there has been a false positive from the test that is suggesting you are in fact allergic. So the likelihood that you are allergic there is affected by the base rate as well as the accuracy of the test. Now at this point I will stop the maths lesson and I will point you to Maths is Fun. And as I say this, I can only think of a former guest on our show, Doug Braiden who felt the calling to mathematics when he left the police force. So go back to season two if you want to have a listen to that one, but maybe try Maths is Fun and you can ponder this a bit more.

Tessa: And again, it’s fine if you find the maths confusing, but always pause and consider base rates when you’re considering the likelihood that something is true or the likelihood of an event happening. I think there’s another related thinking flaw in all of this. We tend to see ourselves as exceptional, don’t we? We might see others becoming very ill because they neglect their health, but think, that would never happen to me. Or we see other people making really bad relationship decisions and think, I would never do that.

Ken: Yeah, that’s an important point. At least in Australia, we have a strong cultural reaction against the word average. Nobody wants to be average. We all want to be above average. But statistically that’s where most of us are going to sit on the bell curve. Of course, we will be better than average at some things, and we’ll be below average in other things, but in many areas we’re average. I might have quoted this line before from the American radio show The Prairie Home Companion, but at the start of every show, the narrator would introduce the fictional town of Lake Wobegon by saying, where all the women are strong, all the men are good looking, and all the children are above average. And it’s a humorous quote that pokes fun at our sense of exceptionalism and our poor understanding of statistics. All right, well, listen, apart from the unlikely scenario of skin rashes and goldfish both arriving in the same week, where else might we bump into base rate neglect?

Tessa: Yes well, it’s really just any situation when you’re evaluating the likelihood or chance of something being true or something happening. Making a judgment on the likelihood of Tom being a librarian is one. What about some of the assumptions people make around gambling? Sports gambling is an extremely high profile form of gambling at the moment in Australia, unfortunately.

Ken: Yeah, and look, it’s a terrible blight on society, in my opinion.

Tessa: Yeah, I completely agree. And then they often seem to offer you great odds at winning. You only have to guess, say, the first point scorer in a football game, and you can win. And the odds on offer seem really, really good for the players who score lots of points. But does anyone sit back and look at the statistics to see how often that particular player scored the first points in a match in the last couple of seasons? That figure would be the base rate and could then be compared with the odds the betting agency is offering. We don’t, and that data isn’t readily available anyway. But if we did, we’d find that the odds are always stacked in the house’s favor because that’s the only way these companies make so much money. And the fact that they can afford so much prime time advertising suggests that they are doing pretty well for themselves.

Ken: Yeah, that’s right. And there often seems to be a large element of superstition around gambling, too, doesn’t there? I mean, some people will get someone else to buy them a ticket because they think the other person is lucky, or they’ll go to a news agent that’s recently sold a winning ticket and buy it from them. And we’ll often hear people joke and say to someone who seems to have had some good fortune during the day: ‘Well, you should go and buy yourself a lottery ticket’. Well, what’s the actual probability that you will win Powerball with your ticket? Well, the odds vary depending on who you ask, but it’s something like one in 140,000,000. Now, by my calculations, and I would encourage you to check this if you buy a ticket each week, then you should expect to win Powerball once every 225,000 years.

Tessa: And that’s why you don’t neglect base rate when you’re making judgments and decisions about the probability of something happening!! And look, Ken, I never buy lottery tickets, so that math is extremely convincing to me, at least.

Ken: Yeah, me, too. And listen, you can apply the same scrutiny of base rates when you’re considering investing time in any new project. In The Scout Mindset, Julia Galef tells the story of how she had aspirations to be an actor. One friend had encouraged her to ignore the odds. Everything’s a risk, but if you want it, you should go for it. If you worry about failing, well, that’s just going to be a self fulfilling prophecy. But Galef did look into it and found out that the median annual salary of actors who were members of Actors Equity was only $7,500. And less than 35% of the members of Actors Equity got any acting work at all in a given year. So she spoke to another friend who was in show business, and that friend gave her more grounded advice. Basically, she said, ‘It’s really tough. That doesn’t mean that you shouldn’t do it. But consider, is acting the only career that you could be excited about?’

All right, well, what’s your key takeaway from this episode, Tess?

Tessa: For me, it’s about when you’ve got a really important decision, it’s worth taking the time to establish the base rate. We really can’t make an informed decision without it. And even if it’s something that will take a little bit of research, it’s worth doing.

Ken: For me, it’s also being careful about forming judgments purely based on gut instinct or even pattern matching. It’s pretty hard to argue with cold, hard statistics, isn’t it?

Well, thanks so much for joining us for another episode of season three of How to Choose. Make sure you jump in and listen to our next episode where we will talk about optimism bias, which is something that I suspect both of us might suffer from a little bit Tess being natural optimists.

Tessa: I think you’ll be right, Ken. And don’t forget to follow the show and tell your friends about us, too. Bye for now.