Have you ever had a period in your athletic or professional career where you kind of felt like you were on fire? Maybe you made a whole streak of consecutive shots in a game, or executed one good idea after another at work.
In his book, The Hot Hand: The Mystery and Science of Streaks, my guest today explores why success sometimes seems to arrive in clusters like this. His name is Ben Cohen and he’s a sports writer for The Wall Street Journal. Ben and I begin our conversation with an explanation of what it means to have a hot hand, and how this phenomenon has often been studied in basketball, but can be seen in a wide range of areas, including the film career of Rob Reiner. We then discuss what may cause winning streaks, whether or not they can be induced, and what Stephen Curry does when he starts feeling hot in a game. We also talk about what the video game NBA Jam can tell us about the psychology of the hot hand. We then dig into what the academic research has found on whether the hot hand truly exists or is really just a cognitive illusion. We end our conversation with what you can start doing today to take advantage of having a hot hand.
If reading this in an email, click the title of the post to listen to the show.
Show Highlights
- How do researchers define he “hot hand”?
- What the film career of Rob Reiner can teach us about the hot hand
- Why does success sometimes come in clusters?
- How Steph Curry thinks about hot streaks
- NBA Jam and hot streaks
- Why some scientists don’t believe the hot hand really exists
- The human tendency to look for and find patterns
- What does the hot hand look like in a trade like farming?
- What is the gambler’s fallacy?
- The undergrads who may have proven the veracity of the hot hand
- What can a person do today to take advantage of the hot hand?
Resources/People/Articles Mentioned in Podcast
- My interview with Steven Kotler about flow
- How to Hack Your Flow
- Rob Reiner
- The Princess Bride
- Gambler’s fallacy
- “The hot hand in basketball”
Connect With Ben
Listen to the Podcast! (And don’t forget to leave us a review!)
Listen to the episode on a separate page.
Subscribe to the podcast in the media player of your choice.
Listen ad-free on Stitcher Premium; get a free month when you use code “manliness” at checkout.
Podcast Sponsors
Click here to see a full list of our podcast sponsors.
Read the Transcript
Brett McKay: Welcome to another edition of the Art of Manliness podcast. Have you had a period in your athletic or professional career where you felt like you were on fire? Maybe you made a whole streak of consecutive shots in a game or executed one good idea after another at work. In his book, The Hot Hand: The Mystery and Science of Streaks, my guest today explores why success sometimes seems to arrive in clusters like this. His name is Ben Cohen and he’s a sportswriter for the Wall Street Journal. Ben and I begin our conversation with an explanation of what it means to have a hot hand, and how this phenomenon has often been studied in basketball, but can also be seen in a wide range of areas, including the film career of Rob Reiner. We then discuss what may cause winning streaks, whether or not they can be induced, and what Stephen Curry does when he starts feeling hot in a game. We also talk about what the video game NBA Jam can teach us about the psychology of the hot hand. Boom Shaka Laka.
We then dig into what the academic research has found on whether the hot hand truly exists or is really just a cognitive illusion. And we end our conversation with what you can start doing today to take advantage of having a hot hand. After the show’s over, check out our show notes at aom.is/hothand.
Alright, Ben Cohen, welcome to the show.
Ben Cohen: Thank you for having me.
Brett McKay: So you’re a sportswriter for the Wall Street Journal, you got a new book out, The Hot Hand: The Mystery and Science of Streaks. So what got you going on this deep dive on whether the hot hand exists in sports?
Ben Cohen: Well, it’s a really compelling subject, first of all. I wrote a few stories about the hot hand for the Wall Street Journal a few years ago. And honestly, usually what happens after I write stories and think about stories and spend time talking to people about those stories, is that by the time they publish, I’m so sick of them, I don’t wanna think about those stories anymore. The opposite happened with these stories that I wrote about the hot hand. I wasn’t really exhausted by them, I was kind of invigorated by them, and I felt like I was still getting started, like I was only scratching the surface of what I could possibly learn about the hot hand. That doesn’t happen often. And the fact that it did happen, made me think that there might be something bigger here.
Brett McKay: Let’s talk about what we mean by hot hand, ’cause I think everyone knows the layman’s definition of a hot hand. So let’s talk about that and then let’s also talk about how researchers, scientists, academics define hot hand.
Ben Cohen: Sure. I don’t think there’s a singular definition of the hot hand. I think it means different things in different industries, but I like to think of it very simply as when success leads to more success. That’s the simplest way to put it. So in basketball, for example… And it’s always been studied in basketball, which is one of the things that really appealed to me about this idea. In basketball, it’s when you make one shot, and then another shot, and then another shot, and you feel more likely to make your next shot. You can’t miss. You’re in the zone. You are on fire. But what’s really irresistible about this phenomenon is that it’s not simply about basketball. This is really about human behavior and I think that we are all familiar with this feeling of the hot hand, those times when we’re on a roll and nothing can stop us. And what I have found is that if we take advantage of those times, they can elevate our careers and maybe even change our entire lives.
Brett McKay: And where else do we see this, the hot hand, besides basketball?
Ben Cohen: Everywhere. Honestly, I know that sounds a little bit ambitious, but what I have found is that once you start looking for the hot hand, you bump into it anywhere you look. And so, that’s movie-making, that’s in your own careers, it’s writing, it’s investing. This power of streaks, there’s a magic to it. There’s something of a mystery to it, too, but this is not limited to basketball or just sports. It has wide-ranging impacts and it applies very, very broadly.
Brett McKay: Yeah. An example of streaks showing up in movies can be seen in the films of Rob Reiner. This guy made hit, after hit, after hit. Even with movies that had previously been difficult to get made and that people didn’t think would ever be a hit.
Ben Cohen: That’s right. The first three movies that Rob Reiner made were Spinal Tap, Stand by Me and The Sure Thing. And there was this incredible newspaper story after these three movies came out that distilled the essence of Rob Reiner’s directing career to its essence. Which basically this newspaper reporter said, “Rob Reiner’s movies are hits, not because everybody expected them to be hits, but because nobody expected them to be hits.” They were these delightful contradictions. And so, what happens after he makes three movies that nobody wanted him to make but turned out to be either critically successful or commercially successful? What happens is that people think that he has the hot hand, and the perception of him has changed in Hollywood. And so he has this incredible exchange with a studio executive around this time where the studio executive says, “We want to be in business with you. We will make any movie you want to make. Just name that movie.” And what Rob Reiner says is, “Trust me, you don’t wanna make the movie I want to make.” And she says, “No, really, just name the movie, tell us.” And he says, “No, really, you don’t wanna make this movie.”
And finally, she puts an end to this Abbott and Costello routine they have going and she says, “Just name the movie. What movie do you wanna make?” And Rob Reiner says, “The movie I want to make is called The Princess Bride.” And the studio executive says, “Anything but The Princess Bride.” And for many years, The Princess Bride had been a riddle haunted by a curse. It was the great white whale of Hollywood, even though it was written by William Goldman, who said it was the best thing he ever wrote. And this is the guy who wrote Butch Cassidy and who wrote All the President’s Men. Even though it was this incredibly rich material. Robert Redford had tried to make it and star in it, and he couldn’t. Truffaut, Jewison, all of these brilliant directors before Rob Reiner had tried to make The Princess Bride and they all failed.
What allowed Rob Reiner to make The Princess Bride, even when nobody else wanted him to make it, and even when he came very close to not making it, it was that hard, was that he did have a hot hand. There were these resources available to him. He had some capital. He had this runway and he was able to use it on this movie that has become this beloved cult classic, one of the most beloved films we have. Now, the cool thing about this movie is that it actually elevated him to an even higher level, because after The Princess Bride comes out, he then rips off When Harry Met Sally, Misery and A Few Good Men, which is like this second hot hand period. But it’s very clear that only because he did have the hot hand was he able to leverage that to his advantage. And the world, in some cases, have never been the same because… I think The Princess Bride, there are these classic lines that are seared into our memory over time.
Brett McKay: Well, and there’s researchers who study this phenomenon of success that comes up in clusters. Do they have any idea why it happens? Is it talent, is it circumstances, is it just luck? What’s going on there?
Ben Cohen: I think it’s actually a little bit of all three. I like to think of the hot hand as when it’s this collision of talent and circumstance and a little bit of luck. I think you put it very well. What these researchers who have studied creativity and workplace success have found is that our best work happens to come in bunches. Our creative hits, they’re clustered. And this is in movie making, but it’s also in science and it’s art and it’s anywhere, these researchers believe, where they would have bothered to look for it. So the people who wrote this paper a couple years ago, they wanted to put these very objective numbers to the very subjective issue of taste.
What makes a movie good and how do we know if that movie is good? And so for movies, they looked at IMDb ratings. For art, they looked at auction prices. And for science, they looked at Google Scholar citations. These are not perfect metrics, but they’re about the best that we have. And what they found is that if they knew what your best work is, they would probably be able to find your second and third-best work because it’s right around that best work. We have these hot hand periods in our careers, and the really interesting thing about that is that they tend to define our careers. They are what people remember about what we do at work. So when we think about Rob Reiner, we think about that hot hand period, we think about The Princess Bride going into Misery and A Few Good Men and When Harry Met Sally, and the movies that made that possible. So the reason they are so interested in this is because they wanna know how do we work and how do we maximize our productivity. And clearly, the hot hand plays some role in that.
Brett McKay: And have they found any ways to induce the hot hand, or is it just a matter of, it just happens?
Ben Cohen: I think it just happens, which is the elusive and the frustrating and this devilishly entertaining thing about it. I actually asked… I figured… I was like, “Who can I ask about this? Who has felt hot before that I could ask if they have any way to predict what’s coming?” And so I thought that the greatest shooter in the history of basketball would be a good person, so I asked Steph Curry about this. And I said, “Do you know when you are about to get hot?” Because watching Steph Curry get hot, I think it’s the most thrilling thing in sports. And what he said, which I think is really interesting, is that he doesn’t know when he’s going to get hot, he doesn’t know where he’s going to get hot, or why, or how he’s going to get hot. But once he does get hot, he has to embrace it. And I think that’s a cool way of thinking about it. Once you do get hot, once you realize you are in that moment, the only thing you could do is embrace it.
Brett McKay: And as you say too, Curry is a good example of how a hot hand occurs from a meeting of talent and circumstance and the special energy that emerges out of that.
Ben Cohen: To me, sometimes circumstance bends your way. And so, if you’re Steph Curry and you come along when the NBA has never put such a premium on three-pointers and people who can shoot from outside, that circumstance that he wouldn’t have had in the 1970s and 1980s. He came along at the perfect time and he had this one game that elevated his career and nothing was ever the same. Not his life, not the fate of the Golden State Warriors, not the future of the entire NBA. But there are also people who live at the wrong time and aren’t able to capitalize on that streak. Now, something else changes, clearly, which is that we have this burst of confidence. Something changes within us, and we are able to feel that momentum and our own behavior changes. Sometimes that’s good and sometimes that’s bad.
But you think about it in terms of basketball, we have these heat check moments where we feel like we have the freedom to do things that we wouldn’t ordinarily do. And so in basketball, that means pulling up from 30 feet and taking a shot with a hand in our face. For Rob Reiner, it meant making The Princess Bride. But clearly something changes. And so sometimes it’s talent taking advantage, sometimes it’s circumstance, sometimes it’s just pure dumb luck. Sometimes you just need things to happen that you wouldn’t expect to happen and probably have never happened at any other time. But if they do happen, as Steph Curry says, you have to embrace it.
Brett McKay: So some other things you looked at about the psychology of what makes streaks so appealing is you went to, I think, a classic video game of a lot of people’s childhoods, I know it was mine, NBA Jam. I remember I saw the title of this book, The Hot Hand, I was like, “He’d better talk about NBA Jam.”
Ben Cohen: Talk about it right away.
Brett McKay: “He’s warming up, he’s on fire!” What can that game teach us about the psychology of streaks?
Ben Cohen: Well, clearly, that they’re powerful. So the thing about that game is that it was made by this brilliant video game designer named Mark Turmell. And when Mark Turmell was growing up, he loved three things. He loved basketball, and he loved video games, and what he really loved was fire. He was actually a bit of a pyromaniac. And he would combine those three childhood loves into the biggest hit of his career. And so, when I grew up playing NBA Jam, when Steph Curry grew up playing NBA Jam, when probably you grew up playing NBA Jam, the game was everywhere. It was ubiquitous.
What I did not realize was that that wasn’t just me or you or Steph Curry, it was everybody. Everybody played NBA Jam. It was one of the most lucrative, successful games ever made. It made a billion dollars in quarters, not a million, a billion with a B, in less than one year. It was this monster hit. And so when I started thinking about why and when I asked Mark Turmell, “Why? What made NBA Jam so powerful? Why did we always wanna play this game?” Clearly, there are any number of theories. It was a fun game to play. Basketball was fun, even though it had nothing to do with basketball. It was a basketball game that was modeled on a sci-fi game based on a post-apocalyptic society. It was not like any other basketball game or sports game.
But what I think is that like it was magical to hear those three words, “He’s heating up.” And then those next three words, “He’s on fire.” There was something alluring about that superpower of the hot hand, and you always wanted to get to that mode where you do three things, and then a fourth thing happens. What I think is that Mark Turmell single-handedly brainwashed a generation of impressionable young minds into believing in this concept of the hot hand. Because until I read all of this literature about the hot hand, it never even occurred to me that there might not be a thing called the hot hand. Because honestly, I played NBA Jam, of course there’s a thing called the hot hand.
Brett McKay: And yeah, that’s the thing… What I think the hot hand in NBA Jam teaches you, you’re being on fire, is that it’s really addictive. You’ll keep going so you can get back in it. ‘Cause once you’re there, every shot you make for the next minute or two, it’s gonna go in.
Ben Cohen: I think “addictive” is a perfect way to do it. Addictive means you wanna keep doing it. You wanna keep feeding quarters into that machine to keep playing NBA Jam to try to get to that mode. And this was purposeful. Mark Turmell, in every game he has made since then over the last 25, 30 years, he has tried to bake some sort of hot hand mode into that game, because he knows it’s addicting and he knows that it makes people want to keep playing his games.
Brett McKay: I think people have maybe experienced the hot hand themselves. I’m not a basketball player, but I’ve had that moment when I’ve played some pickup ball where every ball I put up seems to go in. Steph Curry has experienced it, we’ve probably seen Steph Curry… Everyone’s seen Steph Curry do that. But then you talk about in the ’80s, there’s a group of academics saying, “Yeah, hot hand, it’s actually illusions.” Let’s talk about that research.
Ben Cohen: Yeah, it was this classic paper that was written by Tom Gilovich, Bob Vallone and the great Amos Tversky, who is just one the brightest minds of his generation. And what they did was they looked for the hot hand in basketball, because they had a sense that it was simply going to be a case of seeing patterns in randomness where they don’t exist. And their theory was that what we call the hot hand is actually just a way of rationalizing what we think of as patterns. And so what they were able to do was secure the best data that was available at the time, and it came from the official scorekeeper of the Philadelphia 76ers, a guy named Harvey Pollack, who was way ahead of his time.
He was nicknamed Super Stat because he was one the only people in sports paying attention to statistics back then. And they looked at the chronology of shots, the order in which they were taken. And what they were trying to find was are you more likely to make your next shot after making two or three shots in a row. Now, they asked basketball players this. They asked professional players, they asked players at their schools. Almost all of them, to a man, said, “Of course there’s such a thing as the hot hand, and it’s important to feed the hot hand. When someone has made a couple shots in a row, you want to get them the ball.” And this is an example of changing behavior, it’s exactly what psychologists and economists study. However, once they looked at the data, this play-by-play and the order of shots in which they were taken, they found there was really no evidence to support that shift in behavior.
You are actually not any more likely to make your next shot when you were on fire. It was sort of a cognitive illusion. And that made for this really delicious, contrarian, counter-intuitive paper that they published, and has since become part of the canon of behavioral economics. It’s one of the most famous papers in the history of academic psychology. And the fascinating thing about it is that even after it was published, people just wouldn’t believe it. They took it to a reporter, told the former Boston Celtics coach Red Auerbach about this paper once, and he just sneered in disgust and said, “So these guys make a paper. I couldn’t care less.” Anyone who has been in basketball and seen the hot hand and felt the hot hand just could not wrap their minds around the fact that it actually might not be real.
And that paper held up for like 35 years, and that’s changed a bit in recent years. Now, even though there has been some evidence to the contrary that has come along recently, I have to say, I still find that paper so admirable, because just the contrarian nature of it, the way that they were able to look at something and see something that nobody else had seen, is so cool to me. And I think that they’re probably right about what they found. We still do see patterns in randomness, for one thing. But also, the hot hand is not this exaggerated fireball of our imagination from playing NBA Jam. That is not real life. And so they were clearly onto something. Whether or not there’s no such thing as the hot hand I think is an open question now, and I think that we have reason to think that there is such a thing as the hot hand when circumstance allows for it.
Brett McKay: So, yeah, I think the paper does, it raises the point that human beings have the tendency to find patterns where patterns don’t exist. So an example of this, during World War II when Britain was getting bombed, they thought there was a pattern to the German bombing when in fact it was just random. And then you also highlight how music is shuffled. Our tendency to find patterns can actually mess up the way… It skews the way we think of… When we hit Apple shuffle or Spotify shuffle, we think there’s a pattern going on when there’s actually not a pattern.
Ben Cohen: That’s right. We make playlists and we ask Apple and Spotify to shuffle them, and yet sometimes we think that those shuffle buttons are broken because we think that this random music is not actually random. This was a problem, actually, that both Spotify and Apple had to solve not too long ago, because we would hear the same song twice in a row on a playlist, or we would hear the same artist twice in a row on a playlist when there are 10, 15, 20 artists, and we were convinced that something was wrong. Not only something was wrong with the algorithm, but that there was almost something corrupt happening. The record labels were paying Spotify to play certain artists more than others. That’s not what was happening, it’s just that we see patterns and we remember when we hear two songs by Beyonce in a row, even if there are lots of other artists on that playlist.
And so, what Spotify and Apple had to do was actually tweak their code and change their algorithms, and it was a bit absurd, but what they did was they took playlists and if there are 10 artists on that play list, they would evenly disperse those artists over the course of the playlist to guarantee that you wouldn’t hear the same song or the same artist twice in a row, and like no less than Steve Jobs got on stage at an Apple Keynote about 15 years ago and explained their thinking behind all of this, and it was so absurd that he couldn’t even help but laugh at the situation. Because when you think about it, what they actually did was they made it less random to feel more random, like that’s nuts, and it’s because we have a really hard time wrapping our minds around randomness.
And so, Spotify and Apple didn’t stick to their guns, and say like, “Well, this just purely random and you have to get used to it.” They actually just gave their users what we want and what we want is to not think about pure randomness.
Brett McKay: So believing in the hot hand, let’s just, let’s say the… We’re not gonna even say the hot hand exists, it may or may not, but believing the hot hand in, say, basketball, the stakes aren’t that high, it’s a game. But what would happen if someone believed the hot hand, and say like if they were a farmer, what would be the consequences of that?
Ben Cohen: Yeah, it’s the same thing as like investing in the market. And if you are a farmer. I actually took a trip to a farm on the border of Minnesota and North Dakota not too long ago to meet with a fifth generation sugar beet farmer named Nick Hagan. And I wanted to know, do you believe in the hot hand, and more important, do you behave as if you believe in the hot hand? And what Nick said is like, “Yeah, I believe in the hot hand, I have played sports, I watch basketball, I’ve seen the hot hand for myself, but I can’t believe in the hot hand when I am farming, because if you are a farmer and you look at what happened last year or the year before and you invest your resources accordingly, you’re essentially betting the farm and if you’re wrong, you go broke,” because there’s a difference in farming than there is in basketball, and in farming the way, that Nick says is that, farming is defense, which is a really interesting way to think about it.
Like when Steph Curry is shooting, he’s playing offense, he’s in control. He has agency over his own situation. Nick doesn’t, Nick’s success and his business are based on things that are pretty much random, like the weather, the weather can determine whether or not you have a good year or a bad year. And so when I think of the hot hand, I have to remind myself that what the crucial distinction here is is control. When we have control, we feel that we can have the hot hand. When we recognize that we don’t have control, we kinda know we’re at the mercy of chance, and believing in the hot hand can be dangerous, it can be costly, it can backfire and it can burn us a little bit. So there are plenty of industries, farming is a good one. Investing your money is actually another really good one where you have to recognize when you can and when you can’t have the hot hand and what your environment allows and whether you are in an industry that encourages skilled performance or random performance.
Brett McKay: So another, I guess… People would call it a fallacy, the psychologists who say the hot hand doesn’t exist, another fallacy, it’s sort of the opposite of the hot hand, is the gambler’s fallacy. What does that look like and how does the gambler’s fallacy show up in daily life?
Ben Cohen: You know, it’s funny, ’cause I like to think of the gambler’s fallacy through basketball as well. So in basketball, you make three shots in a row. Everybody in the arena thinks you’re making a fourth shot, that’s the hot hand. In gambling, it’s when you walk into a casino, you walk over to a roulette wheel, and you see the wheel land on red three times in a row. What research has shown is that most people actually bet on black the fourth time. Now, these are really interesting scenarios because they’re essentially the same. Three things happen, what do we do for the fourth. And when we think we’re in control, we have the hot hand, when we recognize that we’re not, we bet accordingly. One is the hot hand, one is the gambler’s fallacy. And the gambler’s fallacy has huge impact on our decision-making, as much as the hot hand. It’s a similar idea, it’s whether or not we are betting on the streak to continue or the streak to end.
Brett McKay: And so what is that… I mean, I’m trying to think of like a example besides gambling where people would make… They say something happens and they do the opposite because they saw a streak.
Ben Cohen: So one good example, when you think about people who make decisions, people in authority positions. There was one paper that looked at the gambler’s fallacy in a few of those industries, one was loan officers, one was baseball umpires. Baseball umpires, if you call two close strikes in a row, you’re much less likely to call a close pitch a third strike the next time, even if it actually is a strike, it’s all because you’re trying to even out the probability in your own mind. The one I write about in the book is asylum judges. This is kind of crushing and it’s a little bit depressing, but if you are a refugee in search of asylum, your application is not simply judged on your merits. They are based on like lots of other things, including when your case is heard. So asylum judges are much less likely to grant asylum if they have just granted a asylum two or three times in a row.
Now, a baseball umpire doing that might be trivial, but an asylum judge essentially has someone else’s life in their hands, and because they are trying to even out the streak, because they have all of this power and they don’t want to encourage a hot hand, they’re trying to stop it, and they’re trying to get to a point in their mind where they sort of embody this regression to the mean, like refugees regardless of the merits of their case, suffer from that which I find really demoralizing and it sort of shows the human consequence of this idea of the hot hand. This is not just about basketball or even behavior, there are huge effects that people can suffer from this beyond making one shot in basketball.
Brett McKay: Alright, so that paper written in ’86 sort of became an article of faith that the hot hand didn’t exist, at least amongst academics. They said it didn’t exist. It’s just… Everything’s random, it just appears like there’s a streak going on. But then these two Harvard students had a hunch that the previous studies that this hot hand paper was based off were flawed and that the hot hand could actually exist. So tell us about these guys and what led them to believe that the original hot hand paper was flawed.
Ben Cohen: Yeah, not just, not like a Harvard student, not like grad students or PhD students or professors. Undergrads, kids in their college dorm. They actually looked at the hot hand as part of an independent study, these Econ majors a couple of years ago, and what they wanted to know was, does today’s data bear out that result from 1985. The data we have now, it was just unavailable to the researchers in the ’80s in their nerdiest, wonkiest, wildest dreams. And what these undergrads were able to do was they were able to control for what happens when somebody gets hot.
Now, think about when someone gets hot. It works the behavior of everybody around them. In basketball, if you’re the shooter, you want to shoot more, you are taking crazier shots, longer shots, riskier shots. Your teammates are passing you the ball, your coaches are calling plays for you. The defense is adjusting as well, they’re sending double teams at you, they are making it their mission to make sure that you do not shoot because you are hot. Now, for a very long time until these kids came along, we weren’t able to control for those shifts. There was no way to know if someone was taking a lay-up, or a three-pointer, or a long three-pointer, or a trickier three-pointer. It was just a shot was a shot and that was it. But they were able to negotiate access to this trove of data that came from these high resolution tracking cameras in every NBA arena starting about 10 years ago.
And because they were able to look at the distance of the defender and the distance of the shot, they were actually able to compute the probability of that shot going in. And once they controlled for that probability, they were able to show that when you do get hot, you’re actually slightly more likely to make your next shot. Because if you control for the difficulty of a shot, it had always been masked, the hot hand. It had been disguised because we take chances, we take riskier shots and longer shots and crazier shots. And when those shots go in, they are actually a sign. And they had been all along, we just had no way of knowing it, that the hot hand actually might be a real thing. And this example was curious to me and was really appealing, because it showed that this data that we have now, this better data, not just bigger data, but better data and more granular data, it can tell us things that we had always suspected, but could never prove for sure because we didn’t have the data, the data wasn’t good enough yet, it hadn’t caught up to our own minds.
And that’s kind of what happened with the hot hand. The data that was used in that 1985 paper was the best data available at the time. But times have changed and so has the data, and the data now tells us something that we all thought to be true, we just couldn’t say for sure because the data had never been good enough.
Brett McKay: And about the same time these two Harvard undergrads were doing this study, this research, there are also two economists who also started arguing that the hot hand fallacy studies that Tversky did, missed something really important when they made their conclusion the hot hand didn’t exist. So what did Tversky miss that these guys saw?
Ben Cohen: Honestly, I have learned that trying to explain this bias that they found…
Brett McKay: Yeah, I had to read it like five times to get it.
Ben Cohen: I know this sounds like a cop out, but just read the book, just look at this table, because you kinda have to wrap your mind around it in a very strange way. And actually, that’s important, because this was a very, very subtle statistical bias that some of the world’s brightest statisticians had missed for 35 years. Nobody had seen this. And so for me to talk about it, for me to describe exactly what they found is tricky. But essentially what they found was that for 35 years, the fact that you shot the same when you were hot, was always taken as bulletproof evidence against the hot hand. You were not more likely, you were the same amount. But what these two young American economists in Europe found, was this bias that shows that if you are a 50% shooter, and you shoot 50% when you feel hot, that actually is evidence for the hot hand.
And we had been looking at this very old problem in the wrong way for almost four decades. And the math is right, it’s been rubber stamped, it’s been published by the top economic journal. All of these brilliant mathematicians and statisticians who have read the paper say it’s right. It’s super trippy, it’s really mind-boggling, but it lent this new chapter to this saga of the hot hand, that it actually kind of flipped it on its head in a little bit. It showed that we’re still thinking about this old problem, and new ways of thinking about that problem can lead us to new conclusions about it. This is sort of like the new entry into this field of hot hand studies, this growing scientific literature, and I think what it did was ensure that this debate is not dying any time soon. There are going to be more papers about this because it’s a topic, it’s an idea, it’s a phenomenon that kinda drives us a little bit crazy, and we just wanna keep thinking about it.
Brett McKay: Well, Amos Tversky’s partner, Daniel Kahneman, they wrote Thinking Fast and Slow, he was at a presentation where these guys were making their argument that they made a statistical error. And Daniel Kahneman was like, “Yeah, you guys are right.”
Ben Cohen: Yeah, exactly, I was there that day. And Kahneman says, “It’s unfortunate they made that error, but I think their point still stands that we see patterns where they don’t exist in randomness and we invent causes to explain them.” And I think that’s right. And I think the cool thing about the hot hand is that we have been talking about this idea for 35, 40 years, and there are very smart people on both sides of this debate. And I think part of the fun of it is just toying around with this idea for yourself, and seeing where you land, thinking about where the hot hand is possible, where it’s not possible, what circumstances allow for it, and what circumstances actively punish belief in the hot hand. That’s sort of the beauty of this world of ideas, is that we can each come to our own conclusions, we can do the work, we can read the papers and we can figure out what we think for ourselves.
Brett McKay: It’d be interesting to see the future of hot hand research, if they’re gonna start doing things like measuring the physiology of players who experienced the hot hand.
Ben Cohen: No, I think so too. I would love to see people strap electrodes to our brains and try to figure out exactly what is happening neurologically inside of us when we do feel hot. I think we’re able to do that sort of research now and it would be kind of fascinating to see it done in a huge population sample. And I think people would be interested in it, because people have been interested in this subject for 35 years at this point.
Brett McKay: So what do you think the practical takeaway is? Someone’s listening to this podcast right now and they’re like, “Okay. This is interesting. The hot hand maybe exists,” but how can they apply this to their lives?
Ben Cohen: Well, I think it’s important to think about when we can take those own heat checks in our lives. Recognizing when we do have the hot hand and taking advantage of them, because they can change a lot. I have felt it in my own career. There have been a few times, not many, but a few, when writing and reporting stories at the Wall Street Journal, when I do feel hot, and something has changed. And I try to remind myself in those moments, “This is the time to really hunker down and do everything you can.” Because the other thing we know about the hot hand is that it does not last forever, it goes away. And that’s the most frustrating thing about it, because we know that it’s not forever, and so we have to take advantage of it while we can. And there are a bunch of other behavioral quirks and changes that we can make.
And so, just thinking about where there is a hot hand or where you might be prone to seeing patterns, or where there might be examples of the gambler’s fallacy and how to adjust for them. In the book, I tell the story of one of these Yale economists who has studied the gambler’s fallacy, and he realized that he was subject to that very same bias that he wrote about. And so now, when he grades papers and assigns his teaching assistants to look at exams, what he does is that he recognizes that sometimes if you read one paper that comes after two A+ papers, that paper is not going to read all that well, even though it might be perfectly fine. It might be an A. It might be an A-. But when it comes after those two A+’s, it might read like a B or a B-. And so what he does is he takes all of his papers and he splits them in two and he shuffles them, and he lets his teaching assistants grade them twice in different orders to try to reduce that bias as much as possible. He will then average those two scores and weight them accordingly. And that’s not perfect, but it’s a whole lot better than what his system was then. So I think there are all types of ways of factoring these biases into our own minds and trying to adjust for them and seeing where they lead us.
Brett McKay: And if you’re not experiencing the hot hand, you just gotta keep pumping the quarters into the machine until you start warming up again.
Ben Cohen: That’s actually the secret of the book. Just keep playing NBA Jam, regardless of what you do, and I think things will work out.
Brett McKay: Well, Ben, this has been a great conversation. Where can people go to learn more about the book and your work?
Ben Cohen: Well, they can find the book anywhere books are sold, and they can read me in the Wall Street Journal and they can find my best stories and more about the book at BZCohen, C-O-H-E-N.com. And I am BZCohen on every social media platform that we’re all trying to avoid these days.
Brett McKay: Well, Ben Cohen, thanks for your time. It’s been a pleasure.
Ben Cohen: Thanks so much.
Brett McKay: My guest today was Ben Cohen. He is the author of the book The Hot Hand: The Mystery and Science of Streaks. It’s available on Amazon.com and bookstores everywhere. You can find out more information about his work at his website, BZCohen.com. Also, check out our show notes at aom.is/hothand, where you can find links to resources where you can delve deeper into this topic. Well, that wraps up another edition of the AOM podcast. Check out our website at artofmanliness.com where you can find our podcast archives, as well as thousands of articles we’ve written over the years about pretty much anything you can think of. And if you’d like to enjoy ad-free episodes of the AOM podcasts, you can do so on Stitcher Premium. Head over to StitcherPremium.com, sign up, use code “manliness” at checkout to get a free month trial. Once you’re signed up, download the Stitcher app on Android or iOS and you can start enjoying ad-free episodes of the AOM podcast.
And if you haven’t done so already, I’d appreciate it if you’d take one minute to give us a review on Apple Podcasts or Stitcher, it helps out a lot. And if you’ve done that already, thank you. Please consider sharing the show with a friend or a family member who you think would get something out of it. As always, thank you for the continued support and until next time, this is Brett McKay, reminding you to not only listen to the AOM podcast, but put what you’ve heard into action.
