The Black Swan: The Impact of the Highly Improbable by Nassim Nicholas Taleb

I was on Slate.com the other day and a review of the recent book "The Black Swan: The Impact of the Highly Improbable" by Nassim Nicholas Taleb jumped out at me.

I'd first heard of the author from my friend Michael Sperger a year ago when he was chairing the SCIP conference in New York and was shortlisting keynotes, having read Taleb's earlier book, "Fooled by Randomness" (Michael chose Jim Surowiecki instead).

Now, I'm not much of a determinist myself but the central notion that highly improbable events through history are the ones that have shaped modern civilization and the inherent unpredictability of these events in the future makes the work of intelligence in business a good deal tougher. In fact, as pattern recognizers, we must help to manage the risk of threats emerging that look like threats we've seen before; but what happens when it's something entirely new and equally as devastating as, say, the 9/11 attacks were on the national psyche?

So I thought it was probably time to buy the bloody book and at least skim it over the Father's Day weekend (as is my custom) before some other highly improbable event comes along and I no longer have the time for it (like, say... an asteroid impact destroys all life as we know it... er, wait a sec, isn't that sort of thing actually more of a statistical certainty... like, it's just a matter of time so we need to get going on that galactic missile shield around the planet to secure the future of the species and all? Anyways, I digress...).

I found a nice few sentences by Taleb in a piece he drafted for a literary journal:

The dynamics of modern luck relate to the intractable variety of uncertainty; they produce events that this author categorizes as Black Swans (sometimes, more technically, Type-2 randomness or, even more technically, large-impact events with small but incomputable probabilities). Unlike traditional uncertainty, these unexpected events are both extremely rare, unexpected, yet command a large impact.

There is a remarkable regularity to these ubiquitous Black Swan dynamics. They are visible across disciplines and human activities. They pervasive in biology (particularly molecular biology), economics, sociology, linguistics, networks, the stock market, showing similar attributes. Literally anything that contains luck will be subjected to it. The spread of ideas and religions, the success of innovations, and historical events also follow these dynamics.

Black Swans are usually associated with 18th century British philosopher David Hume but, as Taleb points out, Hume never used the term. Indeed, Taleb goes to some lengths to establish the originality of this idea as his own... leading some commentators I've seen to call him more than arrogant in response.

The similarity comes from Hume's "Problem of Induction" that says that we can't infer anything outside our own experience. Even though I may have only ever seen white swans, that does not prove that there are no black swans. Indeed, as the graphic at the start of this post alludes, black swans (the Cygnus Atratus) are quite commonplace in Australia. The quote reads,

"No amount of observations of white swans can allow the inference that all swans are white, but the observation of a single black swan is sufficient to refute that conclusion." - John Stuart Mill paraphrasing David Hume

Instead, Taleb is more interested in approaching the same induction problem from the other direction. His Black Swan concept would say one incidence of anything doesn't tell us much about how likely it is that other similar outcomes could happen. In fact, as Taleb would say, we are "fooled by randomness" (this is essentially a sequel to that book) and have a tendency to overestimate the likelihood that unlikely events will repeat themselves.

To illustrate, even probabilities as simple as the assumption that flipping a coin will produce a 50/50 chance of heads or tails operates independent of historical vectors and leads to questions such as, "If I flip a coin 10 times and get 3 heads and 7 tails, then what does that mean?" Well, two conclusions lead to the exclusion bias - that is, either the coin is weighed to produce more tails than heads OR the next 4 flips are likely to "average out" the probabilities to produce an even 50/50 split OR at least, more heads than tails will result and the average will get "better".

But in reality, this phenomenon is luck in action. The probability of a heads-or-tails outcome is still 50/50 - it hasn't changed because of history, nor has some other externality necessarily influenced the results, such as a cheating coin. It simply turned out that way naturally.

While I bought (and indeed skimmed while the kids were napping Sunday afternoon) the book and, rather than give away my early thoughts here, I thought I'd feature what a few other people thought of it - notably, Tyler Cowen in his Slate.com piece:

The Black Swan works best as an advice book. In part, that's because the unpredictable is most undervalued in our personal lives. Too many of us are caught up in routine, or a "status quo bias," as it is labeled by economists and psychologists. We are afraid to move house or change jobs or even to imagine alternative paths. It is disquieting to think we might be making bad choices, so we close off options and we shut down self-critical reasoning, whether subconsciously or by active choice. For instance, we're likely to buy certain commercial products simply because they are familiar and therefore comforting; that is why branding and advertising so influence consumers.

The best parts of The Black Swan come when Taleb mixes financial economics and biology to explain just how much his message runs contrary to human nature. For instance, humans have evolved to draw quick but useful inferences from repeated situations, partly for self-preservation: When you see a tiger approach, run away fast. This tendency once served us. But it now makes us less well-equipped to deal with a more innovative world. We're not always ready to embrace a new paradigm or abandon an old but failed way of doing business. In Taleb's view, the winners of the evolutionary process are mostly lucky. They are not especially well-equipped to deal with the next set of challenges, such as making the right investments or responding appropriately to a terrorist attack.

Another human failing stems from the nature of happiness. In the short run, people's happiness is often shaped more by how many "positive events" occur in their day than by the arrival of one important piece of good news. Winning $100,000 in the lottery feels almost as good as winning $1 million. We therefore look, consciously or not, for small but repeated successes when we should be shooting for "one large win." It's easy to see why: Big payoffs come only rarely, and perhaps late in life; in the meantime, who wants to keep on feeling like a loser?

Nonetheless, getting out of our comfort zones is good for innovation and thus good for the economy. Young entrepreneur Ben Casnocha, in his recent memoir, My Start-Up Life, offers similar business advice: "Expose yourself to as much randomness as possible. Attend conferences no one else [in your field] is attending. Read books no one else is reading. Talk to people no one else is talking to." The late G.L.S. Shackle, a Scottish economist and one of Taleb's heroes, insisted that capitalism was driven by entrepreneurs' abilities to imagine a radically different future. Shackle argued that we live in a "kaleidic society, interspersing its moments or intervals of order, assurance and beauty with sudden disintegration and a cascade into a new pattern."

Along the way, Taleb explains why you should not mimic Casanova (he was lucky to escape the danger inherent in each romantic adventure he took), why venture capitalists make more than inventors (inventors pursue black swans, but they often die too soon to see the biggest payoffs), why you should become a speculator rather than a prostitute (the former is more scalable, in case your career really takes off), and why the self-critical Montaigne is the most important philosopher (he is one of the few writers who understood the limitations of human knowledge).

And Taleb points out, crucially, that when people do see a black swan, they usually overreact. They again focus on the immediately knownnamely, the last black swanand they fail to imagine the next generation of black swans. For instance, we are far too fearful of terrorist strikes or campus shootings, rare but publicity-rich events. If a "black swan" such as 9/11 is broadcast on TV, or if a "black swan" like a rare disease happens to our brother, it suddenly acquires an unrealistically strong hold on our imagination.

Finally, I was also reminded of how exceedingly rare events - such as 9/11 - can work against us in preparing for future risks. Bruce Schneier, noted security expert, had a piece in Wired last month on this idea as the Virginia Tech shootings were reacted to (though Schneier would say, overreacted to). Here's an excerpt featuring a cursory reference to Taleb's book:

Our greatest recent overreaction to a rare event was our response to the terrorist attacks of 9/11. I remember then-Attorney General John Ashcroft giving a speech in Minnesota -- where I live -- in 2003, and claiming that the fact there were no new terrorist attacks since 9/11 was proof that his policies were working. I thought: "There were no terrorist attacks in the two years preceding 9/11, and you didn't have any policies. What does that prove?"

What it proves is that terrorist attacks are very rare, and maybe our reaction wasn't worth the enormous expense, loss of liberty, attacks on our Constitution and damage to our credibility on the world stage. Still, overreacting was the natural thing for us to do. Yes, it's security theater, but it makes us feel safer.

People tend to base risk analysis more on personal story than on data, despite the old joke that "the plural of anecdote is not data." If a friend gets mugged in a foreign country, that story is more likely to affect how safe you feel traveling to that country than abstract crime statistics. We generally give stories more weight than statistics. We give storytellers we have a relationship with more credibility than strangers, and stories that are close to us more weight than stories from foreign lands.

In other words, proximity of relationship affects our risk assessment. And who is everyone's major storyteller these days? Television. (Nassim Nicholas Taleb's great book, The Black Swan: The Impact of the Highly Improbable, discusses this.)

Consider the reaction to another event from last month: professional baseball player Josh Hancock got drunk and died in a car crash. As a result, several baseball teams are banning alcohol in their clubhouses after games. Aside from this being a ridiculous reaction to an incredibly rare event (2,430 baseball games per season, 35 people per clubhouse, two clubhouses per game. And how often has this happened?), it makes no sense as a solution. Hancock didn't get drunk in the clubhouse; he got drunk at a bar. But Major League Baseball needs to be seen as doing something, even if that something doesn't make sense -- even if that something actually increases risk by forcing players to drink at bars instead of at the clubhouse, where there's more control over the practice.

I tell people that if it's in the news, don't worry about it. The very definition of "news" is "something that hardly ever happens." It's when something isn't in the news, when it's so common that it's no longer news -- car crashes, domestic violence -- that you should start worrying.

But that's not the way we think. Psychologist Scott Plous said it well in The Psychology of Judgment and Decision Making: "In very general terms: (1) The more available an event is, the more frequent or probable it will seem; (2) the more vivid a piece of information is, the more easily recalled and convincing it will be; and (3) the more salient something is, the more likely it will be to appear causal."

So, when faced with a very available and highly vivid event like 9/11 or the Virginia Tech shootings, we overreact. And when faced with all the salient related events, we assume causality. We pass the Patriot Act. We think if we give guns out to students, or maybe make it harder for students to get guns, we'll have solved the problem. We don't let our children go to playgrounds unsupervised. We stay out of the ocean because we read about a shark attack somewhere.

It's our brains again. We need to "do something," even if that something doesn't make sense; even if it is ineffective. And we need to do something directly related to the details of the actual event. So instead of implementing effective, but more general, security measures to reduce the risk of terrorism, we ban box cutters on airplanes. And we look back on the Virginia Tech massacre with 20-20 hindsight and recriminate ourselves about the things we should have done. In fact, the incident has been used as evidence both for and against gun control.

Lastly, our brains need to find someone or something to blame. (Jon Stewart has an excellent bit on the Virginia Tech scapegoat search, and media coverage in general.) But sometimes there is no scapegoat to be found; sometimes we did everything right, but just got unlucky. We simply can't prevent a lone nutcase from shooting people at random; there's no security measure that would work.

As circular as it sounds, rare events are rare primarily because they don't occur very often, and not because of any preventive security measures. And implementing security measures to make these rare events even rarer is like the joke about the guy who stomps around his house to keep the elephants away.

"Elephants? There are no elephants in this neighborhood," says a neighbor.

"See how well it works!"

If you want to do something that makes security sense, figure out what's common among a bunch of rare events, and concentrate your countermeasures there. Focus on the general risk of terrorism, and not the specific threat of airplane bombings using liquid explosives. Focus on the general risk of troubled young adults, and not the specific threat of a lone gunman wandering around a college campus. Ignore the movie-plot threats, and concentrate on the real risks.