The Fat Lady Sings Stochastically

Book Review:

Nassim Nicholas Taleb, "Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets", Penguin Books, 2007.

First published in 2001 (with an updated edition in 2004), Nassim Nicholas Taleb's Fooled by Randomness marks the first book in the "Incerto" series. It was selected by Fortune magazine as "one of the smartest books of all time" -- that's setting some high expectations! In my own opinion, this book is somewhat underwhelming after reading The Black Swan (and I recently finished Antifragile, which is also more original and expansive). But that being said, Fooled by Randomness is not a bad book; it is still a showcase of the author's wit, and marks a transition point between his life as a trader and as a literary essayist and professor. One can even witness the author's thinking shift from the "randomness foolishness" of Wall Street (this book), to examining the rare event more broadly (The Black Swan), to looking at the upside of randomness (Antifragile). So, while N.N. Taleb says that the books in the Incerto can be read in any sequence, I'd suggest reading them in the order that they were written.

The cover of the 2007 Penguin edition. Does the black cat signify luck?

The Prologue summarizes the core thesis of the book in terms of the "Table of Confusion": the left column of the table (luck, randomness, survivorship bias, volatility, noise, induction, the contingent, etc.) is often mistaken for the right column (skills, determinism, market outperformance, return, signal, deduction, the certain, etc.). These distinctions get confused in business, politics, literary criticism (where everything is a "symbol"), philosophy and even science. And for Taleb, randomness disguised as non-randomness is more dangerous than the other way around. Especially in the markets, a lucky idiot parading as a skilled investor can be pernicious. The remedy is unfortunately not as simple as telling people to be more rational -- we humans are "flawed beyond repair" (p. xliv), so we need tricks to go around our flaws.

In the Preface to the second edition, the author clarifies that his argument is not that everything is random, just that things are more random than we think. Preparation, hard work, perseverance and so on may necessary for success, but they are not sufficient to cause success. Likewise, Warren Buffett may be skilled, but the point is that a large enough population of investors will inevitably produce a Warren Buffett out of luck. Thus, we need to be skeptical, not nihilistic.

I'll summarize the rest of the book below.

***

Part I: Solon's Warning

The first part of Fooled by Randomness introduces the concept of skewed, asymmetric outcomes (where one failure outweighs many successes) and the problem of induction (how one cannot affirm the statement "no swan is black"). The title of Part I is a reference to the ancient Greek legislator Solon, who allegedly said the following to King Croesus of Lydia:

"The observation of the numerous misfortunes that attend all conditions forbids us to grow insolent upon our present enjoyments, or to admire a man's happiness that may yet, in course of time, suffer change. For the uncertain future has yet to come, with all variety of future; and him only to whom the divinity has [guaranteed] continued happiness until the end we may call happy." (p. 3)

Solon was saying that the gifts of luck can also be taken away by luck. Nassim Nicholas Taleb illustrates this with his fictional characters Nero Tulip (who you may recognize from The Black Swan) and John, both traders. While Nero is risk-averse and trades conservatively, John chases high yields and big houses. John was making more money than Nero, until one day in 1998 he "blew up" (lost almost everything) due to the rare event. For a while, John may have been the alpha male (and made Nero jealous), but he turned out to be a lucky fool in the words of the author.

Taleb argues that, in some sense, people like Nero or even dentists are richer than Wall Street hotshots, because if they were to live the same careers over and over again, on average they would be exposed to fewer unlucky disasters than John. Taleb admits that this is a "bizarre accounting method" (Chapter 2), yet taking into account unobserved possible outcomes makes some sense when you think about the quality of decisions. We should judge a game of Russian roulette not just based on its outcome, but on its alternative histories. Should a winner be used as a role model? Clearly it would be better to get rich through dentistry (insofar it is possible) than through Russian roulette. Similarly, Taleb suggests that the victors of history (e.g. Julius Caesar or Alexander the Great) may have been less lucky in other possible worlds -- so we shouldn't automatically assume that they followed better strategies than the losers. Conversely, those who try to manage the risk of rare events are seldom rewarded, but accused of wasting money on insurance for something that didn't happen.

The idea of alternative histories can be further illustrated by Monte Carlo mathematics. Essentially, you generate a whole bunch of random sample paths (sequences of virtual events) and observe the dispersion of results. (See also: Monte Carlo simulations as tools of decision-making in Super Thinking.) Playing with his Monte Carlo engine is what taught Nassim Taleb to trade options in a way that insures against blowups. It also helps resist what Taleb calls "denigration of history" -- the tendency to dismiss things that happened to others as not applicable to oneself. At this point, the author writes something that is reminiscent of the idea of "future blindness" from The Black Swan:

"It is hard to imagine that people who witnessed history did not know at the time how important the moment was. Somehow all respect we may have for history does not translate well into our treatment of the present." (p. 55)

Of course, this is also related to hindsight bias, which you may have heard about. To learn from history, we don't need grand-scale historical theorizing (Taleb considers that pseudoscience); rather, we need to appreciate the range of scenarios that can happen over the long run, including the mathematical concept of ergodicity (which says that very long sample paths tend to resemble each other, or the average of shorter Monte Carlo sample paths). This has the interesting implication that those with true skill eventually rise over lucky fools in the long run, and therefore that "age is beauty". It also provides a reason not to follow daily news too closely -- most of it is noise rather than distilled knowledge. Looking at the value of your stock portfolio every minute only creates needless emotional burnout. On the other hand, it is okay to be fooled by randomness when it comes to harmless yet beautiful art and poetry.

Monte Carlo methods can also teach evolutionary thinking. Fools of randomness (like John from  Chapter 1) tend to overestimate the accuracy of their economic analysis, get married to their trading positions, pretend that they are long-haul investors (rather than short-term traders) when they start losing money, lack plans for what to do in a market dip, overlook the possibility of their methods being flawed, and deny reality when losses happen. According to Taleb, these people are only successful due to randomness -- having the right style in the right markets -- but as soon as the system changes, they get "wiped out". A naïve view of evolutionary theory is that only the fittest survive; but Darwinism applies to reproducing species over a long time horizon, so in the short-to-medium term, negative traits may actually have a survival advantage. In this sense, evolution can be "fooled by randomness". (See also: Yudkowsky on how evolutions are stupid.)

The remainder of Part I deals with skewness, epistemic opacity, and the problem of induction. By "skewness", the author means that the mean (aka average or expected) survival is not the same as the median survival, because there is an asymmetry in outcomes. For example, if the median survival for stomach cancer is eight months, the mean survival could be considerably longer. To be precise, your expectation is a function of both the frequency and the magnitude of the outcome. The market may be quite likely to go slightly up over the next week, but it might still be better to bet on the market going down if it could go down by quite a lot. For this reason, Nassim Taleb does not like to call himself "bullish" or "bearish" about the market -- he simply waits for rare events that allow him to make a large payoff. Such a strategy can take an emotional toll, however, because we are wired to avoid frequent losses (even though the magnitude is more important). The idea of asymmetry can also be applied to knowledge, in which case we have to deal with the problem of induction. Taleb addresses this at length in The Black Swan, but the treatment in Fooled by Randomness is slightly different. Here, he writes about Victor Niederhoffer, a "statistical arbitrageur" who emphasized empiricism (one should test any testable statement) but didn't integrate it with a structured, deductive methodology. His naïve empiricism fails to distinguish between the statements:

• "No swan is black"; and
• "Not all swans are white".

The first statement cannot be affirmed no matter how many swans you observe (unless you observe all of them), but it can easily be rejected by finding a black swan. Niederhoffer thought that the market's past would tell him about the future. Of course, he blew up. By contrast, the trader George Soros (who is a follower of the philosopher of science Karl Popper) was open to changing his mind, and he seems to have survived. Taleb closes Chapter 7 by reflecting on the strategy of making bets using statistics and econometrics, but not using past history to manage one's risks. Rather, one could use stop losses to limit the costs of being wrong.

***

Part II: Monkeys on Typewriters

Nassim Nicholas Taleb starts the second part with a thought experiment: if an infinite number of monkeys tapping away on typewriters eventually produce the Iliad, we should not be as impressed as we would be if there were only five monkeys. That's because with a larger sample size, there's a bigger chance that one monkey is performing well simply due to luck. Moreover, if we focus only on the winner and forget all the other monkeys, we may fall victim to the survivorship bias: a faulty inference produced by a biased sample.

For example, if you live on Park Avenue in Manhattan, you might feel like a loser compared to your corporate executive neighbors, even if you are doing well relative to the general population. The failures are out of sight, since only the rich can afford to live in the area (Chapter 8). Of course, some professions face more randomness than others -- a dentist is unlikely to become successful if he or she knows literally nothing about teeth, but a fund manager can obtain a "good track record" merely from asset price inflation in bull markets (which, Taleb quips, is easier than frying an egg). Indeed, a Monte Carlo simulation can show us that even in a population of incompetent managers, there will be a minority who will make a profit five years in a row (and they will get all the attention). Even in science, survivorship bias shows up in the form of publication bias: research with no result is less likely to make it to print. Like Sherlock's dog that did not bark, the absence of findings can itself be informative, so this is a problem (Chapter 9). See also: the problem of silent evidence from The Black Swan.

Taleb maintains that extreme success is most often due to luck. He argues that life is full of nonlinearities, where small inputs can translate into disproportionate results. A popular example from chaos theory is the butterfly effect. And the initial advantage may come from the help of randomness: consider a Hollywood actor, who is famous because he is famous... in a different sample history, he may not have won the lead role in the audition if the examiner was in a different mood that day. There are more examples: due to path dependence and network externalities, people use the QWERTY keyboard even though it is non-optimal; book sales explode after crossing a critical level of word-of-mouth; and Google gets more hits than the "National Association of Retired Veteran Chemical Engineers" (which Taleb possibly made up) because it is already such a well-connected hub. There is also what Taleb considers the "good side" of randomness. In cases where randomness can be used to break stalemates, we are dealing with Buridan's donkey. A donkey that is equally far away from food and water will die if he can't decide which one to get to first, unless a nonlinear nudge is injected.

The final chapter in Part II provides an overview of the literature on our brains' inability to understand probability. You can imagine yourself being on vacation in rainy Paris or on the beach in the Bahamas, but not a combination of being 50% in one spot and 50% in the other. Likewise, it is hard to visualize the mathematical expectation of a bet; we tend to be swayed either by the excitement of winning or the fear of losing. Of course, there is no mathematical difference between a 75% fat-free hamburger and a 25% fat one. Yet our brains have evolved to use various shortcuts. In the words of Herbert Simon, we are "boundedly rational" and we tend to "satisfice" rather than optimize. Moreover, psychologists like Daniel Kahneman and Amos Tversky have demonstrated, using experiments, that our thinking under uncertainty is fundamentally flawed. We use quick-and-dirty heuristics that lead to biases, even when there are economic incentives to be right. Some of these heuristics have been known to traders under different names, for example:

• Prospect theory and anchoring (reacting to changes from an arbitrary reference point, rather than absolute performance, and feeling losses more than gains) --> "I'm as good as my last trade"; "life is incremental"; "loss of perspective"
• Affect heuristic, or risk-as-feeling theory (emotionally salient or visible events seem more probable, also related to the availability heuristic) -->  "Sound-bite effect"; "fade the fears"
• Hindsight bias (things seem more predictable after the fact) --> "Monday morning quarterback"; "it was so obvious"
• Belief in the law of small numbers (overstating the importance of small samples, jumping to conclusions) --> "You were wrong"
• Two systems of reasoning (the fast, effortless System 1 and the slow, controlled System 2) --> Brooklyn smarts vs. MIT intelligence
• Overconfidence (taking risks because one underestimates the odds) --> "It will never go there"

The field of evolutionary psychology agrees that people have trouble with probabilistic reasoning, but offers the explanation that we have evolved for a different environment: one where outrunning leopards was more important than efficiently computing the odds. Hence, some heuristics may be "ecologically rational". Meanwhile, neurobiologists (especially Damasio) have suggested that emotions are essential to making decisions. Indeed, we may feel emotions before finding explanations.

All these findings conflict with conventional economic theory, which is based on rational mathematical models. However, this does not mean that we should condone ignorance. According to Taleb, we need to find a balance between the glib lawyer who knows no math or science, and the mathematical economist who misuses his models out of bad judgment. Unfortunately, the media pushes us too far in the probability-blind direction, for journalists often mix up absence of evidence and evidence of absence. TV pundits are selected to look good and sound smart, not to understand statistics. While Taleb frequently attacks journalists, he admits that he too gets fooled by randomness; but at least he knows it.

***

Part III: Wax in My Ears

This brings us to the third and final part of Fooled by Randomness. This part is more personal, as the author reflects on how he lives with "randomitis" by figuratively filling his ears with wax -- in other words, not trying to be like Odysseus, who listened to the sirens' songs while tied to a mast. Taleb gets emotional, especially when reading comments from his critics... so he just doesn't read them. He also mutes the television while watching the news. These are among his tricks for dealing with a world of uncertainty.

Interestingly, people (unwittingly) accumulate various "gambler's tics" akin to Skinner's pigeons. For example, a trader might associate wearing a particular tie with his performance. Even people who should know better fall victim to superstitions of causality, since brain and instinct are not always aligned. Taleb himself recounts the following:

"... I have experienced leaps of joy over results that I knew were mere noise, and bouts of unhappiness over results that did not carry the slightest degree of statistical significance. I cannot help it, but I am emotional and derive most of my energy from my emotions." (p. 232)

So his approach is simply to avoid looking at his performance report. He also calls out charlatans who think they can predict the markets -- including Nobel laureates such as Harry Markowitz, Robert Merton and Myron Scholes (of the infamous LTCM case). Again, in Chapter 13, Taleb shows admiration for George Soros, who can quickly change his mind and admits that he is fallible. Unfortunately, people tend to get married to their ideas. This also applies to scientists, which is why, as the saying goes, "science evolves from funeral to funeral".

Finally, Chapter 14 concludes with a thought on stoicism. In the face of randomness, the best we can do is to preserve our dignity by acting heroically. A stoic is courageous, wise, but still has human emotions. When a misfortune befalls you, show personal elegance and don't pity yourself nor blame others. In the Epilogue, Taleb reiterates Solon's warning by having the character Nero survive cancer and become wealthy, only to crash his helicopter one windy day in London.

Postscript

We are not done yet: Taleb adds three "afterthoughts in the shower". The first is that people's salaries tend to be inversely related to their contribution or skill. This is because top managers can more easily hide their incompetence behind charisma and a small number of large decisions that, thanks to the monkey-on-the-typewriter problem, yield profitable results (even if the process was foolish). In the end, the shareholders get fooled by randomness. The second thought is that randomness can improve our happiness when we learn to let go of strict schedules, and settle for satisficing rather than maximizing (here some foreshadowing of The Black Swan, in which Taleb writes that he doesn't run to catch trains). For Taleb, writing without the certainty of a word limit makes it more enjoyable (incidentally, I also haven't set word limits on my blog). Unpredictability in reactions can also be used to deter one's adversaries. Finally, the third thought is that everything good and wrong, according to Taleb, seems to flow from the following core generator: "We favor the visible, the embedded, the personal, the narrated, and the tangible; we scorn the abstract" (p. 262).

***

If you've read my summary of The Black Swan, you will no doubt recognize many commonalities between that book and Fooled by Randomness. Both books deal with the problem of induction, the denigration of history, the trouble with news, survivorship bias/silent evidence, the butterfly effect, cognitive biases, and stoicism. There are the fictional vignettes, cryptic chapter summaries and opaque headings that are among the signatures of Taleb's work.

As with The Black Swan, Taleb also quotes the baseball coach Yogi Berra:

• "It ain't over until it's over" (p. 4)
• Paraphrased: "Past data has a lot of good in it, but it is the bad side that is bad" (p. 126)

These reflect Solon's warning and Popper's falsificationism, respectively.

And of course, Taleb's irreverence toward economics, econometrics and academic finance shines through in Fooled by Randomness -- it seems to be a common sentiment in his work. Yet Taleb himself admits in Chapter 7 that: "Lumping all economics in one basket shows a bit of unfairness and lack of rigor" (p. 123). In that vein, there are some economists whom he holds in slightly higher regard, including Keynes, Knight, Shackle, Harsanyi, Akerlof, Spence, Shiller, Lucas, and Vernon Smith. Stiglitz may earn plaudits for his work in information economics, but Taleb turns on Stiglitz in a later book (Antifragile, which I shall review here soon), calling him a "fragilista". Oh well.

Taleb's trademark style is always part of the package -- he even writes in the Preface of Fooled by Randomness that he ignored most of the recommendations of his editors (which hasn't stopped him from becoming a best-selling author). This is a subjective thing which some readers will find more irritating than others, but I personally enjoy his writing. Granted, it sometimes feels like the author is just showing off his knowledge of ancient Greek literature or French poetry; but at other times, he comes across as fairly down-to-earth.

Giving Fooled by Randomness a rating was hard, because it seemed at first like an inferior version of The Black Swan. It is shorter, less original, contains fewer images and figures, and does not have a glossary like The Black Swan does. Moreover, as I mentioned before, there is much overlap in content. Yet upon further reflection, I can understand how each book takes a unique perspective on the problems of the Incerto; for example, Fooled by Randomness talks about Monte Carlo engines, ergodicity, evolution, and Buridan's donkey, while The Black Swan addresses Platonicity, the distinction between Mediocristan and Extremistan, the expert problem, the barbell strategy, and Mandelbrotian "gray swans". I think it is worth reading both -- however, the latter still seems a bit more important, if you were only going to read one book. For this reason, I'd rate Fooled by Randomness at 4/5 stars, which is still a decent score.

There is something oddly therapeutic about the book: the reader is reminded that people can, through no fault of their own, be less lucky in life. Yet in the long run, the smart and skillful eventually rise.

***

As for the character Nero Tulip crashing his helicopter, I went back to The Black Swan to check for continuity, and indeed, when Yevgenia Krasnova meets Nero in Chapter 7 he is described as having "a stiff gait since he was recovering from a helicopter crash...".