I loved this book! To me the main message is that we all understand that prediction is hard but we don't do a lot to improve our ability to predict or to calibrate our predictions. In other words, when we make a prediction we treat is as a static item - we don't update it based on new knowledge and we hardly ever do a reasonable analysis to determine whether we have made a good prediction or now. To quote:
A lot of the book is related to the author's own research:
These were funded by IARPA (the intelligence community’s equivalent of DARPA). After studying and identifying a group of superforecasters and their characteristics, Tetlock asked the natural question: Are superforecasters born, or can we all become superforecasters?
The mechanism used to understand whether a prediction is good or not (an idea new to me at least) is a Brier score. The math behind this system was developed by Glenn W. Brier in 1950, hence results are called Brier scores. In effect, Brier scores measure the distance between what you forecast and what actually happened. So Brier scores are like golf scores: lower is better. Perfection is 0. A hedged fifty-fifty call, or random guessing in the aggregate, will produce a Brier score of 0.5. A forecast that is wrong to the greatest possible extent—saying there is a 100% chance that something will happen and it doesn’t, every time — scores a disastrous 2.0, as far from The Truth as it is possible to get.”
If your aim is to improve your own ability to make predictions, Tetlock will both give you valuable advice and explain how following that rather simple-sounding advice may be harder than you think. In summary, to become a super forecaster, you need to:
“PULLING IT ALL TOGETHER We have learned a lot about superforecasters, from their lives to their test scores to their work habits. Taking stock, we can now sketch a rough composite portrait of the modal superforecaster. In philosophic outlook, they tend to be:
Not much:-) But I will say that when you apply this to the business forecasting that we do, is it any wonder that we don't often get it right. The good news is that we can get better.
As said, I enjoyed this book a lot.
Thought this was a good discussion on why “book learning” may not help you and why you need to “do”:
“To demonstrate the limits of learning from lectures, the great philosopher and teacher Michael Polanyi wrote a detailed explanation of the physics of riding a bicycle: “The rule observed by the cyclist is this. When he starts falling to the right he turns the handlebars to the right, so that the course of the bicycle is deflected along a curve towards the right. This results in a centrifugal force pushing the cyclist to the left and offsets the gravitational force dragging him down to the right.” It continues in that vein and closes: “A simple analysis shows that for a given angle of unbalance the curvature of each winding is inversely proportional to the square of the speed at which the cyclist is proceeding.” It is hard to imagine a more precise description. “But does this tell us exactly how to ride a bicycle?” Polanyi asked. “No. You obviously cannot adjust the curvature of your bicycle’s path in proportion to the ratio of your unbalance over the square of your speed; and if you could you would fall off the machine, for there are a number of other factors to be taken into account in practice which are left out in the formulation of this rule.”8 The knowledge required to ride a bicycle can’t be fully captured in words and conveyed to others. We need “tacit knowledge,” the sort we only get from bruising experience. To learn to ride a bicycle, we must try to ride one. It goes badly at first. You fall to one side, you fall to the other. But keep at it and with practice it becomes effortless—although if you had to explain how to stay upright.”