Plenty anecdotes exist for survivorship bias. Mostly due to people using some past performance, of a stock for example, as an indicator that the same things will happen again.
Wiki defines survivorship bias as “the logical error of concentrating on the people or things that made it past some selection process and overlooking those that did not, typically because of their lack of visibility.”
For example, analysts frequently cite companies as doing well (or not) by relying on their past performance. This leads to skewed information, because authors tend to leave out companies or information that did not live up to their artificial expectations and hence do not qualify as being successful.
Good to Great: Not So Great
One example of this occurs with the book Good to Great by Jim Collins. It is one of the most popular books written on company success, having over three million copies of the book sold. In the book, successful companies are identified based on their past performance. These were selected from a group of 1,435 companies—but only the 11 most successful are chosen. Survivor bias, or survivorship bias, comes into play here because it evaluates data based on what has already happened. It takes away the objectivity when the data of different companies are compared after they have already performed. Instead, the author should have taken the entire list of companies and looked to their characteristics to guess which ones have performed well over the 40-year period. Failing to do it this way leaves the evidence with a biased skew. It also does not prove that the characteristics selected have anything to do with success. In fact, many of the companies that did not perform well over the period may share characteristics with those that were considered successful. This means that data is not really valuable when it comes to evaluating success.
Survivor bias also skews data in the case of individual samples. It is easy to say something about a group if the data you have is pre-selected. You could compare it to the idea of probability. For example, consider a situation where you have five playing cards in your hand. Before you look at the cards, the configuration of whatever comes up is three million to one. However, after you look at the cards and know what each of them is, the odds that your cards are those is only 1. You cannot determine probability as being relevant after you have already looked at the cards, because the answer is guaranteed.
This does not mean that studies that have survivorship bias are not worth anything. It does mean, however, that the data presented is biased to prove their point. While it sends the message that science was used to draw the conclusions in the book, the science behind the ideas was not conducted in an unbiased, accurate way. Therefore, you cannot necessarily trust the results.
Then again, why would someone want to read a book about a company that failed? People do not want to look to examples of failure for advice on how to be successful. Consider Steve Jobs. He quit college and moved back home, building Apple Computer with his friends out of his parents’ basement. Now, consider the likelihood of other people doing the same thing. Apple Computer was a revolutionary idea, something that did not exist at the time. It had nothing to do with the model that he followed, it was simple probability. Venture capitalists do succeed—but not at the rate that self-help books would have you believe. You could have all the statistics that companies like this share and still not find success in your venture capital, because the books do not consider all the companies that have succeeded. They ignore the fact of probability and only present the information that is going to prove their point and help sell books.
World War And Statistics
Another example of survivorship bias is the story of Abraham Wald. Abraham Wald was a statistician during World War II, with the assigned job of deciding where to add armor to bomber airplanes so they would be more likely to survive attacks from the enemy. The overall goal is to be protected from enemy fire that would try to take the airplane down, letting the bomber plane and the person inside it come home.
To decide where to add the extra armor, patterns of gunfire were looked at on the bombers that returned home after being in enemy fire. The Allies originally thought they would put armor on these spots, because it seemed they were of interest to the enemy. However, Wald explained that this would be ineffective. The planes that had been shot in these areas were able to make the ride home for repairs. The key to surviving enemy fire would be to armor the other areas of the plane, areas where bombers may have been shot down. Wald’s advice ended up being valid—thousands of Ally air crews returned home because the armory was placed where it needed be, instead of in the obvious places as indicated by the data. The Allies had been swayed by the data that already existed, but Wald looked past this and looked at the data group as a whole. This is where the most valuable, accurate information can be discovered on a topic. This is where most of us often fall short.
Further Reading: https://rationalwiki.org/wiki/List_of_cognitive_biases
Image:Lubo Minar
Leave a Reply