Abraham Wald and his work on WWII bomber damage is a really cool story highlighting why we need to always challenge the way we look at data and consider what our data is really telling us, and what data we may be missing.

Survivorship Bias

The abridged story is Wald was hired by the US Army to evaluate bomber damage and determine how to improve the armor on planes to improve their survivability. The army had been tracking damage to their aircraft and the general consensus was they needed to reinforce the areas where they were seeing damage. But Wald noted there were specific areas on the returning aircraft that were not damaged. Wald had a question though:

Where are the missing holes?

He posited those apparently undamaged areas were targeted at the same rate as the areas with recorded damage, but the population of planes that recorded the missing damage had not returned to base. This suggested damage to those areas was catastrophic to the planes, so the strategy should be to armor the areas that were lacking damage in the dataset they had.

This observation became known as survivorship bias.

The key lesson for us here is to challenge the data we have, questioning what data we are missing. Read more about Wald here:

https://en.wikipedia.org/wiki/Abraham_Wald

https://www.profound-deming.com/blog-1/the-story-of-abraham-wald