Where the Bullet Holes Aren't

During the Second World War, the American military had a problem that looked like a data problem.

Bombers were flying missions over Europe and coming back full of holes, some with wings torn open, some with fuselages peppered, some with damage across the tail. The pattern was visible, countable, and urgent. So was the question:

Where should we add armour?

It was the right question. Armour saves lives, but armour is heavy: add too much and the plane gets slower, shorter-ranged, less useful. You cannot protect everything, so you have to choose. The military did the sensible thing: they looked at the planes that came back and mapped the bullet holes. And the conclusion seemed almost too obvious to argue with: reinforce the places taking the most damage. Wings, fuselage, tail.

Put the armour where the holes are.

Then Abraham Wald looked at the same planes, the same holes, the same maps, and said the opposite. Put the armour where the holes aren’t: the engines, the cockpit, the places with the fewest marks.

At first this sounds absurd. Why protect the places taking fewer hits? Why ignore the visible damage? But Wald had seen what the map was really showing.

The bullet holes were not a map of where bombers were being hit. They were a map of where a bomber could be hit and still come home.

The planes with holes in the wings came back. The planes with holes in the fuselage came back. The planes with holes in the tail came back. The planes hit in the engines, mostly, did not. They were not in the hangar. They were not in the sample. They were in the sea.

That is why the empty spaces on the map mattered most. They were not evidence of safety; they were evidence of absence. The fatal damage was missing from the data, and once you see that, the whole answer reverses. The same holes that read as a sign of weakness become a sign of survivability. The most visibly damaged areas are not the ones most in need of armour. They may be the ones where damage can be survived.

Nothing about the planes changed. Nothing about the data changed. What changed was the assumption underneath the data.

The right question, the wrong path

The mistake was not the question.

It is tempting to tell this story as the military asked the wrong question, and Wald asked the right one. But that isn’t what happened. Where should we add armour? was the correct practical problem; they genuinely needed to know where armour would save the most lives. The error came one step later, in the assumed path from the question to the answer:

Look at the planes that returned. Map the bullet holes. Reinforce the most damaged areas.

That process felt empirical. It felt rational. It felt data-driven. But it rested on an assumption it never stated:

The surviving planes are the right evidence base for deciding where armour is needed.

Wald did not reject the question. He rejected the assumed method for answering it. And that distinction is the whole lesson.

A question carries assumptions: it frames the problem, points attention one way rather than another, decides what kind of answer would count. But the process of answering carries assumptions too, about what evidence counts, what sample matters, what is missing, and how the evidence should be read. In the bomber case the question was sound. The answering process was not, because it treated the returning planes as a neutral sample of bombers that had been hit. They were not. They were the subset that survived being hit, and that single fact changes the meaning of every hole on the map.

The holes no longer say protect this place. They say a plane can survive damage here. And the absence of holes says something worse: planes hit here do not come back.

What the visible evidence hides

This is why the story is more than a lesson about survivorship bias. Survivorship bias names the statistical error; the deeper point is broader.

Evidence does not interpret itself.

The same evidence can mean opposite things depending on the assumptions used to read it. A bullet hole on a returning bomber can mean this part is vulnerable or it can mean this part is survivable. The hole is identical in both cases. The difference is the frame.

That is what makes assumptions dangerous: they do their work before the argument begins. They decide what the evidence is evidence of. And because they sit underneath the process, they rarely feel like assumptions at all. They feel like common sense: look at the planes that came back, study the damage, armour where it’s worst. There is nothing stupid about that reasoning, which is exactly why it is dangerous.

The most dangerous assumptions are not the ridiculous ones. They are the reasonable ones no one thinks to inspect.

Asking backwards

Wald’s move was simple: he asked backwards. Instead of only asking what do the returning planes show us? he asked what would the missing planes show us, if we could see them? That question brought the absent cases into the room, and exposed the assumption that the visible sample was enough.

This is the practical value of inversion. It is not a trick for sounding clever; it is a way of finding the assumption the original path has hidden. When an answer seems to follow cleanly from the data, ask:

• What is missing from the data?
• What are we assuming is representative?
• What would make the visible pattern misleading?
• Which failures are absent because they failed too completely to be measured?

These questions don’t replace evidence. They protect it from being read through the wrong assumption. The point is not that every obvious answer is wrong (most are not). It is that an answer can be perfectly logical once you accept the assumptions beneath it, and completely wrong if those assumptions are false.

That is what nearly happened with the bombers. The military could have answered the question well and still reinforced exactly the wrong parts of the plane, not because the question was foolish, not because the evidence was fake, not because the reasoning was irrational, but because the assumed route from question to answer excluded the cases that mattered most. The planes that mattered most were precisely the ones that were not there.

Two layers, not one

Every serious question has two hidden layers. The question itself carries assumptions: it sets the direction and decides what kind of answer belongs. And the method of answering carries assumptions of its own: about what evidence counts, what is missing, what is representative. To reason well you have to inspect both. Sometimes the question is wrong. Sometimes the question is right and the method is wrong, and that second case is the more dangerous one, because everyone involved believes they are already working on the correct problem.

The generals had the right problem. They had real evidence and a rational method. But the method treated the planes that survived as though they stood for the planes that were hit, and Wald saw that the survivors were not the whole story. They were the filtered remains of it.

The fatal evidence was not in the bullet holes. It was in the missing planes. It was not where the damage was visible.

It was where the bullet holes weren’t.