Survivorship bias is the idea that there might be an unknown filter that’s filtering the data before you even get to see it. In the case of the plane, that’s referring to a story from WW2, where planes returning from combat were recorded for where they were shot. Famously, the recommendation was to thicken the armor on places where the planes weren’t hit, because the “unknown filter” in this case is that if the plane were shot down, then you would never be able to record where bullets hit on that plane. Hence, the most important areas of the plane are actually the places that weren’t shot in the surviving planes.
In the case of the graph, this is a graph compiled from looking through a lot of papers and recording how significant a result is. Essentially a measure of how “interesting” the data is. Here, the unknown filter is that if a result weren’t interesting, then it wouldn’t get published. As a result, there’s a gap right in the middle of the graph, which is where the data is least interesting. In recent times, there’s been a philosophical argument that even uninteresting data should be published, so that at least it would prevent wasted time from multiple people attempting to do the same thing, each unaware that it’s already been done before. Hence the reason why people made the graph in the first place
Tar_alcaran@sh.itjust.works 7 months ago
Z values are measurement of how many standard deviations something is from the mean. 95% of your values fall between -2 and +2. Most “interesting cases” are about outliers, something that’s very uncommon. If it’s common, you don’t tend to talk about Z-values.
The survivorship bias plan shows a world war 2 chart of where the bullet holes were on planes. The conclusion famously isn’t to armor those parts often hit, but to armor the parts that weren’t hit, because no planes hit there returned to be recorded.
ryedaft@sh.itjust.works 7 months ago
Oh, I thought it was about p-hacking