Comment on kiss kiss, bang bang
Contramuffin@lemmy.world 2 days agoSurvivorship bias is the idea that there might be an unknown filter that’s filtering the data before you even get to see it. In the case of the plane, that’s referring to a story from WW2, where planes returning from combat were recorded for where they were shot. Famously, the recommendation was to thicken the armor on places where the planes weren’t hit, because the “unknown filter” in this case is that if the plane were shot down, then you would never be able to record where bullets hit on that plane. Hence, the most important areas of the plane are actually the places that weren’t shot in the surviving planes.
In the case of the graph, this is a graph compiled from looking through a lot of papers and recording how significant a result is. Essentially a measure of how “interesting” the data is. Here, the unknown filter is that if a result weren’t interesting, then it wouldn’t get published. As a result, there’s a gap right in the middle of the graph, which is where the data is least interesting. In recent times, there’s been a philosophical argument that even uninteresting data should be published, so that at least it would prevent wasted time from multiple people attempting to do the same thing, each unaware that it’s already been done before. Hence the reason why people made the graph in the first place
Pyr_Pressure@lemmy.ca 2 days ago
I can see why journals would not want to publish boring papers in the days of paper magazines and limited space but why would they not be published digitally nowadays? Limited by people able to review them?
Contramuffin@lemmy.world 2 days ago
No reason, I suppose. In my opinion it seems to just be a holdover from the previous systems of publishing. The prestige of a journal is ranked based on how often it gets cited (or in other words, how influential the papers are within the journal). Publishing insignificant/uninteresting data would lower a journal’s average citation count, which would make it seem less prestigious than other journals. Hence journals are incentivized to only publish interesting data. It’s a shitty system that everyone knows is shitty but nobody has a good solution for how to fix it
Pyr_Pressure@lemmy.ca 1 day ago
Wouldn’t this mean that other smaller journals could still publish the random common boring studies and then pretty much everyone would be citing those, since the boring stuff like “Trees consume carbon dioxide” would be cited a lot more than specific stuff like “Molecule XX~2~ can affect the brain development of Augustus caesarius if introduced between the ages of 3 and 6 months” and than become super prestigious?
Contramuffin@lemmy.world 1 day ago
A broad statement like “trees consume carbon dioxide” would actually be an incredible paper to publish because it means that there is 1. a lot of interesting data that could back up a statement so broad and 2. extremely applicable to a wide variety of fields. When I say “uninteresting,” I really mean a very specific type of uninteresting, like "sunlight does not affect the growth of the fungus Neurospora crassa. " It’s uninteresting because it doesn’t really tell us what affects the growth of the fungus, only that sunlight does not. If you got this result, you likely wouldn’t even feel like it’s information that’s worth making public, hence the lack of papers that have these sorts of results. But, if it weren’t published, then grad students across the globe would keep testing sunlight and keep finding the same thing again and again, wasting time and money. Hence the argument that all data should be published, regardless of how useless the results are