Post by wvengineer on Feb 11, 2016 19:50:05 GMT
A couple weeks ago I listened to a podcast done by NPR's Planet Money team. They were looking at the statue of modern science and even in peer review journals, there have been a large rash of experimental bias in science. There are several main causes.
Planet MOney 677: The Experiment Experiment
I highly recommend that you listen to it. It's is only 21 minutes long.
To summarize the problem, they found that many published experiments contain the follow bias:
1. Interesting Result Bias. This is where they start an experiment and the get something interesting, so they keep the experiment going, thus they get the more of the same. However, when the do the same test and get a negative result, the test ends and they don't think anything of it. The problem is that the two test are not equal. By continuing one and not the other, they are changing the conditions of the test. They are unintentionally favoring the interesting results over the boring ones.
2. File Cabinet Bias. This is where the interesting studies are much more likely to be published and the boring ones are stuffed into the file cabinet. So in today's world where there is pressure and prestige in publishing research, the more interesting results will get the notice, not the ones that confirm the status quo.
3. Lack of actual peer review. The purpose of publishing experiments in peer review journals is so that other scientist can repeat the same experiment and confirm the results. The problem is that this rarely actually happens. People publish stuff all the time and everyone else is also trying to get published, so that they never go back and cross-check the results of other scientists. This allows studies with faulty experiments to stand because they are never challenged.
It seems like these days the only stuff that gets cross examined is the really big announcement. CERN's "discovery" of faster than light is a classic example. If it wasn't for people calling BS on it, Who knows what it would have done to the world of science. This is something however, that commonly these large profile results are nearly impossible to duplicate because designing and building the experiment takes years and many billions of dollars. It's not like Cal Tech or MIT have a collider on the scale of the LHC in the basement of one of their labs. Other institutions rarely have the resources to do it. So if there is a flaw in the equipment design of one, there really is no one that can check it and they will keep getting the defective results.
For the little stuff, there is rarely incentive for people to check the results a that are published.
The reason I brought this up is that today they announce the discovery of Gravitational Waves, something that Einstein predicted. I don't know if it the science is legit or not, but the above does call into question if the test really legit, and how do you even confirm it?
The other end of this is all these little experiments you hear about constantly. Stuff like a glass of wine a day will reduce your chances of getting skin cancer by 15% or something like that. How often are these findings done with questionable conditions, but it never gets confirmed. How much of our modern science is based on these faulty science and what does that do to daily life?
Planet MOney 677: The Experiment Experiment
I highly recommend that you listen to it. It's is only 21 minutes long.
To summarize the problem, they found that many published experiments contain the follow bias:
1. Interesting Result Bias. This is where they start an experiment and the get something interesting, so they keep the experiment going, thus they get the more of the same. However, when the do the same test and get a negative result, the test ends and they don't think anything of it. The problem is that the two test are not equal. By continuing one and not the other, they are changing the conditions of the test. They are unintentionally favoring the interesting results over the boring ones.
2. File Cabinet Bias. This is where the interesting studies are much more likely to be published and the boring ones are stuffed into the file cabinet. So in today's world where there is pressure and prestige in publishing research, the more interesting results will get the notice, not the ones that confirm the status quo.
3. Lack of actual peer review. The purpose of publishing experiments in peer review journals is so that other scientist can repeat the same experiment and confirm the results. The problem is that this rarely actually happens. People publish stuff all the time and everyone else is also trying to get published, so that they never go back and cross-check the results of other scientists. This allows studies with faulty experiments to stand because they are never challenged.
It seems like these days the only stuff that gets cross examined is the really big announcement. CERN's "discovery" of faster than light is a classic example. If it wasn't for people calling BS on it, Who knows what it would have done to the world of science. This is something however, that commonly these large profile results are nearly impossible to duplicate because designing and building the experiment takes years and many billions of dollars. It's not like Cal Tech or MIT have a collider on the scale of the LHC in the basement of one of their labs. Other institutions rarely have the resources to do it. So if there is a flaw in the equipment design of one, there really is no one that can check it and they will keep getting the defective results.
For the little stuff, there is rarely incentive for people to check the results a that are published.
The reason I brought this up is that today they announce the discovery of Gravitational Waves, something that Einstein predicted. I don't know if it the science is legit or not, but the above does call into question if the test really legit, and how do you even confirm it?
The other end of this is all these little experiments you hear about constantly. Stuff like a glass of wine a day will reduce your chances of getting skin cancer by 15% or something like that. How often are these findings done with questionable conditions, but it never gets confirmed. How much of our modern science is based on these faulty science and what does that do to daily life?
