all posts post new thread

Off-Topic Replication Crisis

Status
Closed Thread. (Continue Discussion of This Topic by Starting a New Thread.)
JeffC

JeffC

Level 7 Valued Member
I am always sceptical of studies. Living next to a major road makes you, insert percentage, more likely to develop Alzheimer's. Eating bacon causes cancer.
The Trouble With 'Scientific' Research Today: A Lot That's Published Is Junk

I wish I could find the podcast, but a Meta-scientist stated that 85% of all the medical studies he and his team try to replicate are not replicable. An experiment being replicable is one of the basic principles of sound science. People take junk science at face value and make life altering decisions based on studies of what you should and should not do.

Further more a study that states you are 17% more likely to such and such when factored into the base probability is actually a miniscual difference.

Especially when it comes to food, supplements, and exercise their is an extraordinary amount of nonsense. I have been spending a lot of time waiting in doctors offices and clinics, as my wife is pregnant, and all these so called men's magazines are filled with nonsense. They are complete trash. Everyone wants an edge, advantage, or shortcut, easy, best, but there is no substitute for sound principles, hard work, and common sense.
 
Last edited:
Geoff Chafe said:
wish I could find the podcast, but a Meta-scientist stated that 85% of all the medical studies he and his team try to replicate are not replicable
Click to expand...

Not sure if you refer to Kamal Patel of examine.com. I've listened to him discuss this very thing but can't remember exactly where, this may be it, at least very similar:
Podcast #285: The Real Science of Nutrition and Supplements | The Art of Manliness

Or this one: How To Know Which Supplements To Take

The titles are not just about supps, he goes into the misrepresentation of data and a number of issues.

......observational studies suggest that doing too many rop ladders affect
Short term memory recall! Well, one observational study does.....is this a trend? Now where did I put my keys?......
 
@ali No, but I like examine.com. I do not like theartofmanliness.com to me that fits in with the junk men's magazines. It was a science podcast Quirks and Quarks. I cannot find which one.

This particular metascientist talked also about how they gear studies toward news to draw attention and funding to their reasearch and that is an aspect of modern science. These do and don't do studies always get media attention and people crave being told what to do.
 
Even if we assumed the outcomes of all studies to be true the presentation of the reaults.is often entirely messed up as well.

For example the recent outcry about red meat being classed a class A carcinogen (the same category as cigarette smoke) was reported so badly. I had to do a teaching session around this for medical students explaining that the relative and absolute risk were never comented on.
Relative risk was a 17%increase in bowel cancer, but this equated to an approxiately 1% absolute increase: i.e. if you took 100 people who never ate red meat approx 6 of them will develop bowel cancer, a group that does eat red meat would have 7 people develop it.
Skepticism is everything when it comes to research.
 
@DavThew That is exactly what I am talking about. People see the headline and all of a sudden eating meat is as bad as smoking. People alter their lives based on a misleading study or misinterpretation of said study, and more misleading junk news headline.
 
Most studies involving real-world humans are so complex with so many variables that it is practically impossible to replicate the study. This is a whole another ballgame compared to dropping an apple from the second story window.

Most of the studies can be dismissed. Some of the studies can be seen significant on a statistical level. I, however am an individual. Still, I would like to avoid heart disease. I'll take a moderate approach.
 
Hello,

To quote Socrates: "The only true wisdom is in knowing you know nothing".

When I can, I always cross information sources

Kind regards,

Pet'
 
Geoff Chafe said:
@ali No, but I like examine.com. I do not like theartofmanliness.com to me that fits in with the junk men's magazines
Click to expand...

Know what you mean. Only listened to a couple of them. Kamal Patel's podcast, wherever I heard it, taps into those issues you raised.
Interesting thing with gluten.......research is very mixed. Some argue that it is nonsense, a fad and taking snippets of research to drop them into claims help only those wishing to cash in on the fad diet. I can absolutely see that....how research is manipulated for potential money making opportunities. And then a few weeks back a study published by Harvard stated that gluten free diets increase a risk for type 2 diabetes. This on the basis that gluten free products have less fibre in them than gluten products! Now I dunno, the gluten free diet that I indulge in is high in fibre, fruit and veg, and as far as I'm aware a banana is gluten free, so is spinach, as are peppers and effing apples and cashew nuts. Wtf! Comparing gluten crap to other gluten free crap. It was done on behalf of the American Heart Association and they really like their bagels and cakes with all the diabetes curing properties. Who is fooling who here? A bagel with gluten v a gluten free bagel......Harvard medical school? Diabetes. Am I missing something? Not being a scientist and all.
 
The problem with studies/documentaries/experiments is that everyone has an agenda and are sometimes being paid to do these things by people who often want a certain result.

It often bleeds through in the final product that the public consumes.
 
Fiverthirtyeight.com has a lot of interesting articles on the state of science.

Here's one that asserts the "replication crisis" is not as much of a crisis as it might appear:
Failure Is Moving Science Forward

Here's one on problems in nutrition research:
You Can’t Trust What You Read About Nutrition

Here's one on the widely reported project to try to replicate studies in the field of psychology:
Psychology Is Starting To Deal With Its Replication Problem

Here's one on some general issues in research:
Science Isn’t Broken

And here are a couple about how mistaken beliefs and incorrect information get propagated:
Who Will Debunk The Debunkers?
Your Brain Is Primed To Reach False Conclusions
 
Editor in Chief of the Lancet---

“The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness.”

Editor in Chief of the New England Medical Journal----

“It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of the New England Journal of Medicine”

And it isn't just medical research but leaks over into all manner of scientific publications. It gets even better if you do any homework on the authors and funding of research, as so much of the Govt funded stuff is every bit as agenda driven as the private sector research.


This is making for a crisis of sorts as one can hardly trust any published source. We as a peoples have common big issues that need to be addressed but how can one form policy when one cannot discern reality? I don't think its a conspiracy so much as a fog of misinformation and information inextricably woven together.

In reference to the gluten example, a generation or two ago there wasn't a medical condition known as "gluten intolerance". One either had Celiac disease to some extent - a condition that can be tested for, or one didn't. Folks might have allergies to smut or some other peripheral problem but not to gluten itself. Now we have an epidemic of gluten intolerance. I myself have taken to mostly eating only bread products that are organic or that I made myself from organic wheat. Any problems I had with gluten have disappeared. Of course all modern farming practices are perfectly safe so what is happening?

Is Autism, MS, and a host of other ailments on the rise or is it just better reporting? If it is on the rise what is driving it? Oh, that's right, here's another study showing rates are actually dropping, no need to look for causality. I just experienced the warmest Winter following one of the warmest years on local record but some studies show the planet is actually cooling and solar activity is expected to decrease further - we need to bundle up, stop worrying about Arctic methane emissions and CO2 and drill more - BTW oil is not a fossil fuel but is formed abiogenically from deep earth hydrocarbons - we'll never run out.............:confused::eek::mad:;)
 
Hello,

Maybe there is some "fashion" in saying one has an intolerance or whatever. In France, there are now tons of products "gluten free", "fat free", everything free excepted to price...

Kind regards,

Pet'
 
I don't disagree with most of what's here. A couple of thoughts, as someone who has published papers (in the field of physics).
  • Scientific papers are not meant to represent scientific "facts". They are meant to report an results or sometimes just a thoughts to other scientists. A paper is meant to be read in the context of the papers that they cite, and the papers that, in tern cite, it. Reading one paper is like hearing one line from a conversation in a bar. Reading one headline, based on a paper, is like hearing that same line after it's been put through a game of telephone at a local elementary school.
  • Science is a process. It's similar to the idea of evolution. Lots of papers can be wrong, lots of individuals can die young, but the knowledge and the species continues to get better.
  • There is a replication crisis. It comes down to incentives. There is simply no appreciation or credit for replicating other peoples work. Add to that the fact that it's difficult to replicate real results and no one does it.
  • There is, in my view a worse problem - No one publishes experiments that "didn't work", for whatever reason. Sometimes its because the result is not unexpected. Some time's its because the results seem ludicrous. Whatever the reason - for any given unrepeatable result there might be people out there who have tried it, before it's even published, and know that's not what happened. There is no formal way to spread this knowledge. It's simply passed on in the field word of mouth.
  • I come from the field of physics where there is not a ton of money at stake in what the truth turns out to be and reality can slap you pretty hard when you get it wrong. I hate to think how bad things can get in fields where there are vast pools of active money at stake and we only notice something's wrong when too many people seem to be dying 40 years later.
 
Geoff Chafe said:
Especially when it comes to food, supplements, and exercise their is an extraordinary amount of nonsense
Click to expand...
so true that I believe, what I like to hear, see or read. Here is a market, marketing and money.
Geoff Chafe said:
Everyone wants an edge, advantage, or shortcut, easy, best, but there is no substitute for sound principles, hard work, and common sense.
Click to expand...
hard to sell, because almost anyone with common sense "knows" it.

pet' said:
To quote Socrates: "The only true wisdom is in knowing you know nothing".
Click to expand...
"I know, I know nothing" do I lie now or do I tell the truth? Self-referentiality or Socrates' irony. Paradox.

That leads me to Wittgensteins' last sentence of his Tractatus: "Whereof one cannot speak, thereof one must be silent."
 
Harald Motz said:
"I know, I know nothing" do I lie now or do I tell the truth? Self-referentiality or Socrates' irony. Paradox.

That leads me to Wittgensteins' last sentence of his Tractatus: "Whereof one cannot speak, thereof one must be silent."
Click to expand...
Hurray for this. Reminds me to pick up GEB again.
 
Status
Closed Thread. (Continue Discussion of This Topic by Starting a New Thread.)
Back
Top Bottom