How to spot bad science
Call out those crazy claims with our quick guide to fake facts
1. Resist clickbait
Outrageous headlines get clicks, but the articles underneath don’t always back up the claims. It’s hard to summarise science in five or ten words, so don’t take the top line at face value.
2. Don’t jump to conclusions
Just because something worked in a lab doesn’t mean it’ll work in the real world. The science might be solid, but if the conclusion sounds too good to be true, it probably is.
3. Spot industry funding
Look twice if a sugar study was funded by a drinks brand, or a smoking study by a tobacco company. It doesn’t always mean bad science, but it’s worth bearing in mind.
4. Look out for correlation
Did you know that more people drown as ice cream sales go up? Just because two things happen at the same time doesn’t mean that one causes the other.
5. Be wary of speculation
If scientists say ‘might’, ‘could’ or ‘may’, it often means that they don’t yet have all of the evidence. More work is often needed to expand brand new findings.
6. Check the sample size
Look carefully at the number of people tested in a study. If results are based on a sample of thousands of people, they’re likely to be more reliable than results based on a sample of five.
7. Watch out for bias
Study groups can’t always represent the whole population. If a test has been done on one group of people, the results won’t necessarily apply to another group.
8. Check for controls
It’s impossible to know if something has changed if you’ve got nothing to compare it to. Good studies should have a control that scientists can refer to as their baseline.
9. Look for blinding
Blind experiments help to reduce conscious and unconscious bias. Where possible, the participants, and sometimes the scientists themselves, should not know which is the test group and which is the control.
10. Find the whole story
Sometimes only part of the story is told, particularly when science news is picked up by the media. But one finding on its own isn’t enough – science is about the big picture.
11. Beware one-offs
If there’s only one study to back up an outlandish claim, be cautious. The strongest science has lots of evidence generated independently by different people.
12. Find the source
The best question to ask is where did this come from? Good science will have been checked thoroughly by other experts in the field and should be published in a peer-reviewed journal.
This article was originally published in How It Works issue 102, written by Laura Mears
For more science and technology articles, pick up the latest copy of How It Works from all good retailers or from our website now. If you have a tablet or smartphone, you can also download the digital version onto your iOS or Android device. To make sure you never miss an issue of How It Works magazine, subscribe today!
