Science Isn’t Difficult; It’s Just Misunderstood
These days, anyone can declare himself an expert. “Experts” can get credentials online by entering a credit card number and either print out a certificate or wait for it to arrive in the mail. During Sunday afternoon’s featured session, “Telling the Story of Science in an Age of Misunderstanding,” author, broadcaster, and medical doctor Ben Goldacre said that experts are not always reliable but science is. In the world of evidence, there is a hierarchy of reliability and validity: (from most useful to least useful) systematic review and meta-analyses; randomized controlled trials; cohort studies; case-control and cross-sectional studies; case series and case reports; ideas, opinions, and anecdotes. Even when a scientific study falls on the higher end of this hierarchy, people distort science to fit whatever narrative they are trying to define. And according to Goldacre, the simplest way to misrepresent science is with numbers, charts, and graphs.
Scientists are also guilty of misrepresenting data. Goldacre said that the most common example of this is when researchers extrapolate results. For example, the scientific finding that red wine could help prevent breast cancer was observed in a petri dish, not in humans. Yet the results from an in vitro study were generalized for humans and publicized as if this phenomenon had been part of a human clinical trial. Goldacre also bemoaned correlative studies because one can find correlates for the most implausible associations; moreover, correlation does not equal causation.
He also pointed out that meaningful human studies are hindered by ethical issues and time restraints. The key to obtaining information that is truly useful is knowledge management, so Goldacre believes that systematic reviews and meta-analyses of randomized controlled trials are best for obtaining definitive conclusions. As great as these are for providing good data, they too have imperfections: Meta-analyses can suffer from publishing bias (i.e., only certain results are published), and randomized controlled trials can either take too long or require too many resources. Goldacre said that efforts must be made to reduce the costs of randomized controlled studies.
Goldacre was most critical of observational studies: He said that observational studies (which include cohort and case control studies) provide the worst data available and should no longer be published. Such studies often conclude “more research is necessary.” According to Goldacre, that statement is rather useless unless it specifically identifies what research needs to occur. “Pseudoscience has consequences. Every time we give in to the desire to play fast and loose with data, we give permission to others to misuse evidence,” Goldacre concluded.