Sense about Science ? equipping people to make sense of science and evidence
‘The review was biased’. ‘They left out relevant studies’. ‘The scope of the review was limited’.
These are all criticisms that have been levelled at systematic reviews on contentious subjects like organic food, homeopathy and vitamin supplements, often followed by a call that the review must therefore be ignored. But these accusations fail to grasp how systematic reviews are conducted, not recognising that for a review to be useful it must set a standard for the studies it includes and focus on a well-defined, question.
So what makes a good systematic review? Why are certain studies excluded? If they don’t contain new research why are they more authoritative than other studies?
In November 2009 we published Sense About Systematic Reviews, a two page briefing, to respond to these questions and explain:
- why systematic reviews matter
- why choosing studies for inclusion in a systematic review is not a personal or political decision but based on scientific reasoning
- the role they play evaluating all the evidence for a particular question and guiding research and policy decisions.
- what questions should be asked when looking at media stories about systematic reviews.
Sir Iain Chalmers, James Lind Initiative: “If scientists embarking on new research do not routinely use scientifically defensible methods for finding out what is known already, research resources are wasted. In addition, in an applied field of research like health care, the result of not beginning and ending reports of research with systematic reviews of other relevant evidence is that patients suffer and sometimes die unnecessarily.”
Dr Richard Horton, Editor, The Lancet: “Placing the results of any scientific study into context is essential to prevent the misunderstanding of new research. Sadly, too often researchers, funders, and journal editors fail to interpret new work in the light of the totality of available evidence. The consequence may well be harm to patients. Without an attempt to integrate the results of new research into existing knowledge, science has no meaning – and can sometimes be dangerous.”
Dr Evan Harris MP: “Politicians need to be able to make decisions based on the best available evidence. Wherever possible, evidence-based policy needs to use systematic reviews because they are the "gold standard" in that they evaluate the totality of evidence for any question and sift the good quality evidence from the bad and the frankly ugly which are thrown at policy-makers by vested interest groups.”
Dr Alan Dangour, London School of Hygiene and Tropical Medicine: "Systematic Reviews are an enormously powerful tool available to scientists to assess the quality, and synthesise the totality, of the available evidence on a specific question in an unbiased way. An understanding of the methods involved in systematic reviews is vital to help appreciate the significance of their conclusions.”