Sense about Science ? equipping people to make sense of science and evidence
ArchiveView full archive
- The Troubled Families debacle
- Citizen science in Europe: How to take a strategic approach
- It's silly to assume all research funded by corporations is bent
- The strange end of the Saatchi Bill
- Here's a plan to help the government to do better than its anti-lobbying clause
- Making the government's use of evidence more transparent
- Sense About Science at the METRICS conference
- Submission to the Independent Commission on Freedom of Information
- The vets are coming!
- The Times 10th October 2015
Posted by on 07 December 2015
Hosted by the Meta-Research Innovation Center at Stanford University (METRICS), the conference gathered a vanguard of reformers around the problems of reproducibility and transparency, and why a lack of each is undermining trust in biomedical research. The goal of the conference was to move beyond describing the problems—although there was much of that—and find at least the outlines of a solution.
Much as the muckraking journalist Upton Sinclair observed that “It is difficult to get a man to understand something, when his salary depends upon his not understanding it,” the conference wrestled with the dissonance in scientists valuing transparency and reproducibility as core virtues of good science even as the academic and commercial world encourages them to assiduously avoid the former and ignore the latter. Openness and collegiality can be fatal to an academic career. Many academic researchers feel they must protect their highly valuable research as not to be “scooped” by competitors, but simultaneously, they realize the value of data sharing and how it can advance science for the common good. It’s a conundrum for any academic aspiring to tenure.
Brian Nosek, Executive Director of the Center for Open Science at the University of Virginia, argued that scientists needed to draw out these core values—quite literally in the form of badges signaling a commitment to data transparency. Such public symbols, while seeming “stupid,” as Nosek put it, actually work: In the journal Psychological Science, badges drove data sharing from 3% to 38%. Why? They signal community standards and enable social norming.
Stuart Buck, Vice President for Research Integrity at the Laura and John Arnold Foundation (which is a funder of both METRICS and Sense About Science USA) argued that regulators needed to set much higher community standards for data transparency: the Food and Drug Administration (FDA), for example, is sitting on a trove of data that doesn’t appear in academic study publication, and which is often vital to clinicians. He also detailed the absurdist lengths that Gilead Pharmaceuticals went to in order to avoid publishing data from its clinical trials: they claimed proprietorial confidentiality on the statistical methods they used. Ask any statistician and they’ll tell you that sharing your statistical methodology is essential to validating sound scientific findings and necessary for proper peer review; but that’s only half the point: how exactly can you justify proprietary claims on statistical methods? These processes are the building blocks of scientific research in the same way that addition and multiplication are the building blocks of math.
Tracey Brown, founder and director of the UK’s Sense About Science argued for the importance of including the public in this process of reform—as the AllTrials campaign has done by working with patients and patient groups to promote clinical trial transparency. Unless the public was engaged in “science about science” (to use Nosek’s phrase), the result would be cynicism: People would not discriminate in its mistrust of science, with disastrous consequences for public policy and individual behavior—along with funding for future research.
One commenter, who strongly advocates for reform of the current www.clinicaltrials.gov reporting system, went so far as to suggest that patient power and strike action—catalyzed by a “Patients Bill of Rights”—would be the most effective way to bring about change. Such a bill would advocate for non-participation in clinical trials if certain criteria were not met; other parties claimed that the standard of patient care was actually better when participating in a clinical trial.
While there are great strides being taken to promote clinical trial data sharing, there are still many hurdles to overcome—many of which involve the researchers themselves, their companies, academic journals, and academic institutions. METRICS showed that academics are ready to explore multifaceted solutions—and also that such change has to be in partnership with the public.
Posted by Tracey Brown on 03 December 2015
This is Sense About Science’s submission to the Commission on Freedom of Information. In it we raise concerns that researchers, research bodies, politicians and civil servants have shared with us, about making it harder to get hold of the research that underpins government policy.
It’s clear from our conversations with people who promote transparency on the use of evidence, whether they’re inside government or in civil society, that government hasn’t yet considered how changes to FoI might affect the public’s access to evidence. So if you value transparency on how government commissions, publishes and uses evidence, now is the time to speak up in whatever way you can.
Dear Lord Burns
Sense About Science is conducting an inquiry, led by the former appeal court judge the Rt. Hon. Sir Stephen Sedley, into concerns that government departments sometimes fail to publish the research they commission for policy promptly and in accordance with their own guidelines. The inquiry will report in Spring 2016, but I want to bring the points below, from initial scoping and evidence, to your attention, in case your commission is not aware of them.
The internal deliberations of public bodies: Should different protections apply to different kinds of information that are currently protected by sections 35 and 36? (Q1)
Removing the public interest test from Sections 35 and 36 suggests a blanket exemption of the deliberations for policy development, which would include related research commissioned by government.
- Independent research organisations and funders, such as the Wellcome Trust, are telling us that their rules require open publication of research, including research they undertake in partnership with others such as government. Restrictions on freedom of publication would prevent them entering into these contracts.
- You might also consider – in light of the public fuss that already occurs when government-commissioned research is withheld and in light of the research community’s support for open science – whether restrictions would make it hard to attract top researchers to government commissions. Our inquiry is already hearing from researchers whose experience of withheld research have made them reticent to work with government again.
- You might consider, too, whether it is defensible for government to insist on data sharing in other contexts
- Researchers and civil servants have shared with us good examples where complex, uncertain and potentially controversial research has been published promptly and in full, and communicated with the public. These show that exemption of research for policy development is unnecessary, and any desire for exemption may be better addressed by improving communication skills and learning across departments to communicate research better. (A matter for our inquiry recommendations rather than yours!)
Your commission should comment on how section 35 and 36 protections would distinguish between strategic advice for policy development and the underlying evidence. It is questionable whether the public interest would ever be served by such a blanket exemption, which would be over-inclusive and would result in government withholding information that could easily be published to improve public discussion.
Protection for information that involves candid assessment of risks (Q3)
It’s important that risks relating to matters of national security and defence are assessed candidly. Arguments that you have heard in favour of increased exemptions from FoI include the possibility of publication creating a chilling effect on this candour.
- However, researchers involved in Ministry of Defence and other security related agencies have told our inquiry that with carefully managed publication, redacting specific information but publishing the rest, even very sensitive risk analysis on things such as readiness for terror attacks can be communicated without compromising security.
- As some defence-related information is already in the public domain, or can easily be obtained, tighter exemptions around sensitive information could put government in a ridiculous position of withholding data that are openly available through other channels. Even in the highly sensitive example of FOI requests to release previous versions of a dossier on weapons of mass destruction in Iraq, the Information Tribunal concluded that “the ‘chilling effect’ would have been quite limited, given that the Hutton Report had not only put into the public domain a great deal of information on the subject but had also provided a detailed description of the circumstances in which the Dossier had been prepared, so that the public was in a good position to place the Williams draft into its correct context.”
- Government departments often find out about the research conducted in other parts of government through its release into the public domain.
Blanket exemptions on risk assessments relating to the delivery of major government projects would also harm public scrutiny. Such assessments are an essential part of the chain of reasoning behind government decisions, forming part of the case for or against a policy. As the Philips inquiry into the government’s handling of the BSE crisis showed, it is better to communicate candid assessments of risks, even if there is uncertainty, than to cover them up.
Fear of losing control of the way difficult topics are discussed is understandable, and may lie behind the desire to withhold sensitive information. But this risks undermining and trivialising the concept of information that is genuinely sensitive on grounds of national security. Exempting all sensitive information could encourage department personnel to behave as dolts who cannot distinguish between such information and that which they can readily publish without causing harm. An indistinct definition of national security is of no benefit to any party.
Executive veto over the release of information (Q4)
The Commission should clarify how any Ministerial veto would affect the publication of research commissioned by government, with reference to other protocols and the guidelines that cover the publication of government research and with reference to conditions of research contracts, which stipulate that findings should be published promptly and in full. Which arrangements would prevail – contractual agreements, ministerial and civil service codes, or a veto?
I hope your Commission reflects on the need to maintain and strengthen the public’s right to scrutinize the evidence underlying government decisions, as I know do many colleagues in the research community and in government who advocate openness on the use of evidence in public life.
Director, Sense About Science