As you will hopefully know by now, we have been writing to five leading medical journals to correct the record on misreported trials. We have sent 58 letters in total, and the responses have been varied: some (such as the BMJ) have issued rapid corrections, whereas others (such as JAMA and NEJM) have refused to publish any correspondence at all. The Lancet have taken a mixed approach: of 19 letters of correction we submitted they have so far published 8, accepted a further 3 for future publication, and rejected 2; with 6 still under editorial review dating back as far as 5 months.

However, there is an extra feature, around an important issue: who should take responsibility for misreported outcomes in clinical trials? We think journals have a clear set of roles and responsibilities here, not least because so many are listed as endorsing the CONSORT guidelines, which explicitly state that all prespecified outcomes should be reported. The Lancet have somewhat absented themselves from this discussion. Our published correction letters have, in most cases, been accompanied by letters from other researchers raising methodological criticisms of the trial in question. In all cases, our published letters have been accompanied by an author’s reply. Where this reply has reiterated significant misrepresentations or misunderstandings around pre-specification of outcomes, we have written a follow-up letter to the journal to point these out. However, each of these subsequent letters has been rejected by the Lancet editors; and our efforts to get the Lancet editors themselves to give a view, on misreported outcomes in their journal, have not yet had a reply. [Read more…]

At COMPare we have been checking for outcome switching in five major journals, and then writing correction letters wherever trials have been misreported, to see whether journals respond appropriately. Along the way we have identified and documented many serious failings.

Today we describe a new issue COMPare has identified. While editors at Annals of Internal Medicine maintain a public commitment to the modern era of trials transparency and reporting integrity, we have already established that in reality the journal preferentially uses unpublished protocols, which are often inaccessible to other readers, behind closed doors, to assess reporting accuracy on issues such as outcome switching. This is a surprising and concerning choice by Annals editors: to use inaccessible documents is itself concerning. Moreover, trial registers were specifically set up over a decade ago to address this need, and to address selective outcome reporting, with legislation from national governments alongside near universal support and endorsement from the medical and academic community, regulators, academic journals, and organisations such as the International Committee of Medical Journal Editors and the World Health Organisation. To undermine and abandon this resource, as Annals editors seem to have done, seems a remarkable choice, and one that warrants extensive public discussion and debate.

More concerningly, we have found that Annals has published a false claim about the accessibility of the one trial protocol that we have so far requested, and have so far failed to correct the record when this false claim has been pointed out. [Read more…]

As you will hopefully know by now, COMPare has been examining all trials published in five major journals to check if they have misreported their results by failing to report their prespecified outcomes, or by adding outcomes that weren’t prespecified. Outcome switching is widely recognised as a bad thing; however it continues to be widespread, despite various promises from journals that they have already addressed the issue.

Previous studies have measured the prevalence of outcome switching, finding it to be extensive, and quietly published academic papers giving an overall summary figure for how often this problem occurs. At COMPare we decided to go one step further: we have written a letter for publication on every trial that misreported its outcomes. Our reasons for doing this were twofold. Firstly, we wanted to test whether the mechanisms of science for correcting the record would function correctly. Secondly, we thought that the responses of the system, or the journals, might help shed light on why this problem has persisted, despite extensive public promises that it had already been addressed. You can see for yourself here the process of assessing a trial, and the kinds of misreporting we have found.

In this piece we describe the responses we have had from Annals of Internal Medicine, the world’s 4th highest impact factor medical journal. Five of the 67 trials we analysed were published in Annals, and all of these reports contained misreported outcomes, to varying degrees. We pointed out these errors in online comments for publication, on all the relevant trials. We expected that Annals editors would publish our corrections, encourage authors to address their errors, and report the prespecified outcomes as per best practice.
[Read more…]