The COMPare Trials research project assessed 5 trials published in Annals of Internal Medicine for misreported outcomes – a well-documented and common source of bias. We found that all 5 trials published in Annals over the 6 week period misreported their prespecified outcomes, and so we sent a correction letter on each trial.

We also identified a systemic problem. None of these trials had a publicly accessible protocol dated before trial commencement, so we took the prespecified outcomes from the trial registry entries, and compared these against those published in the report. This is not necessarily a problem: trial registers were set up explicitly for the purpose of preventing selective reporting of outcomes, with support from august bodies such as ICMJE, WHO, and legislation across various territories.

In response, Annals’ editors claimed that the registry entries are often “outdated, vague or erroneous”, and that the discrepancies we found are accounted for by trial protocols that cannot be accessed publicly, but are reviewed by editors behind closed doors. We view this as a very outdated approach, lacking in transparency, and a problematic view of trial registers. [Read more…]

As you will hopefully know by now, we have been writing to five leading medical journals to correct the record on misreported trials. We have sent 58 letters in total, and the responses have been varied: some (such as the BMJ) have issued rapid corrections, whereas others (such as JAMA and NEJM) have refused to publish any correspondence at all. The Lancet have taken a mixed approach: of 19 letters of correction we submitted they have so far published 8, accepted a further 3 for future publication, and rejected 2; with 6 still under editorial review dating back as far as 5 months.

However, there is an extra feature, around an important issue: who should take responsibility for misreported outcomes in clinical trials? We think journals have a clear set of roles and responsibilities here, not least because so many are listed as endorsing the CONSORT guidelines, which explicitly state that all prespecified outcomes should be reported. The Lancet have somewhat absented themselves from this discussion. Our published correction letters have, in most cases, been accompanied by letters from other researchers raising methodological criticisms of the trial in question. In all cases, our published letters have been accompanied by an author’s reply. Where this reply has reiterated significant misrepresentations or misunderstandings around pre-specification of outcomes, we have written a follow-up letter to the journal to point these out. However, each of these subsequent letters has been rejected by the Lancet editors; and our efforts to get the Lancet editors themselves to give a view, on misreported outcomes in their journal, have not yet had a reply. [Read more…]