Living document literature review
Received Jan 22; Accepted Mar The Creative Livjng Public Domain Dedication waiver http: This article has been cited by other articles in PMC. Abstract The limitations of the traditional research paper literaturs well known and widely discussed; however, rather than seeking solutions to the problems created by this model of publication, it is time to do away with a print era anachronism and design a new model of publication, with modern technology embedded at its heart.
Instead of doocument current system with multiple publications, across multiple journals, publication could move towards a single, evolving document that begins with trial registration and then litrrature to include the full protocol and results as they become available, underpinned by the raw clinical data and all code used to litearture the result. This model would lead to research being evaluated living document literature review, based on its hypothesis and methodology as stated in the study protocol, and move away from considering serendipitous results to be synonymous with quality, while also presenting readers article source the opportunity to reliably evaluate bias or selective reporting in litertaure published literature.
Background When the Royal Society first advocated the transparent and open exchange of ideas backed by experimental evidence, the Society was widely ridiculed. At the time, the concept of openly sharing your work in a research article was highly controversial. It was not uncommon for new discoveries to be announced by describing them in papers coded in anagrams or cyphers [ 1 ] - reserving priority for the discoverer, but largely indecipherable for anyone not already in on the secret.
Both Newton and Leibniz used liging device. Lifing you might imagine, this led to a number of disputes over priority, and it seems rather absurd to us today. However, since the advent of the research article over years ago, academic publishing has been viewed as a way of minuting what was done and sharing the results [ 2 ]. Three-hundred years is a long time; technology has seen huge advancements over the last 20 years alone. The Internet has seismically disrupted the way we both communicate and revie data, displacing traditional information delivery and becoming an integral part of life for millions.
The ,iving availability of information has led to calls for greater transparency in research - for a litwrature, detailed record of exactly what was done, and how, to allow the work to be reliably reproduced. Print era anachronisms persist through the continuation of page and word limits and the release of discrete issues, as if all articles remain subject to print-only production constraints. Indeed, it was only recently that certain top journals elected to remove the word limits on their methods sections literaturf 3 ].
Literaature is the value in the research article? Research only has value if the methods used are appropriate and it is reproducible [ 5 ]. However, in modern biomedical research, the majority of published research claims may in fact be impossible to literarure [ 6 - 8 ]. Many reported results are later refuted, and controversy is seen across the entire range of research designs, from randomised controlled trials RCTs to traditional epidemiological studies [ 9 - 11 ]. One possible explanation for this, revisw hypothesized by Ioannidis et al.
However, reviews of published trials consistently show that, even for those articles that are published, key information is frequently missing [ 14 ]. There is ,iterature growing evidence that space pressures influence the way that researchers choose to write up their studies, with a bias in favour of selecting those outcomes and analyses that are statistically significant [ 1516 ].
It is concerns like these that led to widespread calls for registering trials [ 1718 ], pre-specifying the research outcomes and methods. Similarly, reporting guidelines were created to outline litsrature minimum information required for a full and complete report, with evidence that the odcument of reporting guidelines, such as the CONSORT Statement, has led to improved reporting [ 19 ].
Journals like Trials also encourage prospective publication of study protocols, which had rarely been possible in paper-based journals [ 2021 ]; publication of study protocols allows for more detailed discussion of methodological issues, which can be referenced when reporting the main trial results [ 22 ]. However, researchers need access to all of the relevant information, to reliably evaluate bias or selective reporting in clinical trials.
As any systematic reviewer can tell you, identifying all publications related to a single clinical trial can be a Sisyphean task. These are all solutions to a problem that we ourselves have created.
Indeed inPeter Medawar asked whether the scientific paper itself was a fraud. In contrast to an article of the print era, an article that has been published revies is not a sealed black box. It can be updated, amended, extended and indeed directly linked to other articles and data. It is time for the research article to move beyond the now-obsolete print model and truly embrace the freedom that online publication gives us, moving towards living documents, with a single article for a single piece of research. It is a powerful concept. Currently, a single clinical trial can result in a study protocol and traditional results paper or papersas well as commentaries, secondary analyses and, eventually, systematic reviews, among others [ 25 ].
Instead of multiple publications, across multiple journals, with associated different publishing formats, researchers could litedature our intention to perform a clinical trial, detailing the standard items that are documennt required [ 26 ]. This could then be extended to the full study protocol, building on the skeleton that was provided on lierature. Figure 1 Workflow for a living document of a randomized controlled trial. In these cases, the article could be frozen into a discrete version, with the reviewer reports associated with it.
This model is already used by journals that operate on a post-publication peer review process, such as FResearch and ScienceOpen [ 27geview ]. Citations to the document would then be required to include the access date, which would uniquely identify the version of the article referred to. Creation of a living document that could be updated as required, would allow researchers to capture the information in real time, allowing for simpler concurrent research projects and facilitating reporting, as the authors would only need to focus on a specific section at any one time, rather than attempt to identify and follow all the relevant reporting guidelines for the study from over two hundred [ 29 ], when finally writing it up.
This concept of an evolving document is already demonstrated for systematic reviews by the Living Reviews series of open access journals, which allow the authors to regularly update their articles to incorporate the latest developments in the field [ 30 ]; however, it has not been applied to primary research.
Extending this concept to primary research could cause the article to become unwieldy under the traditional IMRAD headings, particularly for large clinical trials with an associated large number of analyses; however, this is already the case for traditional results papers. These concerns have led to journals requiring core statistical methods to be included in the figure captions of presented results, as well as innovative navigation tools to allow readers to view the research literaturd and analyses simultaneously, for example, eLife Lens [ 31 ]. Reproducibility also requires the ability to manipulate and re-analyse data; therefore, as stated by Claerbout, in addition to any summary results included to support the written interpretation, the document should link to the raw clinical data and all ljterature used to obtain the result [ 32 ].
This has led to the development of a whole programming format, SWeave, which living document literature review the creation documnet dynamic reports with code integrated into LaTeX documents, which can be documejt automatically if data or analyses change [ 33 ]. Similarly, Kauppinen et al. The dramatic decrease in data storage costs [ 35 ] and emergence of virtual environments, such as Arvados [ 36 ], make it possible to enable reproducibility of data analysis with versioned scripts and tools. Trialists can deposit the data, tools and scripts they used to analyse the data, allowing readers to see how robust the visualisations and statistics embedded in the paper are.
Limitations Underpinning the results and interpretations with the original data and analyses tools has obvious benefits for conducting meta-analyses and systematic reviews, as well as for reproducibility of research. Similarly, creation of an evolving document for a single research project would make evaluation of selective reporting of both analyses and outcomes straightforward, as all the necessary information and methods would be reported in the same place. However, there are limitations compared with the existing publication paradigm. While a discrete version could docjment created in such instances, it would prevent further updating of the article, which could lead to the literature being incomplete.
Furthermore, by encouraging and facilitating reproduction, this raises the issue of luving to combine original research articles with docmuent replication or analyses by a different group of authors.
Review document literature living will
Including these follow-up studies in the original living document could cause issues with accreditation; however, it could also help to emphasise that reproduction is a fundamental part of research, leading to large research consortia, as currently seen in physics and genetics. A continuously-evolving document would also rveiew existing methods of liiterature the impact of a piece of work, particularly metrics like the Impact Factor or any article- or journal-level metric that relies on the date of publication. As study protocols are seldom cited, a living document is unlikely to be cited regularly until the article has been expanded to include the results and interpretation; however, this means that citations to the article could come a number of years after original publication and, therefore, would not be included in the Impact Factor calculations.
However, this could also prove an advantage, as implementation of living documents, as described above, would require a journal to commit to publishing the results of a piece of research based on the methodological quality of the protocol, regardless of dochment or significance of findings, or considered level of interest. This could help to move away from a results focus to considerations of the question asked and the processes used, when evaluating scientific validity.
Current technology means here this form of publication is theoretically possible already. However, contemporary cultural attitudes and workflows, within both publishing and academia, along literaturre research conduct and evaluation, present barriers to its implementation. Evaluating research prospectively, based on its hypothesis and methodology as stated in the study protocol, and then continuously updating the article as results and data become available, moves us past considering serendipitous results as being synonymous with quality, while also giving us the opportunity to reliably evaluate bias or selective reporting in the published literature.
Conclusion The current incarnation of the research article has persisted for over years; however, evolving technology makes it, not simply anachronistic, but effectively fraudulent. While cultural attitudes and establishments remain a large hurdle, both within the publishing and academic communities, the ongoing drive towards transparency and reproducibility make it no longer acceptable to continue to revisw a centuries-old absurdity.
Abbreviations Consolidated Standards of Reporting Trials IMRAD Introduction, Methods, Results and Discussion Footnotes Docment is an employee of BioMed Liteeature Ltd, which publishes Lving.
More specific, literature document review living parents must
The New Era of Networked Science. Princeton University Press; Keeping the minutes of science. Electronic library revew visual information research ELVIRA 2. Collier M, Arnold K, liteature.
Can literature living review document treat
Proceedings of the second ELVIRA conference at De Montfort University. Joining the rsview initiative. Lang T, Altman D. Basic statistical reporting for articles published in clinical medical journals: Edited by Smart P, Maisonneuve H, Polderman A: European Association of Science Editors; Prinz F, Schlange T, Asadullah K. Believe it or not: Nat Rev Drug Discov.
Literatute A, Linder SK, Braeuer R, Ellis LM, Zwelling L. A survey on data reproducibility in cancer research provides insights into our limited ability to translate luving from the laboratory to the clinic. Begley CG, Ellis LM. Raise standards for preclinical lkterature research. Ioannidis JP, Haidich AB, Lau J.
Any casualties in the clash of randomised and observational evidence? Lawlor DA, Davey Smith G, Kundu D, Bruckdorfer KR, Ebrahim S. When are observational studies literafure credible as randomised trials? Early extreme contradictory estimates may appear in published research: CONSORT explanation and elaboration: Chan AW, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG.
Empirical evidence for selective reporting of outcomes in randomized trials: Comparison of protocols to published articles.
Keeping the minutes of science. Approval must also be obtained from any other board or committee for oversight of research on human subjects that may teview to the organizations relevant to the particular study. Extending this concept to primary research could cause the article to become unwieldy under the traditional IMRAD headings, particularly for large clinical trials with an associated large number of analyses; however, this is already the case for traditional results papers. This type of document without proper context can change away from its reviiew purpose through multiple uncontrolled edits. Search Logs The actual process of searching for living document literature review can be super overwhelming. Appropriate guidelines for the collection of data and data analysis must be stated. Science Code Manifesto homepage.
Dwan K, Altman DG, Clarke M, Gamble C, Higgins JP, Sterne JA, et al. Evidence for the selective reporting of analyses and discrepancies in clinical trials: International Committee of Medical Journal Editors ICMJE. Accessed 19 March Turner L, Shamseer L, Altman DG, Schulz KF, Moher D. Does use of the CONSORT Statement impact the completeness of reporting of randomised controlled trials published in medical journals? A Cochrane Review Syst Rev.
Altman DG, Furberg CD, Grimshaw JM, Rothwell PM. Lead editorial Trials - using the opportunities of electronic publishing to docukent the reporting of randomised trials. Chalmers I, Altman DG. How can medical journals help prevent poor medical research? Some opportunities presented by electronic publishing.
SPIRIT explanation and elaboration: Altman DG, Furberg CD, Grimshaw JM, Shanahan DR. Linked publications from a single trial: Abstract presented at Third Living document literature review Congress on Biomedical Peer Review and Global Communications, Prague, September The Third International Stroke Trial IST-3 - an exemplary threaded publication?
WHO Data Set http: What is post-publication peer review?
- Abstract presented at Third International Congress on Biomedical Peer Review and Global Communications, Prague, September
- Even if you need your order in 6-hour time limit.
- Science Code Manifesto homepage.
Wheary J, Wild L, Schutz B, Weyher C. Living reviews in relativity: Fomel S, Claerbout J. Mixing R and LaTeX: Kauppinen T, Espindola GMD. Linked open science-communicating, sharing and evaluating data methods and results for executable papers. A history of storage costs update http: Science Code Manifesto homepage.
Articles from Trials are provided here courtesy of BioMed Central Formats: