Research data archiving: Difference between revisions

Browse history interactively ← Previous edit Next edit →Content deleted Content addedVisual WikitextInline

Revision as of 21:26, 13 March 2007 editRonCram (talk \| contribs)Extended confirmed users, Pending changes reviewers5,908 edits →Policy of NSF in Grant General Conditions: italics← Previous edit		Revision as of 12:03, 14 March 2007 edit undoStephan Schulz (talk \| contribs)Administrators26,889 editsNo edit summaryNext edit →
Line 1:		Line 1:
	'''Scientific data archiving''' refers to the long-term storage of scientific data and methods. Scientists are required to archive their data and methods so other scientists can audit the data, replicate the research and build on their findings. When scientists do not properly archive their research, it is known as ] because the research does not meet the requirement of ]. The need for data archiving and due diligence is greatly increased when the research deals with health issues or public policy formation.		'''Scientific data archiving''' refers to the long-term storage of scientific data and methods. Scientists are required to archive their data and methods so other scientists can audit the data, replicate the research and build on their findings.{{cn}} When scientists do not properly archive their research, it is known as ] because the research does not meet the requirement of ].{{fact}} The need for data archiving and due diligence is greatly increased when the research deals with health issues or public policy formation.{{cn}}

	Scientific journals do not perform audits. Reviewers are not paid and so they cannot be expected to do a full audit. Peer-review is only a very cursory review to spot any glaring errors and to judge if the research adds any new knowledge to the corpus. If a particular article produces controversy or spawns new government regulation, it is more inclined to warrant a full audit. If the results cannot be replicated, it fails the audit.		Scientific journals do not perform audits. Reviewers are not paid and so they cannot be expected to do a full audit. Peer-review is only a very cursory review to spot any glaring errors and to judge if the research adds any new knowledge to the corpus.{{fact}} If a particular article produces controversy or spawns new government regulation, it is more inclined to warrant a full audit. If the results cannot be replicated, it fails the audit.

	In order to prevent data loss or corruption, data should not be held by the researcher alone. Data must be archived either at a government data center, an accredited independent site or the publisher of the journal.		In order to prevent data loss or corruption, data should not be held by the researcher alone.{{fact}} Data must be archived either at a government data center, an accredited independent site or the publisher of the journal.{{cn}}

	==Policy of NSF in Grant General Conditions==		==Policy of NSF in Grant General Conditions==

Revision as of 12:03, 14 March 2007

Scientific data archiving refers to the long-term storage of scientific data and methods. Scientists are required to archive their data and methods so other scientists can audit the data, replicate the research and build on their findings. When scientists do not properly archive their research, it is known as pseudoscience because the research does not meet the requirement of testability. The need for data archiving and due diligence is greatly increased when the research deals with health issues or public policy formation.

Scientific journals do not perform audits. Reviewers are not paid and so they cannot be expected to do a full audit. Peer-review is only a very cursory review to spot any glaring errors and to judge if the research adds any new knowledge to the corpus. If a particular article produces controversy or spawns new government regulation, it is more inclined to warrant a full audit. If the results cannot be replicated, it fails the audit.

In order to prevent data loss or corruption, data should not be held by the researcher alone. Data must be archived either at a government data center, an accredited independent site or the publisher of the journal.

Policy of NSF in Grant General Conditions

36. Sharing of Findings, Data, and Other Research Products

a. NSF expects significant findings from research and education activities it supports to be promptly submitted for publication, with authorship that accurately reflects the contributions of those involved. It expects investigators to share with other researchers, at no more than incremental cost and within a reasonable time, the data, samples, physical collections and other supporting materials created or gathered in the course of the work. It also encourages awardees to share software and inventions or otherwise act to make the innovations they embody widely useful and usable.

b. Adjustments and, where essential, exceptions may be allowed to safeguard the rights of individuals and subjects, the validity of results, or the integrity of collections or to accommodate legitimate interests of investigators.

Policies by journals

• Nature: An inherent principle of publication is that others should be able to replicate and build upon the authors' published claims. Therefore, a condition of publication in a Nature journal is that authors are required to make materials, data and associated protocols available to readers promptly on request. Any restrictions on the availability of materials or information must be disclosed at the time of submission of the manuscript, and the methods section of the manuscript itself should include details of how materials and information may be obtained, including any restrictions that may apply. One preferred form of disclosure is a link from the methods section to a copy of the relevant Material Transfer Agreement (MTA) form, which is hosted as Supplementary Information on the journal's web site. Authors may charge a reasonable fee to cover the costs of producing and distributing materials. If materials are to be distributed by a for-profit company, this should be stated in the paper.

Any supporting data sets for which there is no public repository must be made available to referees at submission and any interested reader on and after the publication date from the authors directly, the author providing a URL to be used in the paper on publication.

Such material must be hosted on an accredited independent site (URL and accession numbers to be provided by the author), or sent to the Nature journal at submission, either uploaded via the journal's online submission service, or if the files are too large or in an unsuitable format for this purpose, on CD/DVD (five copies). Such material cannot solely be hosted on an author's personal or institutional web site.

After publication, readers who encounter a persistent refusal by the authors to comply with these guidelines should contact the chief editor of the Nature journal concerned, with "materials complaint" and publication reference of the article as part of the subject line. In cases where editors are unable to resolve a complaint, the journal reserves the right to refer the correspondence to the author's funding institution and/or to publish a statement of formal correction, linked to the publication, that readers have been unable to obtain necessary materials or reagents to replicate the findings.

• Science: Materials sharing After publication, all reasonable requests for materials must be fulfilled. A charge for time and materials involved in the transfer may be made. Science must be informed of any restrictions on sharing of materials applying to materials used in the reported research. Any such restrictions should be indicated in the cover letter at the time of submission, and each individual author will be asked to reaffirm this on the Conditions of Acceptance forms that he or she executes at the time the final version of the manuscript is submitted. The nature of the restrictions should be noted in the paper. Unreasonable restrictions may preclude publication.

Controversies involving data archiving

Heart research

Dr. Singh published research regarding heart attack victims. His research was questioned. The medical journal investigated for 12 years before deciding the research was probably fraudulent. If Dr. Singh had archived his data and methods prior to publication, the issue may have been resolved more quickly.

Academic genetics

Withholding of data has gotten to be so commonplace in academic genetics that researchers at Massachusetts General Hospital published a journal article on the subject. The study found that “Because they were denied access to data, 28% of geneticists reported that they had been unable to confirm published research.”

Climate change research

In 2003, Steve McIntyre and Ross McKitrick decided to audit the published findings of Michael Mann et al from an article published in 1998. Dr. Mann refused access to data and his source code. After a long process, the data was turned over. Numerous problems were found. Dr. Mann also attempted to prevent the publication of research by McIntyre and McKitrick. Dr. Mann eventually published a Corrigendum in which he admitted some errors but denied others. The criticisms of McIntyre and McKitrick were reviewed by the Wegman Panel and the National Academy of Sciences. The findings of McIntyre and McKitrick have been largely confirmed by these reviews. Without access to the author’s data and methods, these corrections could not have been made.

In 2006, Martin Juckes et al published an article claiming the source code used by McIntyre and McKitrick was not archived. McIntyre responded that the accusation was false and may be academic misconduct. False claims regarding data archiving are usually easy to establish.

Stories about data withholding

• Nature: “How to encourage the right behaviour”

• Science: “The Selfish Gene: Data Sharing and Withholding in Academic Genetics”

Misplaced Pages