COPE EUROPEAN SEMINAR Publication ethics from student to professional, London, March 22, 2013.
Much of what is discussed in this paper is founded on a number of publications for which I bear (shared) responsibility, either as (co-)author, or as member of the authoring committee, including:
The European Code of Conduct for Research Integrity, published in 2011 by ESF (www.esf.org) and ALLEA (www.allea.org),
Responsible Conduct in the Global Research Enterprise, published in 2012 by the InterAcademy Council (IAP), (www.interacademycouncil.net)
Investigating Research Misconduct Allegations in International Collaborative Research Projects: A Practical Guide, published in 2009 by the OECD Global Science Forum (http://www.oecd.org/dataoecd/37/17/40188303.pdf),
Falende Wetenschap: De frauduleuze onderzoekspraktijken van sociaal-psycholoog Diederik Stapel (Flawed Science: The fraudulent research practices of social psychologist Diederik Stapel), published in 2012 (2012 Stapel Investigation, powered by WordPress RSS).
The title of this paper speaks about institutional responses to infringements of the norms of responsible research. Quite a few institutes are involved in the research enterprise, and we will briefly analyse such responses for three categories of institutes: Universities and research institutes, funding agencies, and Academies of Sciences. There is a fourth category, journals and publishers, but these are sufficiently discussed in other contributions to this symposium, I presume. Each of these institutions bears specific responsibilities in establishing and fostering standards of responsible conduct in research.
The responses to violations of research integrity depend, of course, on the nature of the misconduct. The most serious violations are fabrication (making up results and recording or reporting them) and falsification (manipulating research processes or changing or omitting data). Another type of misconduct is plagiarism, the appropriation of another person’s ideas, research results or words/figures/tables without giving appropriate credit. The European Code of Conduct also classified improper dealing with such violations (attempts to cover up, reprisals to whistle-blowers, violations of due process) as misconduct. A last category of unacceptable violations are minor misdemeanours, such as some ‘adjustment’ of data, omitting or changing one or two unwelcome observations, summarising incorrectly, cutting a corner here and there…. It should be made clear that here we also deal with unacceptable misconduct. It is falsification in statu nascendi, and may lead to more serious infringements if it is not corrected, as the case of Diederik Stapel shows.
Responses have to depend further on the seriousness of the misconduct. As the European CoC states, the level of intent, the consequences of the misconduct, and other aggravating or mitigating factors should be taken into account, and ‘it has to be shown that the misconduct was committed intentionally, knowingly, or recklessly[…]. It should be stipulated that misconduct does not include honest errors or differences in opinion’.
As indicated we will first pay attention to the three defined institutes: Universities/Research institutes, Funding organisations, and Academies of Sciences. Then we will discuss some specific issues with respect to violations of research integrity: the requisite to investigate the whole oevre of a fraudulent author, the admissibility of statistical ‘proof’ of fraud, and the annulment of a granted degree of an impostor.
Universities and Research Institutes.
The primary responsibility for creating and maintaining an environment that fosters responsible conduct and good practices, and, consequently, for the handling of cases of misconduct lies in the hands of the employer of the researcher: the university or the research institute. They have to create a climate in which the fundamental values of research are practiced and violations thereof are not accepted. They do this by means of the following practices and measures:
The acceptance and publication of a code of conduct, a set of good practices, and well defined procedures in cases of misconduct.
Fostering research integrity within the academic community through educating, training and mentoring; awareness and acknowledgement of the principles of responsible research should be furthered through teaching and mentoring of the students and junior staff, and through setting an example in one’s own research. Responsible conduct should be a mandatory subject in courses on research methodology and experimentation. Students and staff members should recognise that they have a duty to take action in case they observe or suspect serious violations of the rules of responsible research.
The establishment of a number of supporting conditions and regulations, including clear mechanisms for reporting misconduct, the appointment of integrity officers or ombudsmen and integrity committees, the protection of whistle-blowers from retaliation in their work or further career, the use of electronic plagiarism detection systems, the requirement to establish a system of data storage and archiving, while maintaining the accessibility of the data. Universities can ask from their employees at their appointment to sign a statement that they have taken cognizance of the code of conduct and will obey the current standards of responsible research. The Erasmus University Rotterdam has started an interesting experiment to ask all students to sign at their the formal graduation a ‘pledge’ to practice and apply responsible science.
Impartial and confidential mechanisms to investigate suspected cases of misconduct by a formally existing or ad hoc appointed investigation committee. Investigations should follow a clearly defined procedure and must be conducted confidentially, fairly, comprehensively and expediently, and should take place as quickly as possible. There should be a commitment to there being no detriment. Appropriate action should be taken against persons found to have breached the norms, ranging from a formal reprimand to immediate dismissal. The minor misdemeanours, described above, do not necessarily have to lead to a formal dispatch through an investigation committee. At least if it occurs with students or junior staff it can be corrected through supervision and mentorship. But if it occurs with senior staff, and certainly repeatedly, a more serious treatment is called for.
Emphasis on transparency, sharing and discussing findings with colleagues and research group members, and collegial control. Data collection and interpretation should not be an isolated, individual, uncontrolled activity. Supervisors, members of doctoral examination committees, and referees should carefully investigate and check all phases of the research project. Co-authors are fully responsible for the whole publication, unless a clear indication of a partial responsibility (and which part) is formulated.
Fraudulent publications should be retracted. It should be made known to the scientific community that these no longer can be quoted or used as background material. Universities and research institutes should insist that scientific journals formally retract the articles, preferably with mentioning the reason for the retraction.
The alleviation of an emphasis on metrics (number of publications, citation index, number of citations in high impact journals, H-factor, etc.) in the incentive systems and promotion policy, and the search for additional ways of quality measurement. It is not unlikely that the ‘publish or perish’ culture has contributed to the prevalence of misconduct. Dysfunctional craving for ‘high scores’ leads to behaviour that crosses the limits of what is admissible.
Funding agencies, be they private or public, have important responsibilities in the promotion of responsible conduct as well. They provide the, often badly needed, research funds that enable the researchers to make a contribution to the development of their field. They can, therefore, also insist on responsible research practices and make such practices conditional for providing the grants.
In most European countries the actual power and opportunities of funding agencies to investigate possible breaches of norms and to exert control on the integrity of the research are rather limited. This differs from for instance the USA where the Inspection Office (NSF) and the Research Integrity Office (NIH) do have search and executive powers. In the Netherlands for instance the national research council (NOW) carefully sees that no conflict of interests or lack of independence occurs within its staff or within the members of the peer review committees, but further presumes that the subjected proposals are honest and that the progress or final reports are based on proper and responsible research. NWO considers the integrity control to be a responsibility of the universities or research institutes (Dekker, 2010).
Nevertheless they do have some instruments and means for fostering responsible research at their disposal, including:
Reserving part of their funds for research into the nature and prevalence of research misconduct. Still too little is known about its occurrence, about the driving motives, and mechanisms to detect and to prevent this phenomenon;
Increasing the alertness and sensitivity of reviewers for possible violations of research integrity. Indications could be: results are ‘too good to be true’, unlikely high correlations, insufficient sample definitions, and the like;
To provide funding only to researchers who work in an environment where it is shown that responsible research practices are warranted (proper infrastructure and procedures for fostering research integrity);
Requirement for the applicant to store the original data in an accessible manner, in order for colleagues to be able to carry out replication or verification studies;
To support more replication and verification studies, and not only new, innovative and revolutionary research, as is often the case at present.
Insistence to publish also negative outcomes, so as to avoid publication bias (Fanelli, 2012).
Academies of Sciences and Interacademy Organisations
Academies and interacademy organisations have also a responsibility in the promotion of responsible research conduct. They should reflect on the basic norms and standards in science and scholarship, study the nature, prevalence and causes of breaching these norms. Most Academies have a Standing Committee on Science and Ethics with an advisory function on ethical issues vis-à-vis the Board of the Academy, which can develop and prepare an Academy policy on research integrity. Academies should help to develop standards and guidelines for research integrity, and to disseminate those guidelines nationwide. They would do right to develop this policy in dialogue with other players in the field, such as (the association of) universities and the national research council. They should also participate, again together with the same actors, in a national council for research integrity that could act as an advisory high court of appeal for difficult and controversial cases.
Academies should not act as investigating bodies with the task to examine particular cases of alleged misconduct, nor as a decisive court of appeal where either party (accuser or accused) can lodge an appeal against a decision made by their regular employer. Academies are not equipped nor do they have the legal authority for such a task. On the other hand, they can serve in a consulting and advisory role for universities or research institutes in cases that are difficult and complicated.
Some European Academies, such as the Austrian, the Dutch and quite a number of Central and Eastern European Academies, also superintend, sometimes quite some, top research institutes. In that capacity they bear, of course, the normal employer’s responsibility for ensuring a prevailing culture of research integrity, and for dealing properly with allegations of misconduct.
We have just described the task and contribution of national Academies operating in a national context. At a transnational level (Europe, Middle East, Asia, Americas) analogous roles could be played by the respective Associations of Academies, such as for instance All European Academies (ALLEA) should do for Europe.
In cases of demonstrated fraud committees of enquiry usually restrict themselves to an investigation of articles reported to be suspect or to a small sample of the published work of the impostor. In many cases this can indeed deliver sufficient proof of fraud. This may be sufficient if it seems likely that we deal with an incidental case of for instance plagiarism or manipulation of data.
But if there is reason to believe that the accused author has breached the standards more often, and that we may deal with a pattern of misdemeanours, there is a greater interest at stake. The fundamental cumulative process of building and validating theories in science through careful empirical testing is seriously disrupted if fraudulent data come into play. Time and money may be wasted on replication studies and meta-analyses. Harmful applications may be derived (think of medicine or engineering). In such cases the university or research institute has the responsibility to scrutinise the whole of the fraud’s body of scientific work, including all his/her own publications, the co-authored publications, and the dissertations completed under his/her supervision.
The three Committees of enquiry that investigated the extensive fraud of the Dutch social psychologist Diederik Stapel did work along these lines and investigated his whole oevre. They carried out re-analyses and statistical tests on the reported or available data in all his publications and in all dissertations of his PhD students. As a results the committees concluded to proven fraud in 55 of the 137 publications examined, a high probability of fraud in another 10 publications, and to fraud in 10 out of the 18 dissertations of his promoti. Granted, this was a very labour-intensive and costly operation, but the universities owe this to themselves and to the whole scientific community.
The Drenth-Committee, that had to examine the possible fraud of Diederik Stapel during his Amsterdam period (1995-2000) was faced with two difficulties: In the first place all original research data were lost (understandably after more than 12 years), and secondly Stapel did not ‘remember having violated any norms of responsible research in that period’. Consequently, this Committee’s analysis had to focus exclusively on suspected irregularities and statistically highly implausible results. A number of such irregularity indicators have been used. But the most important one was handed in by Chris Klaassen, professor of Statistics and member of the Drenth-Committee. He developed a Baysian formula, indicating the probability of the composition and distribution of the reported data in the observed and control groups being manipulated by comparing these with the composition and distribution of such data if the experimental and control groups had been selected according to scientific principles. The quotient is then an indication of ‘proof of manipulation’.
Using these indications all Amsterdam publications of Stapel were judged on the probability of fraud on the following scale: not applicable – none – negligible – slight – relatively strong – strong. The last two categories have been taken as sufficient ground for the judgement ‘evidence of fraud’. It has to be pointed out, however, that this ‘evidence’ implies a high degree of probability from a statistical perspective, but does not constitute legal proof of fraud.
I want to call attention to the wide applicability of the Baysian formula. For any committee of enquiry, board of a faculty or university, or editorial board of a journal, that has to deal with an allegation of fraud, but with no verification data available, nor with a confession of the accused author, this formula could prove helpful in reaching a conclusion.
Annulment of a granted degree
A last and for universities interesting issue concerns the question whether a degree can be rendered void as a result of established fraud. Such an action was brought in in the case of the physicist Hendrik Schön. Also in the case of Diederik Stapel the Committee Drenth was asked to examine the studies on which his PhD dissertation was based, and, in case proof of fraud would be furnished, to advise the College of Deans of the University of Amsterdam on the question whether his degree could be invalidated. In a way Stapel made it easy for the committee since he surrendered his doctoral certificate of his own accord.
However, the question itself retains its relevance. It seems that under the legislation of many countries a degree can be nullified if there is proof that in the research on which the dissertation was based serious infringements of research integrity have taken place. In the Netherlands the College of Deans that granted the degree can also reconsider this decision and retract the degree. It is not certain, however, whether the judge, if the case were to be brought to court, would accept the above described ‘statistical evidence’.
Legislative changes would seem necessary to withdraw degrees if the instigation for such a withdrawal would result from misconduct perpetrated after the conferral of the degree. Under the present Dutch law this would be impossible. However, opinions vary also on the principle. A PhD is granted because the doctorandus has shown to be able to design, carry out and report independently a piece of research that has met with the doctoral examination committee’s approval. Some people seriously oppose the idea that this can be reconsidered because of later committed violations of norms of integrity.
In this paper a number of measures and actions of the responsible academic institutions in dealing with infringements of the norms of responsible research have passed in review. Not always is there one best approach. Some questions remain unanswered. Sometimes practical or legal obstacles hinder a desired response. We hope, however, that discussions like the one to-day contribute to a greater awareness and a sense of urgency to develop policies and strategies how to deal with, and, more importantly, how to prevent the unethical and harmful violations of the principles of research integrity.
Dekker, R.J.P. (2010), Wetenschappelijke integriteit bezien vanuit de onderzoeksfinancier (Scientific integrity seen from the perspective of a funding organisation). In: P.J.D.Drenth (ed.), Wetenschappelijke integriteit (Scientific integrity). Amsterdam: KNAW Press, pp 45-50.
ESF/ALLEA (2011), The European Code of Conduct for Research Integrity, ESF (www.esf.org) ALLEA (www.allea.org),
Fanelli, D. (2012), Negative results are disappearing from most disciplines and countries. Scientometrics, 90, 891-904.
InterAcademy Council (IAC)/InterAcademy Panel (IAP) (2012), Responsible Conduct in the Global Research Enterprise.www.interacademycouncil.net/ http://www.interacademies.net, ISBN9789069846453.
Levelt, W.J.M., Drenth, P.J.D. & Noort, E. (2012), Falende Wetenschap: De frauduleuze onderzoekspraktijken van sociaal-psycholoog Diederik Stapel (Flawed Science: The fraudulent research practices of social psychologist Diederik Stapel, WordPress RSS). Report Tilburg University, University of Amsterdam, University of Groningen.
OECD Global Science Forum (2009), Investigating Research Misconduct Allegations in International Collaborative Research Projects: A Practical Guide, (http://www.oecd.org/dataoecd/37/17/40188303.pdf),