Evidence on the effects of flame retardant substances at ecologically relevant endpoints: an open systematic map protocol

By Prof Kathryn Arnold, Professor of Ecology, Department of Environment and Geography

Summary

Open research practices and principles have been embedded into the research training of the York-led Centre for Doctoral Training programme ECORISC. Our output provides an example of how Lowenna Jones, a PGR based at Sheffield, and co-trained and co-supervised through York, has developed a protocol for a systematic scoping review paper which has been openly and transparently developed via the journal and its novel review process. This initiative is a useful case study and raises some pertinent issues around the nature of how open research is produced for PhD theses and how this is presented for assessment.

Case Study

The primary objective of this initiative was to develop and publish an openly accessible protocol for a systematic evidence map (SEM) on the ecologically relevant effects of flame retardant substances. SEMs are an emerging and underutilised tool in chemical risk assessment and can reliably collate and characterise a large body of existing evidence. The main output of an SEM is a freely available, coded and interrogable database which end users (e.g. decision makers, researchers, risk assessors) can explore and extract data of interest. This data could be used to inform future research (e.g. hazard assessment) or synthesis (e.g. meta analysis), inform regulatory action (e.g. restriction) or aid the development of evidence based methods in chemical risk assessment. The protocol serves to document the work in advance of synthesis and has been prepared in accordance with the Reporting Standards for Systematic Evidence Syntheses.

Lowenna and I used open and participatory research methods to engage and canvas feedback from technical experts on the research objectives and study design. Technical experts (n=63) were asked to read and comment on any aspect of the first draft of the protocol (openly available on OSF) and to specifically give feedback on the research objectives (section 1.4), information sources (section 2.2) and eligibility criteria listed in table 3. Effort was made to ensure diversity in level of expertise, institution, gender and ethnicity to encompass a range of global perspectives and representation of technical experts, however this was hard to achieve with less than 20% (n=10) of experts providing feedback. We consider the feedback provided by the technical experts as invaluable for the development of the protocol. For example, it meant that our research will be useful and relevant to regulators, industry and researchers.

Screenshot of the openly published article

The second open research practice that we embedded was to participate in new open publication and peer review models by publishing in a journal (Evidence Based Toxicology [EBT]). This journal uses open peer review and requires the authors to pre-register a study design on Zenodo (a repository for European Commission funded research). This ensured fully accessible, documented and traceable version control of both the protocol and reviewer comments. Publishing open access research can be a significant cost for researchers and to overcome this challenge we discussed our initiative with the Editor-in-Chief of EBT who offered to waive the Article Processing Charges if the protocol was submitted within a period of 3 months.

Publishing this initiative in EBT allowed for our research outputs (i.e. protocol, tables and figures, supplementary material, etc.) to be published using open licenses on the Open Science Framework. Embedding this open research practice in our initiative allowed for broad dissemination of our research, whilst also supporting the FAIR principles on data management and stewardship. The FAIR principles were engaged throughout this initiative with persistent identifiers (DOIs) included for any resources mentioned in the protocol, as well as detailed and transparent reporting of the methodology to allow for external replicability. The CReDIT contributor roles taxonomy was also used to ensure detailed information on individual contributions to the work was documented and shared. At times, this new approach to co-creation, review and open sharing of everything seemed slow and a lot of work, but ultimately resulted in a much better protocol than our original one, and is fully transparent and accessible to the research community.

This paper will form a chapter of Lowenna's PhD thesis and we will ensure that the contributions of the participants, technical experts, open peer reviewers and editors are acknowledged in the development of this protocol. We hope to work with the Open Research team and York Graduate Research School to develop some standard statements for inclusion in PhD theses and guidance around declaring the contributions of other researchers in the production of materials submitted for assessment.

Other contributors

Lowenna Jones - PhD researcher at Sheffield, co-supervised by Kathryn Arnold at York and funded via a York-led Centre for Doctoral Training (ECORISC)

Oliver Allchin - Librarian at University of Sheffield

Links

Open access protocol: https://www.tandfonline.com/doi/full/10.1080/2833373X.2024.2375113#abstract
Open Science Framework: https://osf.io/uszfh/?view_only=128383c0c7e94526ad1190a8d18c83b1
Evaluation reports: https://zenodo.org/records/12572605

Funding

This work was supported by the Natural Environment Research Council (NERC). The funder had no input into the designing of this protocol.