Please find the updated FORRT Website here where you can find what we are up to these days. Check out the self-assessment learning tool, FORRT’s Manuscript, FORRT’s curated resources, and FORRT Code of Conduct.
Current norms for the teaching and mentoring of higher education are rooted in anachronistic practices of bygone eras. Improving the transparency and rigor of science is the responsibility of all who engage in it. Ongoing attempts to improve research credibility have, however, neglected an essential aspect of the academic cycle: the training of researchers and consumers of research. Principled teaching and mentoring involve imparting students with an understanding of research findings in light of probabilistic uncertainty, and moreover, an understanding of best practices in knowledge advancement. We introduce a Framework for Open and Reproducible Research Training (FORRT). Its main goal is to provide educators with a pathway towards the incremental adoption of principled teaching and mentoring practices, including open and reproducible research. FORRT will act as a resource to support instructors, collating existing teaching pedagogies and materials that may be reused and adapted for use within existing courses. Moreover, FORRT can be used as a tool to benchmark the current level of training students receive across six clusters of open and reproducible research practices: ‘reproducibility and replicability knowledge’, ‘conceptual and statistical knowledge’, ‘reproducible analyses’, ‘preregistration’, ‘open data and materials’, and ‘replication research’. FORRT will strive to be an advocate for the establishment of principled teaching and mentorship as a fourth pillar of a true scientific utopia.
Above is the abstract for our manuscript introducing FORRT https://osf.io/bnh7p. We welcome input on FORRT itself, and on the manuscript; you can access the working document here (please select suggested changes): tinyURL.com/FORRTworkingDOC.
We are actively seeking help to develop FORRT into the pedagogical resource we envisage. If you would like to help us support and advocate for teachers, please get in touch here. Please also see the ‘implementation plan’ tab for more information on our current plans.
FORRT is centred around a framework comprising six clusters of open and reproducible research practices. Each cluster has six sub-clusters.
Summary: Attainment of a grounding in the motivations and theoretical underpinnings of reproducible and open research. Integration with field specific content (i.e., or grounded in the history of replicability);
Summary: Enacting this principle indicates that students attain a grounding in fundamental statistics, measurement, and its implications.
Summary: Reproducible analyses allow the checking of analytic pipelines and facilitate error correction. Enacting this principle requires students to move towards transparent and scripted analysis practices.
Summary: Enacting this principle indicates that students have attained a grounding in open data and materials in both; using and sharing.
Summary: Preregistration entails laying out a complete methodology and analysis before a study has been undertaken. This facilitates transparency and removes several potential QRPs.
Summary: Replication research takes a variety of forms, each with a different purpose and contribution. Reproducible science requires replication research.
Although not exhaustive, these concepts provide a broad coverage of each FORRT cluster.
Our developing implementation plan is as follows (we are also open to feedback and suggestions):
We are actively seeking help to develop FORRT into the pedagogical resource we envisage. If you would like to help us support and advocate for teachers, please get in touch here.
Authors note. This project was initiated at the 2018 meeting of the Society for the Improvement of Psychological Science in the “Teaching replicable and reproducible science” hackathon led by Kristen Lane and Heather Urry. The initial framework was developed in a subsequent working group consisting of: Sam Parsons, Flavio Azevedo, Carl Michael Galang, Kristin Lane, Lisa DeBruine, Benjamin Le, Donald Tellinghuisen, and Madeline Harms (we apologise if we have missed anybody - please let us know). The current version of the paper introducing FORRT was drafted by Sam Parsons & Flavio Azevedo, with integral feedback and support from Carl Michael Galang. The FORRT website has been designed - and is maintained - by Flavio Azevedo. To further develop FORRT we are seeking additional contributors and will endeavor to acknowledge all that provided feedback on this project.↩︎