SORTEE two-year anniversary: our members’ voices

By SORTEE | December 2, 2022

Between June and September 2021, we started the #SORTEEvoices blog series by asking inaugural members to choose a few questions to answer from a list of 30 options (15 questions about open science, reproducibility, and transparency; 15 miscellaneous questions)1. Responses from sixty-four inaugural members were posted on our blog every week until October 2022. To celebrate SORTEE’s 2-year anniversary in December 2022, we’d like to look back and summarise our members' voices.

Who answered?

Most members who answered our questions were academic researchers: 38% PIs (professors, researchers, lecturers), 33% postdocs, and 20% graduate students. PIs were overrepresented in this blog series compared to our full list of members which skews more towards early-career researchers (in 2021 and 2022). We also had a couple of members of NGOs and of research support staff.

Which open research questions were the most answered?

The answers to the four most answered questions provided inspiring stories of personal paths toward the adoption of open research practices, as well as some interesting perspectives on areas for improvement in members' research fields.

How did you become interested in open research?

Many members felt lucky to have been exposed to the subject early on (team, mentor, teacher), or to have met like-minded people. At SORTEE, we work to create a welcoming environment for exchanges and to give a sense of community to our members, including through our conference and our Slack workspace. We hope to continue to foster these connections for many more of our members going forward.

Many members mentioned particular events that made them start caring about open research. For example, several shared their experience of not being able to access research outputs (e.g. articles, data, or code) that they needed for their work. Some lost access after moving out of academia (e.g. to work at NGOs), while others had limited access to articles if they worked at a non-wealthy institution. Several members also reported struggling to replicate others' findings, or to publish null results, and a couple of members even witnessed cases of scientific misconduct. SORTEE’s main goal is to promote higher standards of good research practices in the fields of ecology and evolutionary biology, by providing training and building a community of practice, notably through our webinar and workshops series and our participatory events during our conference.

A few other members mentioned they read about the replicability crisis in other fields (e.g. psychology or medicine), came into the open research movement from a philosophy of science background, or realized more generally the current academic research culture and its incentive structure is broken. Finally, one person mentioned the philosophical implication of meta-analyses. With SORTEE, we plan on engaging in advocacy work with stakeholders across the sector to reform the design of incentives and policies; we have the ambition to similarly inspire other fields to join the reform movement; and overall, we strive to increase the impact of research by making outputs open and reusable for all.

What open research practice did you adopt that you found most useful?

The winning practice in terms of usefulness for the adoptees was version control (for code and sometimes data), for instance through Git and GitHub. Our members also praised the usefulness of the Open Science Framework (OSF) to share their research outputs. Finally, some members highlighted some specific programming skills (e.g. RMarkdown) to increase the comprehensibility of their codes for others, the use of preprints as a way to share their articles in an open access format, and the adoption of preregistration to carefully plan their studies prior to conducting them.

With SORTEE’s webinar and workshop series, we will strive to raise awareness and teach these skills that were found most useful!

If you had the power to change one thing in your field, what would it be?

One member wished for more and better statisticians to be involved through the lifetime of projects, and another wished that journals across the board would implement reporting guidelines for authors to share statistical information so as to be reusable in secondary analyses such as evidence synthesis.

All the other members who chose to answer this question deplored the current academic research culture and particularly the current ‘publish or perish’ reward system. Members expressed their desire to perform careful, nuanced research as opposed to racing to publish first. A shift towards publishing more Registered Reports could help researchers be rewarded for the robustness of the experimental design and quality of data. Several wished to further reform the publishing system, for instance using a GitHub-like system with community discussions, to continuously track and log updates of research outputs and document how results change as more data or better methods become available.

Which open research practice do you think deserves more attention?

There were mainly two categories of open research practices that were thought to deserve more attention.

  1. Code and data sharing such that they are discoverable, checkable, and reusable. One member also acknowledged that this requires rewarding the production of these additional research outputs.

  2. Pre-registration, particularly in the form of Registered Reports, to eliminate (conscious or unconscious) p-hacking, confirmation bias, HARKing, selective attention bias, publication bias, and incentivise addressing interesting questions with rigorous methodology as opposed to incentivising finding sexy-results.

Again, with SORTEE’s webinar and workshop series, we will strive to raise awareness of these practices deserving of attention!

The most answered personal question was: Where were you born and raised?

This showcases some of the diversity of background within our community, an aspect we take pride in!

drawing of a flying bird (perhaps an albatross?) carrying all the names of the countries of origin of the members who contributed a blog post and answered that question. The size of the country’s name defines its frequency; in order: UK, USA; then Chile, Colombia; then Australia, Brazil, Canada, Croatia, France, India, Iraq, Netherlands, Nigeria, Norway, South Africa, and Spain
 

Given the interest in these blog posts and personal stories, we are launching a second wave of ‘interviews’ with our newest members! Choose from a pool of new and old questions to respond to and contribute your thoughts on open science/reproducible research and/or tell us more about yourself! Take a few minutes to fill out the #SORTEEvoices form.

You can become a member here: https://www.sortee.org/join/. Find all our interviews at https://www.sortee.org/tags/member-voices/.


Acknowledgements

Summary blog writer: Malika Ihle

Q&A survey distributor: Kaija Gahm

Blog post editor: Colin Strine and Rose O’Dea

Blog series instigator, maintainer, and advertiser: Rose O’Dea


  1. Pool of questions to pick from in the first round of interviews:

    Open research questions:

    How did you become interested in open research?

    What’s an open science practice or topic that you’ve changed your views on within the last few years? Why?

    What ‘ORT’ practice have you introduced into your research practice that you’ve found really helpful?

    What ‘ORT’ behaviour have you or your students found difficult to change?

    Do you have ideas about how this could be made easier?

    If you had the power to change one thing about current research practices in your field, what would it be?

    If you had the power to change one thing about current incentives in your career path, what would it be?

    What questions in your field do you really want to have reliable answers to?

    What’s an ‘ORT’ subject or practice that you think deserves more attention?

    What strategies/approaches do you think are most likely to lead to a research culture change?

    Do you see any downsides to some open research practices?

    What institutional policies do you see as most important to change to improve the reliability of science? (‘institution’ broadly defined including funders, journals, universities, etc.)

    Tell us about a paper that influenced your thinking on an issue of open / reliable / transparent research.

    If you could recommend one paper to the community of ecologists and evolutionary biologists on an openness / reliability / transparency topic, what paper would you choose?

    What do you see as the greatest challenge facing the open / reliable / transparent science movement at large or specifically in ecology and evolutionary biology? What is an open / reliable / transparent science practice that you admire but have not yet adopted in your own work?

    Personal questions:

    Where were you born and raised?

    What’s the last book that affected the way you currently think about things?

    What was your worst kitchen disaster?

    Where is someplace you’ve never visited but would like to?

    Why do you want to visit there?

    If you could start your career over again, but had to switch to another field or subdiscipline, what would you switch to? Why?

    Tell us about one of your hobbies.

    What do you now know about the way science gets done that you would have found surprising before you started your training as a scientist?

    Do you have a favorite non-human organism?

    What is it and why is it your favorite?

    What is something about you that your scientist colleagues might find surprising?

    What is something about your work in science that your friends outside of science (or the general public) would find surprising?

    What was your most embarrassing field work moment?

    What was your most embarrassing lab work moment?

    What was your most embarrassing coding moment?

    Tell us about a teacher who helped you enjoy learning Ask and answer your own question!

    With thanks to 2021 SORTEE Committee Members for drafting these questions. ↩︎