View all news

How could research assessment recognise the ongoing impact of the pandemic on researchers?

6 December 2022

The immediate impacts of the COVID-19 pandemic on researchers and scientists have been regularly explored during the most extreme period of the pandemic. As well as responding to the pandemic with research that helped the immediate effort, researchers also sought to understand the impact of the pandemic on research and researchers themselves. This included work to understand disruption to researchers’ work, wellbeing, work-life balance and the nature and frequency of scientific outputs.

We now know that the negative impacts of the pandemic on researchers have been unequally distributed, with early career, women, caregivers, and minority ethnic colleagues worst affected. Here at the Elizabeth Blackwell Institute, we are acutely aware of these impacts too, because we had asked colleagues what the most challenging barriers to their ongoing research projects were. We learned that disruption to data collection and analysis, decreased access to facilities and research participants, and decreased contact with colleagues and mentors meant that researchers needed more time and financial support for their careers to outlast this difficult period. In response, we launched our COVID-19 support scheme to help researchers to extend the duration of their study to weather the most intense period of the pandemic.

But we continue to grapple with more uncertainties:

What are the impacts on research colleagues that have stretched into 2022 and which are bound to continue in 2023 and beyond? How can we be confident that the return to ‘business as usual’ is not masking or exacerbating long-term consequences of the pandemic on researchers and their careers? What empirical evidence is out there now that we can learn from? How can we, as a research funder, ensure that these impacts are acknowledged in funding decisions that we make now and in the future?

Research that is being conducted now and which directly answers these questions is sorely needed. Studies from 2021 have shown decreases in starting brand new research projects, new publications, and new co-authorships, despite signs of returning to pre-pandemic productivity levels (Gao et al., 2021[1]). We also know that researchers unable to pivot their projects towards pandemic-related research have been particularly affected, as some journals prioritised COVID-19 science at the expense of non-COVID-19-related outputs (Cosentino et al., 2022[2]). Not surprisingly, a drop in fresh initiatives was sharpest among women and individuals with young children. Having already experienced more challenging disruptions at the start of the pandemic, these groups are now facing a double jeopardy with output and collaboration gaps as long-term effects of the pandemic and lockdowns continue to make their presence felt.

The recent Agreement on Reforming Research Assessment[3] signed by UKRI[4] and over 150 signatories from 30 countries does not explicitly recommend that funders heed the potential long-term effects of the pandemic. Helpfully, the Agreement takes a broad view and states that ‘[r]eformed practices for assessing individual researchers for the purposes of recruitment or career evaluation should consider their individual contexts and careers’ (page 8). Individual contexts could of course reasonably include the circumstances relating to the pandemic as well as many other matters. We feel that a continuous commitment to enabling researchers to provide narratives to contextualise their research careers and outputs is urgently needed, especially in the face of the ongoing and largely unknown enduring effects on careers. To attend to this challenge, we will continue to include a voluntary ‘special circumstances’ statement in the Institute’s funding application forms.

The ‘special circumstances’ section is designed to enable applicants to let peer reviewers and panel members know about the impact of adverse personal events on research trajectories and outputs. We ask applicants to focus on elaborating about the impacts, rather than sharing personal details of the circumstances. These impacts can result from illness, family circumstances, or care responsibilities. To ensure that the practice is effective, fair, and fit for purpose, we have set out to evaluate the usefulness of the ‘special circumstances’ statement by reviewing the available data we hold and asking applicants and review panellist to share their perceptions and experiences to date. Having reviewed the last three years of the Institute’s funding schemes, we are encouraged that typically 1 in 4 applicants have made use of this space. Most applicants were satisfied the Elizabeth Blackwell Institute provided a clear section in the form to include this type of information. Given the available empirical evidence on unequal impacts on female researchers mentioned earlier, we understand why women might have been more likely than men to report special circumstances in the application form.

However, more work needs to be done to ensure that our review process guidelines are clearer and more transparent. We would like to use this opportunity to share with colleagues, funding decision makers, reviewers, and the broader research ecosystem our preliminary guidance we are developing to inform this process to nurture equality, diversity, and inclusion. This evaluation exercise has highlighted that both applicants and reviewers desire further guidance to improve the practice of declaring and handling information relating to special circumstances of researchers.

How to help your Applicants?

Applicants may think that by disclosing their personal circumstances they are making themselves vulnerable which can cause discomfort and apprehension. We recommend the following actions to adapt this practice to both protect and inform applicants:

  • Enable and encourage ‘getting to know’ the institution, its people, its processes, and past and current fellows and supervisors for integrity, transparency, and trustworthiness
  • Explain why you are inviting applicants to declare ‘special circumstances’ to reassure applicants the aim is to nurture equality and inclusion, and minimise bias
  • Describe what information to include while declaring ‘special circumstances’,  include examples to clarify what is/isn't expected. Encourage openness without oversharing
  • Inform who will view and review and have access to the form
  • Inform how long the application with the 'special circumstances' will be stored for
  • Explain how the information will be used for a fair evaluation

How to help your Reviewers?

Reviewers need to be guided on how to review and evaluate these data to inform the evaluation of applications in a fair and unbiased way. Reviewers are not as personally invested as applicants, however, reviewing ‘special circumstances’ can also bring some discomfort and worry for reviewers too. If reviewers feel that evaluation does not take ‘special circumstances’ into account as it should, this may also influence whether the review processes are perceived to be fair and trustworthy. We recommend the following actions to adapt this practice to guide funding application reviewers:

  • Reassure reviewers that applicants have been provided with clear and transparent guidelines
  • Explain why applicants are invited to declare special circumstances
  • Reassure that reviewers can only review what is declared and avoid second guessing someone’s circumstances
  • Provide simple but detailed guidance on how to evaluate applications with ‘special circumstances’
  • Decide how ‘special circumstances’ declared during interview, but not in the application should be treated: should information provided orally be treated as equally valid?
  • Emphasise that reviewers do not rank ‘special circumstances’ but use it to contextualise research outputs and inform the bigger picture
  • Remind reviewers of the confidentiality and GDPR when handling ‘special circumstances’ information
  • Require all reviewers to have completed implicit bias training in recruitment and selection

[1] https://www.nature.com/articles/s41467-021-26428-z

[2] https://link.springer.com/article/10.1007/s11606-022-07647-6

[3] https://coara.eu/app/uploads/2022/09/2022_07_19_rra_agreement_final.pdf

[4] https://www.ukri.org/news/ukri-signs-agreement-on-reforming-research-assessment/

Further information

If you would like to connect about or comment on this initiative, please contact Elizabeth Blackwell Institute's Research Associate in Equality, Diversity, and Inclusion: ola.thomson@bristol.ac.uk

Edit this page