Although difficult to define, descriptive summaries change the way we think about researcher assessment.
For their proponents, descriptive academic summaries are a way to circumvent aspects of a research assessment culture that is overly focused on the scope and location of publications. Based on a sample of work promoting this format, Frederic Bordignon, Laurenne Chegnon And Daniel Egret, to show how these texts are more likely to highlight the problems they are intended to solve than how the format will work in practice. However, at the same time, they argue that the emergence of this new format and terminology is reopening the debate about the role that academic assessment can play in promoting better research culture and practice.
In the academic world, a resume is a common document that combines a list of scientific achievements, a summary of professional experience, an administrative requirement, and a resource for finding a job (see below). Cañibano and Bozeman). While the summary feature is common and public, the document itself is known to exist in a variety of formats and has evolved over time.
For several years now, various organizations have been recommending the use of descriptive summaries to improve researcher evaluation methods. The academic community is increasingly concerned about bias in research evaluation, especially in the evaluation of researchers. They are critical of the overuse of bibliometric indicators to the detriment of the public impact of research. This concern is reflected in a number of initiatives such as DORA declaration, Leiden Manifesto, metric tide report, and more recently work initiated rudea coalition of organizations created as a result of the European Open Science Conference 2022 and Paris Call for Exploratory Evaluation.
Alongside these initiatives, a number of recommendations were made, including the use of descriptive summaries. But what is a narrative summary? Since there is not much research on this type of document, we relied on what its promoters say about it, as well as feedback from various organizations that have made it mandatory for their recruitment processes. We identified 28 voices from advocacy coalitions, universities, donors, learned societies, governmental or intergovernmental sources, and the media. These documents mention the scientists’ point of view, including quotes collected through interviews for newspapers or blogs. We spent careful analysis of these documents reveal the main principles behind the narrative summary, as well as the problems it is designed to address, including in real situations with a review of 7 experiments in different countries. As a result, we identified the following frequently cited features of the narrative summary: against the misuse of metrics, listing, and narrow definition of impact, and in favor of a broader range of research input, contextualization, inclusiveness, and diversification.
Against the misuse of metrics
The most common argument for developing narrative summaries is against the use of bibliometric indicators to evaluate researchers. They are criticized for prioritizing quantity over quality and increasing pressure on researchers to publish in reputable journals when other publishing methods could have a greater impact on society. However, it is clear from the pilot projects that the metrics have not been completely abandoned. For example, NWO (Dutch Research Council) banned the use hour-index or impact factor. However, he allowed the use of metrics, provided that it is stated why the metric is of interest in the context in question.
Against Lists
Traditional resumes are criticized for preferring “labels”, which leads to “hasty judgment” and disadvantages, for example, researchers whose careers have been cut short, leaving unexplained gaps in their resumes. But not all organizations that require a descriptive summary have completely abandoned the convenient presentation of it in the form of lists. The Swiss National Science Foundation has developed resume template based on both lists and free text. The same applies to the University of Glasgow, which stands for hybrid resumewhich seems to be the format preferred by users.
Against a narrow definition of influence and in favor of a wider range of scientific research
For most proponents of the narrative summary, its main advantage is that it expands the range of contributions what researchers are recognized for, i.e. going beyond publications and taking into account other “contributions to the real world” activities, such as projects that help local communities, participation in committees, training and mentoring.
In favor of contextualization and selection
A descriptive summary is a way of providing contextual elements to give a “much richer and more detailed picture of an individual scientist’s contribution”, taking into account the specifics of the discipline, academic age, as well as personal circumstances. Some organizations advocate choosing the most relevant publications rather than listing them all, which gives assessors a chance to read them. However, the refusal to complete the list of publications met with a mixed reaction from both applicants and experts, since Luxembourg National Research Foundation learned during his pilot project. Some fear that they will only have an incomplete profile of the candidate and that the examiners will not have the information they need to verify what is stated in the descriptive sections.
In favor of inclusiveness and diversification
A descriptive resume allows you to vary the evaluation criteria and, for example, value mentorship or participation in committees. The singling out of this activity works to the benefit of women, ethnic minorities and other underrepresented demographics as these have been shown to be areas in which they surpass.
Return to the old format
Surprisingly, commentators pay little attention to the narrative aspect from a formal point of view, usually writing and using full sentences rather than short statements in the form of lists. However, this requires extra effort on the part of those who are not native speakers of the language in which they are supposed to write their resumes. And this exercise puts at a disadvantage those who do not know how to “sell” themselves.
Even more surprising is that what is being called a revolution in how people present their careers is actually a return to the typical format of the 1950s and 1960s (see below). summaries of german scientists). At that time, the summary was text that explained the intellectual choice of the candidate and emphasized certain aspects of his research, in particular, highlighting important publications from others. In France, there is also an “Analytical Summary” required to apply for an Associate Professor position.
In any case, whether this recent enthusiasm for narrative summaries is justified or not, it must be said that the mere fact of proposing and promoting a new term opens up debate, raises awareness, and challenges raters (and candidates). themselves, potential future evaluators) about the bad practices and biases that exist in the evaluation process of researchers. After all, the narrative nature of the resume is just an excuse to generate interest and work on implementing best practices.
By the time we have completed our research, new terms such as context summary, motivated resumeor evidence summary have already appeared, showing what else is many thoughts given to the form of the document itself and its place in the evaluation process, and the narrative is no longer the main factor.
This post relies on an article by the authors, Promoting descriptive summaries to improve research evaluation? Review of opinions and experimentspublished in Research Evaluation.
The content created on this blog is for informational purposes only. This article represents the views and opinions of the authors, but does not reflect the views and opinions of the Impact of Social Science blog (blog) or the London School of Economics and Political Science. Please see our comment policy if you have any concerns about posting a comment below.
Image credit: Invisible Studio via Unsplash.com.
