Catálogo de publicaciones - libros

Research assessments in the humanities are highly controversial. While citation-based research performance indicators are widely used in the natural and life sciences, quantitative measures for research performance meet strong opposition in the humanities. Since there are many problems connected to the use of bibliometrics in the humanities, new approaches have to be considered for the assessment of humanities research. Recently, concepts and methods for measuring research quality in the humanities have been developed in several countries. The edited volume ‘Research Assessment in the Humanities: Towards Criteria and Procedures’ analyses and discusses these recent developments in depth. It combines the presentation of state-of-the-art projects on research assessments in the humanities by humanities scholars themselves with a description of the evaluation of humanities research in practice presented by research funders. Bibliometric issues concerning humanities research complete the exhaustive analysis of humanities research assessment.

The ‘Mesurer les performances de la recherche’ project was funded through project-related subsidies of the Swiss Confederation allocated by the Swiss University Conference. Over the period 2008–2012, the project supported the exploration of new approaches to measure aspects of research that cannot be captured by conventional bibliometry. The project followed the Swiss Way to Quality in the Swiss universities (CRUS ), where the Rectors’ Conference of the Swiss Universities (CRUS, since 1 January 2015 called swissuniversities) is committed to a number of quality principles to guide its quest for university system quality. These principles were formulated on the basis of the CRUS understanding that quality is driven by the following two dimensions: international competition among each university related to specific stakeholder needs and cooperation through complementary specialization and coalition building among Swiss universities. In the long run, these quality principles should contribute to Switzerlands ambition to become a leading place for research, education and knowledge transfer. The project supported accounting for research performance rather than controlling the involved researchers. It also aimed to develop useful tools for the internal quality assessment procedure of Swiss universities according to the guidelines of the Swiss University Conference, devise strategies for Swiss universities and critique academic rankings. The project was successfully finalized by the end of 2012. As of 2013, the ‘Performances de la recherche en sciences humaines et sociales’ programme is up and running and pursues mainly the same goals as the previous project, but with a more specific focus on the humanities and social sciences. This project aims to develop instruments that will foster the visibility of research performance by scholars in the humanities and social sciences in terms of highlighting strengths of different research units located at Swiss universities. It will also strengthen a multiplicity-oriented approach to research evaluation, which aims to support the diversity that characterizes research in the social sciences and humanities.

In this contribution I argue that the Humanities, just like any other mature field of knowledge, should have or develop a system by which its research can be assessed. In a world that increasingly asks for justification of public funds, where public money becomes scarcer, so that less amounts have to be distributed among more players, where research funds are being concentrated and distributed on a highly competitive basis, we as humanists cannot shy away from research assessment with the argument that ‘we are different from the rest’ or that ‘we don’t need it’. Of course the humanities are a distinct member of the body of academic knowledge, but that holds true for every discipline. If we agree that for instance that bibliometry does not suit most players in our field, the question becomes: what will suit us better? Case-studies? This contribution also contains a warning: let us stop arguing about the language issue. English is the modern Latin of academia and its use enables us to communicate with one another, wherever we are or who we are. Without providing definite solutions, my argument is that we, humanists, should take the steering wheel ourselves in developing adequate forms of research assessment. If we leave it to others, the humanities will look like arms attached to a foot.

This paper summarizes key findings of our research on peer review, which challenge the separation between cognitive and non-cognitive aspects of evaluation. Here we highlight some of the key findings from this research and discuss its relevance for understanding academic evaluation in the humanities. We summarize the role of informal rules, the impact of evaluation settings on rules, definitions of originality, and comparisons between the humanities, the social sciences and history. Taken together, the findings summarized here suggest a research agenda for developing a better empirical understanding of the specific characteristics of peer review evaluation in the humanities as compared to other disciplinary clusters.

The assessment of research performance in the humanities is linked to the question of what humanities scholars perceive as ‘good research’. Even though scholars themselves evaluate research on a daily basis, e.g. while reading other scholars’ research, not much is known about the quality concepts scholars rely on in their judgment of research. This chapter presents a project funded by the Rectors’ Conference of the Swiss Universities, in which humanities scholars’ conceptions of research quality were investigated and translated into an approach to research evaluation in the humanities. The approach involves the scholars of a given discipline and seeks to identify agreed-upon concepts of quality. By applying the approach to three humanities disciplines, the project reveals both the opportunities and limitations of research quality assessment in the humanities: A research assessment by means of quality criteria presents opportunities to make visible and evaluate humanities research, while a quantitative assessment by means of indicators is very limited and is not accepted by scholars. However, indicators that are linked to the humanities scholars’ notions of quality can be used to support peers in the evaluation process (i.e. informed peer review).

This paper is a brief report on the European Science Foundation (ESF) Scoping Project, installed in 2009, results published in 2010, which examines the potential for developing some form of research output database that could be used for assessing research performance in Social Sciences and Humanities (SSH). Suggestions were made as to how such a database might look.

The ‘Norwegian Model’ attempts to comprehensively cover all the peer-reviewed scholarly literatures in all areas of research—including the preferred formats and languages of scholarly publishing in the humanities—in one single weighted indicator which makes the research efforts comparable across departments and faculties within and between research institutions. This article describes the main components of the model and how it has been implemented, as well as the effects and experiences in three of the countries that are making use of the model, and where it has been evaluated: Belgium (Flanders), Denmark and Norway. The article concludes with a discussion of the model from the perspective of the humanities.

This chapter reflects on how journals and book publishers in the fields of humanities and social sciences are studied and evaluated in Spain, particularly with regard to assessments of books and book publishers. The lack of coverage of Spanish output in international databases is underlined as one of the reasons for the development of nationwide assessment tools, both for scholarly journals and books. These tools, such as RESH and DICE (developed by ILIA research team), are based on a methodology which does not rely exclusively on a citation basis, thus providing a much richer set of information. They were used by the main Spanish assessment agencies, whose key criteria are discussed in this chapter. This chapter also presents the recently developed expert survey-based methodology for the assessment of book publishers included in the system Scholarly Publishers Indicators.

‘European Educational Research Quality Indicators (EERQI)’ was a research project funded under the EU 7th Framework Programme from 2008 to 2011. The mission of this project was to develop new approaches for the evaluation of quality of educational research publications. Traditional methods of assessing quality of scholarly publications are highly depended on ranking methods according to citation frequency and journal impact factors. Both are based on methodologies that do not reflect adequate coverage of European scientific publications, namely in the social sciences and humanities. Hence, if European science or institutions are exposed to these evaluation methods, not only individual researchers and institutions are widely ignored, but also complete subject domains and language areas. The initiators of the EERQI project, as well as numerous researchers and evaluation bodies within the European Union, recognized the need to remedy the inadequacies of this situation.

In this chapter, the possibility of using bibliometric measures for evaluating research in the humanities is pondered. A review of recent attempts to develop bibliometric methods for studying the humanities shows that organizational, epistemological differences as well as distinct research practices in research fields ought to be considered. The dependence on colleagues, interdisciplinarity and the ‘rural’ nature of research in many humanistic disciplines are identified as factors that influence the possibilities of applying bibliometric methods. A few particularly promising approaches are highlighted, and the possibility of developing a ‘bibliometrics for the humanities’ is examined. Finally, the intellectual characteristics of specific disciplines should be considered when quality indicators are constructed, and the importance of including scholars from the humanities in the process is stressed.