What are Doing Research Assessments?
Doing Research Assessment is the method used to assess the performance of a social science research system. The methodology is further detailed in the GDN working document on Doing Research Assessments. Elaborated on the basis of the pilot studies results, it reflects the fact that doing quality research requires much more than scientific skills and depends also on numerous other factors such as socio-economic, political and historical context, international dynamics, characteristics of the market for research, supporting policies and services, and many others. The implementation of Doing Research Assessments begins with an overall assessment of the context for doing research along economic, political, historical and international dimensions (step 1), followed by a mapping of national research actors to identify research producers and users (step 2). The context assessment and mapping of national research actors are then used as inputs into the Doing Research Assessment Framework, using a combination of secondary data, surveys and interviews (step 3).
Measuring the Production, Diffusion and Use of Social Science Research
Factors that enable quality academic knowledge generation are described under the function of production, which targets the creation of research, its output, and the long-term objective to build a quality knowledge base and critical mass of people valuing and understanding research. In that sense, the “production” column describes the factors impacting the academic community, from the necessary inputs in terms of people and resources, to the publication of research articles and the role of research in education and training. The second column describes the function of diffusion and looks at factors that enable the circulation and discussion of research-based products within different audience groups in the wider society. The “diffusion” column is therefore about sharing research products and understanding how research is mainstreamed and the role it plays in society and the media. The third column is about using research to support better policies, and reflects a view shared by many researchers and practitioners that it is desirable to strengthen the link between the research and policy communities. The “policy” column is about understanding how research supports policymaking, both directly when researchers are consulted or research is commissioned by policymakers, or indirectly by looking at factors that strengthen the research-to-policy nexus.
The external context for doing research is assessed along the economic, historical, political and international dimensions to allow a contextualized reading of the research system. It is document by a qualitative review of the elements comprised in the “context prism”:
Mapping Research Actors
The mapping of research actors is done to better identify and visualize the research actors – producers and users – composing the research system. The mapping is directed at a very macro level analysis and does not aim to count each and every university or funding agency. Instead, we identify and describe the importance of these groups towards each other and nature of their relations, and identify the main players within each group. Looking at the types of organizations composing the research system will be a first leap into identifying and analyzing actors for the third step of our approach, but is also largely a contextual element due to the way the management and development research organizations influences the research culture in a given country.
Research actors are divided into four categories: higher education institutions, government and funding agencies, industry, and civil society. These categories have sub-groups, for example HEIs have public and private universities which can be for-profit or non-profit, industry has for-profit think tanks and consultancies, civil society has NGOs, opinion leaders, non-profit think tanks and the media. Government and funding agencies is the most hybrid category, as it gathers national ministries, research councils, as well as public or private foreign donors.
Doing Research Assessment Framework
The Doing Research Assessment Framework (DRAF) is the tool used to implement Doing Research Assessments. After a context assessment and a mapping of research actors, the researcher implementing a DRA measures a list of indicators about key aspects of research systems, which are then compiled into the DRAF. The emphasis on the research system allows the DRAF to highlight features that are usually not measured in traditional country-level bibliometrics, such as the availability of local academic journals or the average duration of a research project. The framework has this structure.
Data and indicators
The data collection approach to document the list of factors which we have defined involves a combination of secondary data, surveys and interviews following a mixed method approach. The nature and collection of indicators will also be refined over time with the publication of results and additional observations, critiques and challenges, resulting in progressive improvement of the DRAF.
See selected Doing Research Publications