Improving the capacity of local researchers in low- and middle-income countries is a recognized way to advance health and development. Numerous approaches to health research capacity strengthening (health RCS) exist including training for individuals, improving research systems within institutions, and international collaborations among health research agencies.

But how do we know if these approaches work and which are the most effective? Interventions are complex and diverse and take place in different types of institutions with their own particular challenges. It is therefore difficult to measure and assess the changes that have taken place and identify lessons that can be applied more broadly. Donors who want to assess returns on their investment and demonstrate value for money find it challenging to do so. Health institutions that receive funding often face administrative burdens when confronted with monitoring and evaluation requirements.

So what can be done to streamline monitoring and evaluation of these initiatives? How can health institutions and staff benefit from the process? And how can learning be applied and shared more broadly?

In 2008 a group of funding agencies came together as the ESSENCE on Health Research initiative. Among other things, the group realized that there was little coordination among funders regarding monitoring and evaluation of health RCS. In 2011, ESSENCE published a Planning, Monitoring and Evaluation (PM&E) Framework as a guide for their member funding agencies and grantees, and encouraged the sharing of lessons about health RCS evaluations.

A group of researchers recently explored the use of this framework and other approaches to evaluating health RCS. The research identified some core tensions at the heart of carrying out evaluations of health RCS:

Degree of stakeholder participation

An external, non-participatory evaluation may be better for donors to assess value for money and ensure accountability. However, when funding recipients participate in the evaluation, they are more likely to learn, feel greater ownership over the project and implement recommendations.

Understanding processes and measuring impact

There are often tensions between the desire for a few common measurable and reliable indicators about process and the need for more extensive sets of indicators to evaluate project outcomes. A trade-off exists between having valid evaluations and dealing with complexity and time constraints.

Demonstrating accountability and enhancing knowledge

Funders face tensions between being able to demonstrate accountability and value for money, and supporting wider sharing and learning.

Reconciling short-term funding with long-term sustainability

Tensions exist between short-term funding to conduct and evaluate projects, longer-term funding over five to ten years to develop sustainable capacity and the two decades needed to show impact.

Researchers found that most evaluations fulfilled the needs of the funder but the evaluations would be more effective if they were based on broader theories of change. Findings suggested that donors should actively involve funding recipients and other stakeholders in all stages of the evaluation process. Such involvement can help people detect and correct problems with the project early on, make the decisions underlying the evaluations more visible and encourage sharing and use of the evaluation results. The research also found that there is increasing interest among funders to harmonize evaluation efforts.

To learn more about the findings of the research and key recommendations to donors and policy makers, take a look at three syntheses of the research papers.

1. Tensions in evaluating health research capacity strengthening

2. Frameworks to evaluate health research capacity strengthening

3. Indicators to evaluate health research capacity strengthening

Or contact:

• Garry Aslanyan, ESSENCE on Health Research initiative hosted at the Special Programme for Research and Training in Tropical Diseases (TDR) World Health Organization, Switzerland
• Imelda Bates, Liverpool School of Tropical Medicine, UK
• Alan Boyd, Manchester Business School, University of Manchester, UK
• Donald C Cole, Dalla Lana School of Public Health, University of Toronto, Canada

By Alison Dunn – Writer, Editor, Communicator for Write Space