Monitoring & Evaluation

The Data Revolution: Finding the Missing Millions

This report sets out the evidence that, even when people are counted, the counting is frequently not good enough. What is assumed to be an empirical fact – a statistic – is too often the result not of direct observation but of inference, assumptions or extrapolation, or political negotiation. In sub-Saharan Africa, some 133,000 women may have died from childbirth-related causes in 2013, or twice as many. We cannot be sure. [from introduction]

Assessment of Provider Competence and Quality of Maternal/Newborn Care in Selected Latin American and Caribbean Countries

To obtain a snapshot of the maternal and newborn care provided by different types of maternal and child health providers in Latin America and the Caribbean (LAC) to 1) better inform advocacy and programmatic strategies and interventions to improve the quality of those services in the region, and 2) determine the need for more rigorous study of the issues. [from abstract]

Geospatial Analysis in Global Health M&E

Geospatial Analysis in Global Health: A Monitoring and Evaluation Guide for Making Informed Decisions provides monitoring and evaluation (M&E) practitioners an overview of geospatial analysis techniques applicable to their work. This guide shows how geospatial analysis can be used to support public health program decision-making along with routine planning and M&E. [from abstract]

Mobile Technology for Monitoring and Evaluation and Health Information Systems in Low- to Middle-Income Countries

In many developing country settings, limited resources constrain the development and expansion of certain technology infrastructures — such as high-speed Internet — that could facilitate more efficient data collection and reporting in the health sector. Mobile technologies may offer an affordable alternative to (or alongside) other electronic health applications in many low-resource settings. In this paper, current uses of such mobile technologies in the developing world are discussed, and how these approaches can be applied to improve national monitoring and evaluation (M&E) systems.

Guide to Monitoring and Evaluation System Design for Value Chain Projects

The intention of this guide is to give project design teams, project M&E staff and project managers the tools and guidelines they need to effectively plan for and manage highly-effective systems for monitoring and evaluating value chain projects. In doing so, the guide aims to enable CARE to improve the performance of value chain interventions and improve CARE‘s ability to test the Market Engagement Theory of Change.

Supporting Close-to-Community Providers Through a Community Health System Approach: Case Examples from Ethiopia and Tanzania

Close-to-community (CTC) providers, including community health workers or volunteers or health extension workers, can be effective in promoting access to and utilization of health services. Tasks are often shifted to these providers with limited resources and support from CTC programmes or communities. Two cases are presented. [from abstract]

The New HIV/AIDS Program in Peru: The Role of Prioritizing and Budgeting for Results

This study aims to evaluate and develop recommendations on how the HIV/AIDS program in Peru can achieve better Value for Money. [from abstract]

Aligning Vertical Interventions to Health Systems: A Case Study of the HIV Monitoring and Evaluation System in South Africa

Since responsibility for health services management lies at the district (sub-national) level, this study aimed to assess the extent to which the HIV M&E system is integrated with the overall health system M&E function at district level. This study describes implementation of the HIV M&E system, determines the extent to which it is integrated with the district health information system (DHIS), and evaluates factors influencing HIV M&E integration. [from abstract]

Real-Time Assessments of the Strength of Program Implementation for Community Case Management of Childhood Illness: Validation of a Mobile Phone-Based Method in Malawi

Health surveillance assistants (HSAs) in Malawi have provided community case management (CCM) since 2008; however, program monitoring remains challenging. Mobile technology holds the potential to improve data, but rigorous assessments are few. This study tested the validity of collecting CCM implementation strength indicators through mobile phone interviews with HSAs. [from abstract]

An Integrative Review of the Role of Remittances in International Nurse Migration

This review seeks to understand the role of remittances in international nurse migration within the context of three theories of international migration: equilibrium approaches, social networks, and globalization. [from abstract]

A Systematic Review of Health Technology Assessment Tools in Sub-Saharan Africa: Methodological Issues and Implications

Health technology assessment (HTA) is mostly used in the context of high- and middle-income countries. Many “resource-poor” settings, which have the greatest need for critical assessment of health technology, have a limited basis for making evidence-based choices. This can lead to inappropriate use of technologies, a problem that could be addressed by HTA that enables the efficient use of resources, which is especially crucial in such settings. There is a lack of clarity about which HTA tools should be used in these settings.

Human Resources for Health in Botswana: The Results of In-Country Database and Reports Analysis

The aim of this study was to analyse the existing databases on health workforce in
Botswana in order to quantify the human resources for health.

Systems thinking for health systems strengthening

Systems thinking provides a deliberate and comprehensive suite of tools and approaches to map, measure and understand
these dynamics. In this Report, we propose “Ten Steps to Systems Thinking” for real-world guidance in applying such
an approach in the health system. [from summary]

Assessing the Impact of mHealth Interventions in Low- and Middle-Income Countries – What Has Been Shown to Work?

Jeffrey Sachs, the Director of the Earth Institute, has suggested that ‘Mobile phones and wireless internet end isolation, and will therefore prove to be the most transformative technology of economic development of our time’. The mHealth community believes that this extends to healthcare. This review aims to summarise and assess the evidence of impacts that mobile technologies have had on improving health in countries categorised by the World Bank as low- and middle-income (LMICs), through mHealth (mobile health) interventions. [from abstract]

Emerging Opportunities: Monitoring and Evaluation in a Tech-Enabled World

Various trends are impacting on the field of monitoring and evaluation in the area of international development. Resources have become ever more scarce while expectations for what development assistance should achieve are growing. The search for more efficient systems to measure impact is on.

Improving Health Care: The Results and Legacy of the USAID Health Care Improvement Project

This report is not just a summary of a USAID-funded project: It is an extensively-documented milestone for global efforts to improve health in lower- and middle-income countries (LMICs). Increasingly, the central strategy for global health efforts to save lives focuses on selected, high impact interventions. Organizations have
supported these interventions chiefly by providing the required resources, such as training, drugs, and technical
assistance. But in order to implement any kind of service, the health system uses standardized processes for both
clinical and non-clinical activities.

District Health Management Information System (DHMIS) Standard Operating Procedures: Provincial Level

These Standard Operating Procedures aim to clarify the responsibilities and procedures for effective management of aggregated routine health services. These SOPs for provinces present basic and practical steps to be followed by
provincial health information management personnel, programme/line managers and clinic supervisors at provincial level to ensure that data is appropriately handled and used to improve service delivery at local level, prior
to submission to next level of the health system, within the specified time frames.

District Health Management Information System (DHMIS) Standard Operating Procedures: National Level

These Standard Operating Procedures aim to clarify the responsibilities and procedures for effective management of aggregated routine health services. These SOPs for the National DoH present basic and practical steps to be
followed by national health information management personnel, programme/line managers and clinic supervisors at national level to ensure that data is appropriately handled and used to improve service delivery at local level, prior
to submission to next level of the health system, within the specified time frames.

United States Government. Global Health Principles Monitoring and Evaluation Resource Guide

This guide provides clear monitoring and evaluation definitions, global indicators, and country-level indicators for a variety of U.S. agencies, to enabled them to collaborate more closely on shared objectives in global health. Interagency teams worked together, with MEASURE/Evaluation, to explore existing monitoring approaches, review the evidence, and develop meaningful and specific indicators. [from resource]

Effective Coverage: A Metric for Monitoring Universal Health Coverage

A major challenge in monitoring universal health coverage (UHC) is identifying an indicator that can adequately capture the multiple components underlying the UHC initiative. Effective coverage, which unites individual and intervention characteristics into a single metric, offers a direct and flexible means to measure health system performance at different levels. We view effective coverage as a relevant and actionable metric for tracking progress towards achieving UHC.

Measuring Customer Service in a Private Hospital

This study measures service quality management in a private hospital in Gauteng, South Africa. This was done by
determining the current standard of service quality management, identifying the gap between the value and the satisfaction of the service quality dimensions, as well as the influence of gender on the perception of service quality. [from abstract]

Strengthening Monitoring and Evaluation (M&E) and Building Sustainable Health Information Systems in Resource Limited Countries: Lessons Learned from an M&E Task-Shifting Initiative in Botswana

To improve monitoring and evaluation (M&E) of health programs in Botswana, 51 recent university graduates with no experience in M&E were recruited and provided with on-the-job training and mentoring to develop a new cadre of health worker: the district M&E officer. Three years after establishment of the cadre, an assessment was conducted to document achievements and lessons learnt. [from abstract]

Pay for Performance: An Analysis of the Context of Implementation In A Pilot Project in Tanzania

Pay for performance schemes are increasingly being implemented in low income countries to improve health service coverage and quality. This paper describes the context within which a pay for performance programme was introduced in Tanzania and discusses the potential for pay for performance to address health system constraints to meeting targets. [from abstract]

National Research for Health Systems in Latin America and the Caribbean: Moving Towards the Right Direction?

National Research for Health Systems (NRfHS) in Latin America and the Caribbean (LAC) have shown growth and consolidation in the last few years. A structured, organized system will facilitate the development and implementation of strategies for research for health to grow and contribute towards people’s health and equity. [from abstract]

A Hard Look at Global Health Measures

Since 2002, rich countries have poured more than $10 billion into malaria control. The money has helped pay for planeloads of bed nets treated with insecticides, hundreds of millions of doses of a powerful combination therapy, widespread indoor spraying of homes, and prophylactic treatment of pregnant women, an especially vulnerable group. The generous, large-scale programs have saved the lives of hundreds of thousands of people, most of them African children.Or have they? It may sound strange, but some analysts say we don't really know.

Evidence-Based Public Health: Not Only Whether It Works, But How It Can Be Made to Work Practicably at Scale

Because public health must operate at scale in widely diverse, complex situations, randomized controlled trials (RCTs) have limited utility for public health. Other methodologies are needed. A key conceptual backbone is a detailed “theory of change” to apply appropriate evidence for each operational component. Synthesizing patterns of findings across multiple methodologies provides key insights. Programs operating successfully across a variety of settings can provide some of the best evidence. Challenges include judging the quality of such evidence and assisting programs to apply it.

Exploring Accountable Care in Canada: Integrating Financial and Quality Incentives for Physicians and Hospitals

[Accountable Care Organ] ACOs are promising vehicles for aligning physician and hospital interests in improving quality and reducing cost. However, successful implementation and realization of the ACO mission requires that attention be paid to supporting capacity building within the ACOs, development of a culture of learning and improvement, as well as rigorous monitoring and evaluation. [from executive summary]

Straight to the Point: Assessing Partner Capacity Building Needs

This tool helps users conduct a concise assessment of a partner organization’s (or potential partner’s) strengths and weaknesses, helping to identify areas where technical assistance will be needed to successfully implement a project. The tool addresses key capacity areas including: human capacity, basic management capacity, M&E capacity, absorptive capacity, and community connectedness. The tool is meant to serve as a guide for interviewing multiple stakeholders at a partner or subgrantee organization. [from introduction]

An Evaluation of the Observance Rate of Component Information Management in the Health System of Chahar Mahal Bakhtiyari Province Based on World Health Organization Standards

Given the essential role of data collection and management in the health system, this study intended to evaluation of the
observance rate of component information management in the health system of Chahar Mahal Bakhtiyari province based on World Health Organization standards. [from abstract]

How Do We Know if a Program Made a Difference? A Guide to Statistical Methods for Program Impact Evaluation

This manual provides an overview of core statistical and econometric methods for program impact evaluation (and, more generally, causal modelling). More detailed and advanced than typical brief reviews of the subject, it also strives to be more approachable to a wider range of readers than the advanced theoretical literature on program impact evaluation estimators. It thus forms a bridge between more basic treatments of the essentials of impact evaluation methods and the more advanced discussions. [from abstract]