Monitoring & Evaluation

This handbook offers health managers, researchers and policy makers a comprehensive, standardized and user-friendly reference for monitoring and evaluating human resources for health. It brings together an analytical framework with strategy options for improving the health workforce information and evidence base, as well as country experiences to highlight approaches that have worked. [from preface]

Evidence Based Practice (EBP) involves making clinical decisions informed by the most relevant and valid evidence available. Adopting an evidence-based approach to practice requires differing competencies across various domains including literature searching, critical appraisal and communication. This paper examines the current tools available to assess EBP competence and compares their applicability to existing assessment techniques used in medicine, nursing and health sciences. [from abstract]

This paper aims to describe the design, methods and approaches used to assess the effectiveness and cost-effectiveness of the skilled care initiative in reducing pregnancy-related and perinatal mortality in Ouargaye district, Burkina Faso. [from summary]

This article offers guidance regarding the benchmarking of health workforce sufficiency as a critical component of monitoring and strengthening the performance of national health systems. [adapted from introduction]

This presentation outlines a methodology to improve quality and accessibility of health care with greater involvement of the community in defining, implementing and monitoring the quality improvement process.

This presentation discusses the QIQ method of evaluating the quality of service delivery and provider performance.

While evaluation is, in theory, a component of training programmes in health planning, training needs in this area remain significant. Improving health systems necessarily calls for having more professionals who are skilled in evaluation. This article describes a four-week course taken by two cohorts of health professionals from 11 francophone African countries. We discuss how the course came to be, its content, its teaching processes and the master’s programme results for students. [from abstract]

This chapter is an excerpt from “Community-Based Health Care: Lessons from Bangladesh to Boston.” Using tools of statistical quality control, simple field epidemiology can both motivate and lead community health efforts to achieve higher coverage of essential services. Even basic health workers can measure their accomplishments, which motivates all involved to strive toward agreed-upon goals. The importance of repeated measurement at the local level is well illustrated in this chapter. [adapted from author]

Several high-profile meetings have focused global attention on critical human resources for health (HRH) issues, providing much needed high-level support and calls for action to address the HRH crisis. The Capacity Project’s HRH Action Workshop series was intended to extend this work by focusing on specific HRH actions and experiences, what is being done in countries, what is working and what is not.

The community score card process is a community-based monitoring tool that is a hybrid of the techniques of social audits and citizen report cards.The CSC is an instrument to exact social and public accountability and responsiveness from service providers. By linking service providers to the community, citizens are empowered to provide immediate feedback to service providers. [from author]

Drawing upon a combination of complementary data sources, both new and existing, can result in useful and rich information for measuring and monitoring health workforce stock and flows, and the impact on health and health systems. [from author]

Measuring and monitoring the whole education and training pipeline is essential to the planning, management and quality control of the health workforce in a country. This requires timely and reliable data on each of its phases. [from author]

This easy-to-use web-based tool compiles and analyzes country data from multiple sources, provides charting options, and generates automated country fact sheets, helping users to assess the performance of the country’s health systems.

Improving service delivery for the poor is an important way to help the poor lift themselves out of poverty. This resource presents and evaluates tools and techniques to measure service delivery and increase quality in health and education.

This brief provides a list of facility-based data collection tools that have been developed by the World Health Organization and other partners. The resources can be used to meet a wide range of specific information needs on human resources in health systems. [adapted from summary]

Nurses provide the bulk of primary care services in South Africa. Post-apartheid health legislation envisions the provision of comprehensive primary services at all public clinics, which implies the need for a cadre of primary care nurses able to render such services. The objective of this study was to identify core competencies of clinic nurses and develop an evaluation tool for primary care nursing in South Africa. [from abstract]

Family Medicine training and assessment is becoming more formalized and developed in South Africa. Assessment of competency in relation to clinical skills can involve observation in the clinical setting, but is more usually assessed in an examination. Summative assessment of family physician’s clinical skills now usually includes an Objective Structured Clinical Examination (OSCE). Standardisation of the OSCE is required to define the pass mark above which a candidate performs at the level expected of a family physician. [from abstract]

The aims of these guidelines are to provide information about the concepts and processes essential for quality assurance of basic nursing and midwifery education in the African Region; propose a process and content for evaluating existing basic nursing and midwifery education programs; stimulate ideas for establishing a quality assurance system for basic nursing and midwifery education; guide allocation of human and financial resources in current and future programmes and services; provide well-defined international and regional standards of education. [from introduction]

Uganda has been successful in implementing the national “Health Management Information System” (HMIS). Disease surveillance reports and monitoring of key output indicators within the health sector seem to be the areas with the most remarkable advance. But little mention has been made on the importance of the use of information for monitoring performance indicators and for management/decision making purposes. The existing HMIS makes this possible. In this paper, the authors present the contribution of the PNFP health sector to the operationalization of the HMIS and of its use as a tool geared towards performance assessment and informed management decision-making. [from abstract]

To evaluate the quality of maternal clinical care, Immpact, a global research initiative, developed an innovative method, called TRACE, to trace adverse and favourable events in pregnancy care. It is based on the confidential enquiry technique, whereby expert panels of health care professionals assess the quality of health care provided to clients in an
adverse event, such as a maternal death. [author’s description]

This article assesses interventions aimed at improving family planning mechanisms and reviewing the problem-solving processes to build an effective monitoring system of the interventions at the local level of the overall system of the Ministry of Health and Family Welfare of the Government of Bangladesh. [adapted from author]

This manual describes the purpose and processes for developing and using a system of Human Resources for Health (HRH) performance indicators in order to monitor the activities of a public sector health service. It is particularly aimed at enhancing the management process in developing countries and draws on experiences of pilot work. [from purpose]

This paper aims to review current knowledge and experiences from ongoing efforts to monitor and evaluate capacity building interventions in the health sector in developing countries. It draws on a wide range of sources to develop a definition of capacity building and a conceptual framework for mapping capacity and measuring the effects of capacity building interventions. Mapping is the initial step in the design of capacity building interventions and provides a framework for monitoring and evaluating their effectiveness. Capacity mapping is useful to planners because it makes explicit the assumptions underlying the relationship between capacity and health system performance and provides a framework for testing those assumptions.

The focus of this guide is the measurement of capacity for the purpose of monitoring and evaluating capacity-building interventions. It responds to a demand among public health planners, evaluators, and practitioners for advice on assessing the many aspects of health programming that fall under the rubric of capacity building. [author’s description]

This IUD assessment tool provides performance standards for IUD provision based on type of visit-new client or revisit. It may be used as part of a programmatic performance improvement initiative, by the site or individual providers to self-assess or by external evaluators to assess achievement of the standards. Often, performance standards are used to affect change at multiple sites. [author’s description]

A conceptual framework of motivation processes is presented and used to identify strategies and options for the measurement of health worker motivation in developing countries. Measures of motivation are broadly organized into determinant and consequent categories, and determinants are further distinguished in terms of measures that influence worker–organization goal congruence (“will do” motivation) and those directed toward goal striving (“can do” motivation).

Problems related to health worker motivation are remarkably pervasive, but to-date little attention has been paid to them in developing and transition countries. Basic tools to measure the determinants and consequences of motivation have not been adapted to contexts outside the industrialized world. This paper assesses the feasibility of transferring psychometric tools, typically used in industrialized countries to measure motivational processes, to other contexts. The paper draws upon two field studies conducted in two hospitals in the Republic of Georgia and two hospitals in the Hashemite Kingdom of Jordan.

As the complexity of the challenges facing the public health workforce has increased, many have argued that insufficient resources have been devoted to the preparation of the workforce, including its leaders. Here we describe the growth of national advocacy for public health leadership and workforce development. We discuss the creation of the National Public Health Leadership Development Network (NLN), a consortium of institutes providing a system for leadership development, and we review the network’s creation of the Leadership Competency Framework for core curriculum design and development of performance standards for public health practice.

This cost analysis is based on QAP research on the effectiveness of two interventions (self-assessment and peer review) in sustaining or increasing the effectiveness of interpersonal communications training that midwives had taken. The research had measured the effectiveness of the interventions in terms of the number of utterances midwives made during family planning consultations, and this case study followed on, measuring the cost of each intervention in terms of the number of utterances generated. Activities/tools: Sample provider self-assessment form, sources of costs, evaluation of marginal benefit.

This case study describes how five Integrated Management of Childhood Illness (IMCI) trainers and supervisors conducted an assessment of provider knowledge and skill to carry out IMCI at 38 facilities in two districts in Kenya. [author’s description]