Monitoring & Evaluation

Childhood immunization is a major public health concern in Cambodia. Given the high infant and child mortality rates and the low uptake rate of immunizations, a study of knowledge, attitudes and practices (KAP) of communities and health workers was conducted to identify barriers to immunization and inform future information, communication and education (IEC) strategies. Quantitative and qualitative research was conducted to discover the KAP of communities and health workers towards immunization services and the introduction of hepatitis B vaccine.

This article captures some of the knowledge gained through a growing body of field experience; in particular, it highlights a handful of tools and discusses existing challenges and opportunities for applying performance improvement methodologies in low-resource settings. [author’s description]

By looking at the many projects completed and under way, we can speculate about what lies ahead for HPT (called simply performance improvement in our community) in the places we work. [The author] make some predictions for the future of performance improvement in developing country public health. [He] thinks performance improvement use will become ubiquitous, more flexible, more inclusive of other approaches, and more comprehensive in looking at performer support systems. [adapted from author]

This IUD assessment tool provides performance standards for IUD provision based on type of visit-new client or revisit. It may be used as part of a programmatic performance improvement initiative, by the site or individual providers to self-assess or by external evaluators to assess achievement of the standards. Often, performance standards are used to affect change at multiple sites. [author’s description]

This guidance was developed to help staff of the ACQUIRE Project understand and explain to counterparts and field partners the improvement approaches and tools used by ACQUIRE. ACQUIRE brings together partners with proven, effective approaches to improving provider performance and the quality of services and to mobilizing communities to drive improvements in health care: performance improvement, quality improvement, and participatory learning and action.

The goals of this presentation include introducing the performance improvement (PI) approach, framework and factos influencing performance; how PI works; and the benefits of using PI. [adapted from author]

A conceptual framework of motivation processes is presented and used to identify strategies and options for the measurement of health worker motivation in developing countries. Measures of motivation are broadly organized into determinant and consequent categories, and determinants are further distinguished in terms of measures that influence worker–organization goal congruence (“will do” motivation) and those directed toward goal striving (“can do” motivation).

Problems related to health worker motivation are remarkably pervasive, but to-date little attention has been paid to them in developing and transition countries. Basic tools to measure the determinants and consequences of motivation have not been adapted to contexts outside the industrialized world. This paper assesses the feasibility of transferring psychometric tools, typically used in industrialized countries to measure motivational processes, to other contexts. The paper draws upon two field studies conducted in two hospitals in the Republic of Georgia and two hospitals in the Hashemite Kingdom of Jordan.

In Albania, the PHRplus Project developed and tested a series of tools designed to introduce family medicine concepts and strengthen primary health care (PHC) services. PHC facility managers will find the toolkits useful reference materials when developing strategies and tools to improve quality of care and monitor and evaluate PHC strengthening efforts. This series comprises three toolkits: PHC Service Delivery Toolkit; PHC Quality Improvement (QI) Toolkit; and PHC Health Information Systems (HIS) Toolkit. [from abstract]

This issue of The Manager will help managers at all levels understand the principles of local-level performance assessment and improvement. It also presents the concept of essential public health functions as a useful policy framework for decentalizing service management while maintaining and improving the coverage and quality of services. [editor’s description]

As the complexity of the challenges facing the public health workforce has increased, many have argued that insufficient resources have been devoted to the preparation of the workforce, including its leaders. Here we describe the growth of national advocacy for public health leadership and workforce development.

This cost analysis is based on QAP research on the effectiveness of two interventions (self-assessment and peer review) in sustaining or increasing the effectiveness of interpersonal communications training that midwives had taken. The research had measured the effectiveness of the interventions in terms of the number of utterances midwives made during family planning consultations, and this case study followed on, measuring the cost of each intervention in terms of the number of utterances generated.

This case study describes how five Integrated Management of Childhood Illness (IMCI) trainers and supervisors conducted an assessment of provider knowledge and skill to carry out IMCI at 38 facilities in two districts in Kenya. [author’s description]

The Assessment Approach is designed to provide a rapid yet comprehensive assessment of key health systems functions and related resources. The approach is organized around technical modules which guide data collection, and cover the following areas: stewardship, health financing, service delivery, human resources, pharmaceuticals, and ealth information systems. [publisher’s description]

Contraceptive provision in many settings continues to be based on outdated medical information, unproven theoretical concerns, and provider biases. Studies have found that in some developing countries 25-50% of women seeking contraceptives are refused services until they are menstruating. Coupled with effective training, checklists can be important tools for health care workers at various levels to apply the latest WHO medical eligibility criteria and guidelines for contraceptive use.

The main objective of this study was to determine the perceptions of Iranian nurse educators and students regarding the evaluation of teaching effectiveness in university-based programs. [from abstract]

In this setting, physicians were already making site visits to clinics to monitor technical standards of care. An intervention was designed to reinforce doctors’ interpersonal communication (IPC) training. Under the intervention, doctors received IPC job aids, self-assessment forms, and tape recorders. They taped themselves during consultations and assessed their skills from the recordings, using the forms and in consultation with their supervisors. The self-assessment form and the supervisor assessment form were modified to be reproduced in this report. [publisher’s description]

This study examines the relationship between IMCI compliance and three cost issues: the length of client-provider consultations, treatment drugs, and workload. The setting was 26 health clinics in Niger, the only developing country setting where QA was implemented before IMCI. [publisher’s description]

The article analyzes the practice of physicians and nurses working on the Family Health Program. A questionnaire was used to assess the evidences of assimilation of the new values and care principles proposed by the programme. The results showed that a great number of professionals seem to have incorporated the practice of home visits, health education actions and planning of the teams’ work agenda to their routine labour activities. [abstract]

The research described in this report investigated whether facility-based teams that had been trained and coached to develop and implement improvements in performance of the Integrated Management of Childhood Illness (IMCI) algorithm through problem-solving teams would improve case management. The study compared 21 facilities with teams and 14 without, all in rural facilities in Kenya.

This report presents data on in-hospital care for childbirth and obstetrical emergencies in 14 hospitals. This report examines intervals between critical events

This report presents the results from Rwanda for the of the Quality Assurance Project’s three Safe Motherhood Studies: competence of skilled birth attendants, the enabling environment for skilled attendance at birth, and the causes of the delay in receiving medical attention after a woman arrives at a healthcare facility. The Rwanda study focused on an urban referral (tertiary care) hospital with an active maternity department, two mid-sized referral (secondary care) hospitals, and four health centers. [publisher’s description]

This speech contends that a well performing workforce is the most critical determinant of the performance of services. Well functioning infrastructures and equipment do not serve if the people who deliver the services are not there (be it that there are none available, that they are absent from work, or that they are on strike). Well trained health workers, without the appropriate tools and supplies cannot do much either. And even when all ingredients are there, if workers are not motivated, services underperform. Advocacy to convince policy and decision-makers, as well as international agencies and donors to do something to improve the performance of the health workforce (HW) has been fervent these last 3-4 years, and it has been quite successful.

As part of the High-Level Forum on the Health Millennium Development Goals, this issue paper discusses improving health workforce performance as a key factor in meeting MDGs. The required scaling up of interventions towards the MDGs depends on effective health services delivery systems (HSDS). The availability, the skills, the attitudes, motivation, and behaviors of health workers are key to well-functioning HSDS. [adapted from author]

Through its Safe Motherhood Research Program, the Quality Assurance Project carried out three studies to explore issues regarding competence of skilled birth attendants, the elements that contribute to an enabling environment and the causes of the delay in receiving medical attention after a woman arrives at a healthcare facility in countries with high maternal mortality ratios.

This report discusses care provided to 245 women during labor, delivery, and immediate postpartum and their newborns during immediate postpartum. The quality of care for different tasks (e.g., monitoring fetal heart rate) is presented by country, by hospital type, and overall. The report details performance on recommended tasks and should inform program managers and providers in finding similar weaknesses in their own care delivery systems. Report includes 21 data tables and the data collection instrument for observations. [publisher’s description]

Case management maps (CMMs) are a type of job aid: a sheet of paper with information that guides healthcare providers in treating patients. Each patient has his or her own condition-related CMM, which is maintained in the patient’s chart or on the wall near the patient’s hospital bed to inform providers of the treatment protocol, what treatment was provided when and by whom, what to do should a critical event occur, etc. This report describes a study that implemented two CMMs that were introduced about a year apart in a 500-bed hospital where such job aids had not previously been used.

This study sought to better understand how to sustain provider compliance with standards, using local (Mali) standards (on care for fever and structural quality). The intervention had two parts: a self-assessment instrument that providers used weekly to assess their performance with a feverish client and a review of that performance by a colleague who had observed the consultation. The study found that when used regularly, such an intervention can have a significant effect on compliance.

Research on the Integrated Management of Childhood Illness (IMCI) shows that it is a scientifically sound way to treat sick children, but ways to ensure that it is implemented properly are lacking. This 1997-98 study examined and compared three implementation approaches: structured feedback of performance data, structured feedback of performance data where quality improvement (QI) teams were in place, and the formal World Health Organization training in districts with QI teams.

This user’s guide contains materials needed to design and implement the Quick Investigation of Quality (QIQ) in a given country. QIQ refers to the set of three related data collection instruments designed to monitor 25 indicators of quality of care in clinic-based family planning programs. This volume includes an overview of the QIQ (including objectives, short list of indicators, and methodological and ethical issues), guidelines for sampling and training of field personnel, instruments and guidelines for data collection and summary results from short list of indicators (tabular and graphic forms).