THE ROLL BACK MALARIA MONITORING AND EVALUATION REFERENCE GROUP
The RBM MERG was established in May 2003 to act as an advisory body for the RBM Partnership on monitoring and evaluation. MERG develops and maintains consensus on priority strategies for monitoring and evaluating activities to be supported by all RBM partners. This includes provision of technical advice on state-of-the-art approaches to monitoring and evaluation of malaria programmes and development of products that are useful for implementing monitoring and evaluation activities and reporting at regional, national and international levels. The technical focus of MERG has initially been on the global indicators (5) to ensure consistency and accuracy in national and regional reporting. The geographical focus of the advisory body has been on Africa; however, attention to issues related to other regions has been increasing since the inception of MERG. MERG itself is not involved in implementation; rather, the work of MERG is implemented by NMCPs with support from the RBM intercountry teams and RBM partners.
MERG works in collaboration with RBM partner agencies with an interest in monitoring and evaluation. Under the guidance of MERG, for example, WHO, UNICEF and the World Bank collaborated with the GFATM to produce the Monitoring and Evaluation Toolkit, a framework and composite guideline for planning monitoring and evaluation activities related to HIV/AIDS, tuberculosis and malaria (7).
MERG has developed five task forces: four for prioritizing and better defining several RBM coverage and impact indicators (5), and one for overall improvement of monitoring and evaluation capacity.
The task forces advise on operational definitions, possible measurement methods and data sources for the indicators, and how to improve monitoring and evaluation capacity at country level.
Mortality Task Force
The MERG Malaria Mortality Task Force recommends that for highly-endemic areas of Africa south of the Sahara, the primary burden and impact indicator to be monitored by countries is the allcause under-5 mortality rate. This is best measured by nationally representative household surveys, such as DHS and MICS (10), in addition to national census data. Mortality should be reported together with the coverage of malaria interventions, in particular the use of ITNs, the coverage of prompt and effective antimalarial treatment of children under 5 years of age, and the use of IPT by pregnant women. These coverage indicators are measured by the same surveys. The time trends in all-cause mortality and intervention coverage together could be used to model the trend in malaria-specific mortality (and morbidity) in children under 5 years of age. It is important to note that mortality levels measured retrospectively through birth histories have an inherent time lag. Therefore, the mortality impact of malaria control could be evaluated at the earliest at around five years after the onset of intensified control measures (20).
For African countries that are approaching the Abuja targets of 60% coverage with ITNs and prompt and effective treatment, it becomes relevant to evaluate the trend in malaria-attributable mortality, for triangulation with all-cause under-5 mortality. This might include data from small-scale sentinel demographic surveillance sites based on verbal autopsies, although it is recognized that verbal autopsies have limited sensitivity and specificity for malaria and could thus underestimate the actual impact of effective malaria control efforts on malaria-specific mortality (18).
Outside Africa south of the Sahara, vital registration and health facility records could provide a reliable indication of malarial deaths; if such records are not complete, they are at least likely to be valid as a trend indicator. In 2005, the task force plans to explore additional ways of estimating malaria-attributable mortality in non-African countries.
Morbidity Task Force
The MERG Malaria Morbidity and Incidence Task Force has worked to develop concensus on a method for estimating the incidence of clinical malaria episodes for all countries. The estimates will allow regular updating to track trends and progress towards RBM and Millennium Development Goals, and will serve as input to WHO’s next statistical report on the Global Burden of Disease. During the first task force meeting in October 2004, a draft estimation method that synthesizes maps of malaria endemicity, data from surveys on intervention coverage, research data on malaria incidence rates and HIS data on proportions of falciparum malaria cases was proposed and discussed, and possible improvements agreed (2) (Box 2). Draft country-level estimates of total clinical incidence and falciparum malaria incidence as of January 2005 are shown in Maps 3 and 4, respectively.
Refining the WHO map of populations at risk of malaria transmission
As the basis for the incidence estimation, the WHO map (Map 1) of the distribution of populations living at risk of malaria transmission (57) is being updated and refined from national to subnational level. The updated map distinguishes between low, moderate and high intensity malaria transmission. An update of Map 1 is being produced with input from a range of technical experts and with focused Geographic Information Systems work conducted in collaboration with Oxford University, England (21).
Parasite infection prevalence as an additional impact indicator?
RBM has not recommended monitoring of changes in malaria parasite prevalence as an indicator for monitoring the impact of control efforts (5) since in areas of stable transmission malaria infection alone does not necessarily reflect actual disease. Conversely, interventions might successfully reduce malaria morbidity and mortality without immediately producing detectable reductions in parasite prevalence. Nevertheless, since parasite prevalence in children can be precisely measured in representative household-based surveys, the outcome is worth further exploration as an additional burden and impact indicator. For a meaningful interpretation, parasite prevalence surveys should be conducted during or shortly after the malaria transmission season, in areas considered at risk of malaria transmission. Outside Africa, given lower endemicity and less acquired clinical immunity in the population, parasite infection might more closely correlate with morbidity than it would in Africa south of the Sahara. There, older age groups would also be relevant to sample.
Malaria-related Anaemia Task Force
In view of the documented reductions in childhood anaemia in response to malaria prevention or treatment in endemic areas in Africa south of the Sahara (73), the MERG Malaria-related Anaemia Task Force has proposed that childhood anaemia may be useful as an additional impact indicator in these areas. Although anaemia is not a specific indicator of malaria, in very young children— less than 60 months, or less than 24 or 36 months in the most endemic settings—malaria may account for a large proportion of moderate and severe anaemia. The prevalence of childhood anaemia can be precisely measured through household surveys such as the national DHS, and it should be considered for inclusion in the laboratory component of national and subnational MIS. The task force has noted that care should be taken not to infer time trends between subsequent surveys if they are conducted during different seasons, especially in areas of seasonal malaria transmission.
Task force members at the London School of Hygiene and Tropical Medicine, England, are conducting an estimation of the burden of malaria-attributable anaemia in African children under 5 years of age, which will become available in mid-2005.
Survey and Indicator Guidance Task Force
The MERG Survey and Indicator Guidance Task Force has worked to coordinate survey activities among organizations and involved partners. With Macro International, the task force coordinated the development of another survey tool, the MIS, to be used at a national or subnational level.
The potential advantages of MIS include the following:
The entire MIS package—questionnaire, training manual, guidance on sampling and sampling sizes with costing—will be available for use by countries in 2005 in hard copy, on CD-ROM and through the Internet.
Strengthening Country Capacity for Monitoring and Evaluation Task Force
MERG recognizes the need for strengthening country capacity in monitoring and evaluation of RBM activities. In order to identify specific country-level monitoring and evaluation capacity development needs and ways to meet these needs, the Strengthening Country Capacity for Monitoring and Evaluation Task Force was established to develop a conceptual framework for strengthening monitoring and evaluation capacity at country and subregional levels. Through an extensive review of monitoring and evaluation capacity and practices in Africa, the task force reported that monitoring and evaluation within NMCPs has remained weak, despite significant investment from RBM (76). These weaknesses are primarily caused by limited human resources, lack of equipment, lack of an enabling environment and weak linkages with other programmes and partners.
The task force recommends strengthening capacity by establishing and institutionalizing monitoring and evaluation systems within NMCPs. These systems should, as a minimum, include a monitoring and evaluation component within the NMCP, linked to a monitoring and evaluation subcommittee that is part of the country coordinating mechanism for malaria control. The monitoring and evaluation effort should be adequately staffed and equipped. Staff should have the necessary skills, clear job descriptions and adequate office and storage space to deliver the products of the monitoring and evaluation system. The monitoring and evaluation component should establish links with other institutions within and outside the MoH in the form of a subcommittee charged with promoting best practices in monitoring and evaluation and coordinating RBM monitoring and evaluation within the country. This will maximize available resources, technical capacity and data collection efforts.