This 71-page evaluation report explores the initial 4-year cycle of the Child Survival and Health Grants Program (CSHGP), which was carried out by Health Alliance International (HAI) in 7 Districts in the Democratic Republic of Timor-Leste. Launched in September 2004, the 4-year project (extended 1 year due to a balance of unexpended funds, and the procurance of a second grant), combined efforts with the Ministry of Health (MOH) to improve overall maternal health (including child spacing) and newborn care. Funded by the United States Agency for International Development (USAID), this was designed to respond to the demographic statistics of the country, which - according to 2006 United Nations Children's Fund (UNICEF) data - has the highest fertility rate in the world, a maternal mortality ratio of 660 per 100,000, and an infant mortality rate of 44 per 1,000 live births. Specific goals of this effort to improve health and reduce mortality and morbidity for mothers and infants included:

• improve the health policy environment and ensure national policies reflect the most up-to-date research in antenatal care, delivery care, and postpartum care;
• support cross-cutting areas such as information collection and supervision tools; expand the capacity of the district and health facility to deliver maternal and child health (MCH) services; and
• improve selected behaviours among the community with a focus on the following goals: 70% of pregnant women will receive antenatal care; at least 30% of women will deliver with a skilled provider; and 45% of newborns will exclusively breastfeed for 6 months.

The intervention mix of the 4-year period evaluated here combined community-level health promotion activities and health system strengthening. The strategies focused on training, behaviour change, materials development, and community outreach. These strategies were implemented under the overarching strategy of embedding all activities within the national framework and on the national implementation schedule. The project also invested in human resource development and systems strengthening, with an emphasis on supervision and use of data for programme decisions. For example, the supportive supervision portion of the intervention included modeling positive behaviour, developing a supervision check list that helps to guide the interaction, and providing logistical support of vehicles and fuel. HAI technical staff, already trained in essential newborn care, now conduct joint postpartum care visits with district counterparts and demonstrate the skills necessary to promote newborn health. In addition, the modules for training are ready to be used as soon as the Ministry is able to prioritise the training.

Key messages in HAI materials included a major film, which is detailed within Annex 1 of the report ("The Role of Video in Behavior Change"). HAI's baseline study of maternal health practices in Timor-Leste found a strong preference for home births and lack of appreciation for the need for skilled birth attendants. These were among the behaviours that HAI hoped to change using a vigorous media campaign as its strategy to educate the larger community on good health practices. The culturally relevant film was the centrepiece of the campaign, reflecting key messages that also are presented via other modes; it was made available to communities through facilitated public screenings. Specifically, "HAI engaged Max Stahl, a renowned international film maker, to create the film Feto Nia Funu, The Women's War. His film team comprised young Timorese and they built a foundation of trust and understanding with the subjects, gaining extraordinary access to shoot scenes of women giving birth. Using a film-maker of such a caliber elevated the interest in the film and provided immediate credibility. HAI engaged a local NGO [non-governmental organisation] to screen the film and to provide follow-up discussion groups for the community to clarify the key messages. At present, HAI has shown the film to over 8,000 adults...[and] over 3,000 children have seen the film out at the district level....Despite the need to cut some of the scenes to conform to Timorese experience, this film remains a very valuable tool for both education and discussion."

An excerpt from the Conclusions and Recommendations section follows:

"During this evaluation, it became apparent that HAI has achieved the status of being an external agent of the MOH in implementing the national health platform. HAI has been invited to provide technical assistance and support to every major health initiative promoted by the Ministry....As a partner with USAID and the government of Timor-Leste, HAI has been an able agent for creating positive change and meeting the objectives of improving maternal health in Timor-Leste. Because of the intensity of its process and the practice of working within the framework of national initiatives, HAI has been delayed in fully meeting its objective of improving newborn health...

Data from the KPC [Knowledge, Practice, and Coverage] survey done in June of 2008 indicates that HAI has met or exceeded most of their objectives, for example increasing the percent of mothers with children age 0-23 months who received one or more antenatal care visit from 50% to 82% and more than doubling the percentage of women who deliver with a skilled attendant from 15% to 32%. An objective that still needs further pursuit is the distribution of Vitamin A, which achieved 44% rather than the targeted 60%, but which did demonstrate a positive trend since baseline was 28%.

Results for indicators of improved newborn care were of a mixed nature. Exclusive breastfeeding was reported by 68% of women who had an infant less than 6 months of age, far exceeding the target of 45%. Results were distressing for the other indicator of improved newborn care, recognizing signs of newborn illness. Out of a target of 50% of mothers, only 2% of mothers could cite three or more serious symptoms of newborn illness warranting consultation with a health professional. However, fully 88% of mothers were able to identify at least one sign, with fever the sign most frequently reported. This gap was likely the result of the project's lack of success in getting district midwives trained in newborn care...

...HAI can attribute its program interventions progress to the following:

• Development and funding of the Maternal Child District Program officer position within the national health staffing structure.
• Development of key reproductive health and maternal child messages including the use of skilled providers, immediate and exclusive breast feeding and seeking antenatal care.
• Through field research, careful assessments of health facilities, anthropological beliefs influencing birth outcomes, and Emergency Obstetric Capacity, among others.
• Innovative media for transmitting messages, including two films and one street drama, as well as a host of radio spots and the photo cards.

...Best practices for the project include the following:

• Careful selection of senior technical staff and investment in the development of their skills; making senior staff a resource available to the Ministry of Health.
• Accepting direction, guidance and pacing from government counterparts, even at the risk of slowing program implementation. Ultimately, the payoff was that each of HAI's efforts became part of the national fabric instead of being parallel activities.
• Research and assessments completed, including extensive community consultation, before developing the materials used in health messages and clinical standards.
• Extensive use of videography and photos as the most appropriate medium for transmitting health messages when appropriately reinforced by trusted sources.

Based on these findings and the assessment of data available, some key recommendations are...

1. Rethink the approach to district representation for the delivery of technical assistance...HAI is extremely lean in its organizational structure. With the expected continued emphasis on district level, they might need to consider supporting the placement of more staff at the district level. The staff...should have clinical and technical skills necessary to provide support to the counterparts.
2. Increase advocacy at the national level for ownership of the proposed newborn interventions. Despite sound technical assistance and multiple proposals for various kinds of newborn interventions, including training and materials development, it appears there is not yet be a significant level of ownership for the newborn program. HAI has already begun to address this through field based modeling of technical skills but also need to persist in raising this with the Ministry...
3. Continue the integration of the reproductive health and maternal health programs and continue to emphasize the need for child-spacing as part of antenatal and post-natal care. Consider refining the target audiences as many older women (>35) already have basic knowledge about contraceptive methods because of the Indonesia programs, while many younger women who became of reproductive age post-Independence, have more limited knowledge and a greater fertility horizon.
4. Develop communication materials to help with the group discussions after showing the film A Woman's War. Most of the informants feel this film is making a real difference in knowledge of the good health practices surrounding pregnancy and delivery. HealthNet, who has aired the film in four districts, feels that there should be a three month post-screening follow-up to reiterate the messages. They also feel that there should be materials available that facilitate the viewing experience...clarifying any misperceptions after viewing the film...
5. Ensure opportunities within HAI to provide senior level skills development for national staff...
6. Reassess the roles and functions of Country Director...
7. Determine the best way that HAI can contribute to the national data base and HMIS [Health Management Information Services...
8. Obtain approval and a new Memorandum of Understanding with the Ministry of Health...
9. HAI should ensure that professional staff are equipped with a lap-top...

... HAI will take these recommendations and lessons learned and share them with the greater development community...

...Because all of its materials and program efforts take place within the national development channels, the program is inherently scaleable...

In conclusion, HAI has run a tightly focused but flexible project that has improved the status of maternal/newborn health in Timor-Leste..."

Development Experience Clearinghouse (DEC) Express, from the United States Agency for International Development (USAID) on April 3 2009.