This reports shares the findings of an evaluation of the Oxfam Great Britain (OGB)-supported Raising Her Voice (RHV) project in South Africa (RHV-SA). The purpose of the evaluation was to rigorously assess the effectiveness of the project in terms of its contribution to greater women’s empowerment. As explained in the report, the global RHV programme aims “to promote the rights and capacity of women to engage effectively in governance at all levels through increased voice and influence, and greater institutional accountability. Comprising 17 different country-specific programmes, RHV is an ambitious attempt to advance Oxfam’s work on good governance and gender equality.” In South Africa, the project was implemented by People Opposing Women’s Abuse (POWA). POWA designed the RHV-SA project to address specific challenges related to women’s rights and participation in governance processes in South Africa. In particular, POWA identified the need to work at the intersection of gender-based violence (GBV), HIV and AIDS, and poverty, as GBV and economic marginalisation increase women’s vulnerability to HIV and AIDS; contracting HIV in turn exacerbates marginalisation and social exclusion. At the same time, the project aimed to contribute to a regional initiative to support the Pan-African Solidarity for African Women's Rights coalition (SOAWR) in its work to ensure the implementation of the Protocol to the African Charter on Human and Peoples' Rights on the Rights of Women in Africa, better known as the Maputo Protocol.
Over the project’s lifetime, a range of activities were implemented, which are described in detail in the report. These included a rapid assessment study, a baseline survey, and the development of a conceptual framework that sets out a detailed understanding of the intersections between GBV, poverty, and HIV and AIDS. Specific communication-related activities included:
- The adaptation of a radio drama series: the RHV-SA project began its activities by adapting a six-part radio series originally produced for audiences in Kenya, called Crossroads. The radio series explores issues related to the Maputo Protocol, tackling women’s rights issues in a dramatic and engaging way. The series was broadcast on local radio stations, which stimulated calls from listeners who could then put their questions and comments to RHV-SA project staff. The Crossroads radio series was accompanied by a discussion guide intended for use by radio presenters, teachers, community leaders, and group facilitators. This led directly to the establishment of eight women-only ‘listening groups’ that listened to the broadcasts and then discussed the various issues it raised for them. These listening groups became the catalyst for women to begin taking action on issues of GBV and women’s rights more broadly, eventually transforming into what are now called community action groups (CAGs).
- Establishment of CAGs: These groups, comprising around 8–10 women, received training and were supported to develop a one-year advocacy plan, which they then had responsibility for implementing and reporting against.
- Workshops and training programmes: In addition to CAGs, RHV-SA project partners received training on advocacy skills via workshops designed to assist groups in identifying advocacy opportunities with their local communities.
- Digital stories and community dialogues - The project produced a 20-part series of ‘digital stories’ on the subject of violence against women to be used during community dialogue events with local government, to mobilise communities while holding local government to account.
The evaluation aimed to assess confidence in the project’s claim that its training and support to marginalised women contributed to their newly engaging in local governance processes. As explained in the report, “[U]sually, evaluations under Oxfam’s Women’s Empowerment thematic area are evaluated using quasi-experimental impact evaluation techniques. In this case, given the characteristics of the project, a different impact evaluation technique called process tracing was used to assess the project’s contribution claim. To further strengthen the findings, an application of Bayesian updating (probability) was applied to quantify confidence in the project’s contribution claim.”
The brief describes the evaluation process in detail as it relates to the different activities and outcomes of the project, including some of the challenges. These included lack of adequate documentation, gaps in data, and inadequate monitoring systems. The following are a selection of outcomes emerging from the evaluation:
- Final outcome: More women and women’s groups newly engaging in local governance processes - Whilst the evidence found is limited, there is more confidence than not that the project has influenced more women and women’s groups to newly engage in local governance processes.
- Intermediate outcome: Women who participated in the training found it to be relevant and that it increased their knowledge of their rights and how to claim them - Based on the evidence, there is confidence that project participants did indeed find the training they received to be relevant and that it increased their knowledge of their human rights and how to claim them. This is a clear success of the project.
- Intermediate outcome: An increase in women’s collective action to share and disseminate knowledge following the training - Based on the evidence observed - such as formation of the community action groups and the action plans they have developed - there is reasonable certainty that project participants have increased collective action to share and disseminate knowledge.
The evaluation also shares a number of learning points for OGB decision makers to consider that will improve future planning, monitoring, of and evaluation of Oxfam’s programmes. For example, “one key learning is to conduct future programme design qualitatively, with a well developed and designed monitoring and evaluation framework that contains success indicators that focus on the change as well as tracking results. Oxfam in South Africa also recognises that entering into contractual agreements with partner organisations on an annual basis creates instability and uncertainty for partners and recommends that future projects of this design have a much longer term plan for predictability.” In addition, greater promotion of theory of change as a planning, monitoring, and evaluation tool is required, especially in projects and programmes concerned with good governance work. The project should also invest in more systematic ways of capturing women’s stories of change to ensure evidence of impact is preserved as project staff and participants change over time.
Click here for the 2-page summary of the evaluation results.
Oxfam Policy and Practice website on July 27 2017.