Communication for Social Change Consortium

Informed by recent evaluation literature and the author's experience evaluating communication for social change processes, this 13-page paper highlights the challenges - and the potential - of more inclusive, educational, and empowering approaches to the evaluation of social development and communication for social change processes and projects. Ailish Byrne focuses here not on how to evaluate these processes but, rather, on the underlying questions and assumptions that shape how this type of evaluation is carried out - and that provide insight as to what its value is. In short, Byrne here offers a framework for the evaluation of communication for social change, in the broader context of participatory development.

Drawing on her reading of and reflection on select evaluation literature, Byrne begins the paper by examining how various characteristics of the developmental sector impact our thinking about evaluation. She outlines and questions core assumptions about development practice and its evaluation, for example (referencing Doug Reeler's work): "(i) project interventions being seen as vehicles to deliver development, while indigenous social change processes are at best ignored; (ii) problems are visible to practitioners through cause-and-effect analysis; (iii) unpredictable factors are 'inconveniences to be dealt with along the way'; and (iv) logical and linear assumptions about achieving desired outcomes." In response to what she describes as frustrations with this paradigm, Byrne indicates that alternative evaluation paradigms and practices have been articulated, and she outlines their central tenets as follows:

• "Recognizing that development inevitably entails unpredictable, non-linear, messy processes: This highlights the need to deconstruct intervention as typically assumed as a linear-rational process...
• Appreciating multiple realities and perspectives on reality, that is, the practical relevance of epistemology:...Appreciating this means seeing knowledge-making as a complex, ongoing, evolving process that inevitably reflects power differentials and encompasses struggles over contested meanings....Within a participatory paradigm, as in communication for social change approaches and others advocated here, emphasis is consciously on the voices, lived experience, perceptions and knowledge of intended beneficiaries and less powerful stakeholders (the 'voiceless')...
• Focusing on dialogue as the essence of development:...[G]rowing evidence demonstrates the importance of strategies that give voice to affected groups and prioritise communication environments of interpersonal communication, dialogue and debate...
• Having an explicit values basis: Conscious attention to addressing power imbalances and privileging the voices of people who are traditionally marginalized and 'voiceless' implies challenging hierarchies at all levels and revisiting what constitutes 'knowledge'. From such a perspective planning, implementation and evaluation are focused on the priorities and needs of intended beneficiaries, with 'success' determined by how far these are being met...
• Focusing on the 'primacy of practice':...[G]ood practitioners spend time with people observing and learning, asking questions, building relationships and trust and facilitating mutual learning from experience. That is, practicing action learning. This requires particular attitudes, competencies and abilities and highlights the importance of developing capacity at different levels..."

In Section 2 of the paper, Byrne explores shifts in evaluation theory and practice in recent decades, arguing that evaluation should be grounded in lived practice and should be integral to organisation learning and social development - importantly, as part of a process, not as an element to be tacked on at the last minute. She outlines the fundamentals of evaluation within a participatory paradigm, which (as echoed in the above excerpt) include an explicit values basis, application of the implications of multiple agendas and perspectives, explicit focus on reflection and learning from experience, and grounding evaluation in practices and "messy realities". Byrne argues that this approach "calls for demystifying the field, letting go of traditional 'expert' roles and, rather, encouraging and supporting people to ask their own questions, share experience and learn from each other on a more equitable basis....There is no blueprint and appreciating the particularities of each context is crucial. Thus the evaluator carefully determines features of the situation at hand....Evaluators need to engage with a range of interests, broker exchanges of information between different groups and foster understanding and mutual learning....Readiness and ability to deal with unpredictable, complex processes and differences are crucial..."

Byrne moves on to consider the immediate and longer-term implications of these perspectives for those working in participatory monitoring and evaluation (PM&E). She outlines 15 factors for facilitating PM&E. They include a number of factors internal to organisations, such institutional openness and willingness to participate, and factors applicable to the wider developmental sector, such as policy development to enable the involvement of less powerful stakeholders, the facilitation of supportive networks for mutual exchange of experience and learning, and a firm grounding in local context and culture. To Byrne, these factors suggest the need for those involved in PM&E to work to strengthen and equalise relationships and foster trust (e.g., spending time with people listening and learning from their experience, rather than imposing). In addition, she suggests that PM&E processes need to be linked to and integrated within broader processes of organisational learning and development; this process involves time invested in building relationships and communicating the need to understand others' lived realities. Bryne also discusses the importance of broadening accountability and ensuring high-level support (both conceptual and financial).

In conclusion, the author calls for "greater realism about potential outcomes and impact, and lessening the pressure on organizations to evaluate their work in questionable ways from which they themselves gain little"....Instead, it is important to grant space, time and support for more inclusive processes of sharing, reflection and learning, or developmental evaluation. Much evidence bears witness to the benefits of participation in evaluation processes themselves....I highlight the importance of devoting space, time and resources to reflecting on and learning from experience, in recognition that an imperative to evaluate...lies at the heart of professional practice....Within a communication context, evaluation has a major role to play in fuelling positive change."

MAZI 13, November 2007.