Working paper, Bridging Theory and Practice, Issue 07
BBC Media Action
"This working paper is designed to identify what evidence is being used in practice and to highlight what is now needed to move the media for development evidence base forward. It aims to be used as the start of a sectorial conversation to inform an 'evidence agenda' for media for development."
This BBC Media Action Working Paper is based on a series of interviews with practitioners, evaluators, and donors working in the media and development sector. The paper is intended to open conversation and comment towards a final publication in the end of 2014 or early 2015. "[I]t examines their understanding of 'what counts' as evidence and their views of the existing evidence base, and
- Argues that compelling evidence of impact does exist and is being used - particularly in the field of health communication
- Contends that more evidence is needed to answer specific questions about the ways media and communication affects people and societies - and how this varies by local context
- Highlights where evidence gaps need to be filled through strategic, longer term collaborative investment in evaluation to build the evidence base
- Shares priority research questions for the sector
- Calls for an 'evidence agenda' to create shared evidential standards to systematically map existing evidence, and establish fora to discuss and share existing evidence."
At its outset, the paper defines media for development: "the use of media and communication to facilitate development outcomes (sometimes called development communications). Techniques include information dissemination and education, behaviour change communication, social marketing, social mobilisation, communication for social change and community participation, and creating platforms for dialogue and debate." This paper examines media for development and does not consider "media development[:]... the capacity strengthening for institutions and individuals to support the development of a free, independent and plural media." The paper calls for an "evidence agenda, which creates shared evidential standards to systematically map the existing evidence, establishes fora to discuss and share existing evidence, and uses strategic, longer-term collaborative investment in evaluation to highlight where evidence gaps need to be filled in order to build the evidence base."
Its five sections include:
- Section 1 contains Philip Davies' (Head of International Initiative for Impact Evaluation (3ie) in London, United Kingdom) analysis of the current debates in evidence.
- Section 2 focuses on existing evidence and gaps, highlighting interviewees’ theories and models of the role of media and communications in social and behaviour change and their best examples of impact evidence.
- Section 3 focuses on identification of the importance of evidencing both the direct and indirect outcomes from media and communication interventions.
- Section 4 focuses on the challenges around evidence generation in the sector.
- Section 5 introduces the steps to building an evidence agenda.
The paper concludes with the need to have:
- "Clearer standards for evaluation reporting
- Greater investment in rigorous formative and summative evaluations and implementation research
- Greater efforts to collate, comprehensively review and share the evidence that does exist
- Appropriate fora to critique and understand research and evaluation findings"
It recommends a research question-based agenda, not focused on privileging particular methodologies or research approaches. In answer to "What should count as evidence?", it suggests beginning with clarifying social context in societies - "this would need to be qualitative, projective and exploratory in nature. Media landscape and media consumption data, along with attitudinal and value measurement, needs to be combined with quantitative research looking at effect sizes. Additionally, the use of more longitudinal research tracking outcomes over time would be valuable for both learning and evidence. Greater use of local, national or even regional quantitative and qualitative case studies over time will also enable comparison. Overall, what we are looking for is the balance of evidence...."
The following quality criteria are set forth:
- "Theory of change
- A clear measurable set of research questions
- In-depth social understanding of the audience as the starting point
- Local context based - using a social-ecological framework
- Mixed methods
- Transparent reporting on what works and what doesn’t in the practice."
On its website, BBC Media Action invites comment: "This working paper is open to consultation. The appendices contain further information, including the complete list of evidence examples and priority research questions."
The BBC Media Action website, August 15 2014.