Evaluating Communication, Development and Change Programmes

What's Up?

Exploring the development of indicators derived from a social change and social movement perspective

From the work by The Rockefeller Foundation's Communication and Social Change Network

Prepared for The Communication Initiative Forum

Communications for social change is a process of public and private dialogue and linkages through which people define who they are, what they want and how they can get it.

Social change is positive change in people's lives – as they themselves define such change

This work seeks particularly to improve the lives of the politically and economically marginalised, and is informed by principles of tolerance, self-determination, equity, social justice, and active participation for all.

Behaviour change communication is a process of understanding people's situations and influences, developing messages that respond to the concerns within those situations and using communication processes and media to persuade people to increase their knowledge and change the behaviours and practices which place them at risk.

Characterisation of where we are at presently - the very positive base from which to move forward.

The Rockefeller group's definition attempts to rebalance strategic approaches to communication and change by taking the overriding emphasis:

Away from people as the objects for change …

• …and on to people and communities as the essential components of their own change
  

Away from designing, testing and delivering messages ….

• …. and on to supporting dialogue and debate on the key issues of concern
  

Away from the didactic conveying of information from technical experts….

• …. and on to sensitively placing that information into the dialogue and debate
  

Away from a focus on individual behaviours….

• ….and on to social norms, policies, culture and a supportive environment
  

Away from persuading people to do something ….

• ….and on to negotiating the best way forward in a partnership process
  

Away from technical experts in ‘outside' agencies dominating and guiding the process…

• ….and on to the people most affected by the issues of concern playing a central role

How Come?

• The scale of the development challenge
  
  
  

• The nature of the development challenge
  
  
  

• Principles for development
  
  
  

• Programming experience
  
  
  

• Evidence
  
  
  

• Strategic observation
  
  
  

• Communication opportunities and trends
  

Why Measure?

Accountability - to the people engaged in the communication intervention through to funders.

Progress - impact on the priority issues

Improvement - information for strategic and fine tuning programme decisions

Motivation - a sense of achievement is crucial to all involved

Credibility - enhance the acknowledged credibility of the work

Measure what?

The long-term goal:positive change in the issues of concern:

• less poverty
• higher employment levels
• better gender equality
• less HIV/AIDS
• more girls in school
• higher educational achievement
• less child and maternal mortality
• better nutrition
• fewer accidents and no violence
• etc etc - These are the crucial measures of success.
  

Evaluation Challenge [1]:

Time Scale - these changes can take a long time

They will be the subject of overall study.

Programme and policy people need more immediate data that indicates the contribution being made

Therefore require agreement on indicators which:

Evaluation Challenge [2] :

When measured in the short term need indicators that:

• indicate a strong likelihood of long term change in the issue being addressed
  
  
  

• are applicable across issues
  

Evaluation Challenge [3] :

This happens in many fields.

• Increased immunisation levels predict decreased child mortality
  
  
  

• Increasing numbers of girls in school is often cited as a predictor of economic progress.
  

In communication, intent to change has been used as predictor of actual change.

Evaluation Challenge [4] :

Communication and social change needs a similar set of agreed indicators both to measure and indicate progress and to drive the nature of the programming.

These need to be derived from reliable information and analysis

Indicator 1:Expanded public and private dialogue and debate

Key measurement questions

[In relation to the issues of concern]

What increase has there been in:

• Family discussion?
  
  
  

• Discussion among friends?
  
  
  

• Discussion in community gatherings?
  
  
  

• Coverage and discussion in news media?
  
  
  

• Problem solving dialogue?
  
  
  

• Focus and discussion in entertainment media?
  
  
  

• Debate and dialogue in the political process?
  

Example: The approach to HIV/AIDS in Uganda.

Indicator 2: Increased accuracy of the information that people share in the dialogue/debate

Key measurement questions

[In relation to the issues being addressed choose]

• 5 pieces of data over which there is general consensus
  
  
  

• 4 different perspectives on the issue.
  

Test the extent to which these are accurately reflected in the locations for dialogue and debate mentioned above among friends, within the family, etc.

Increased accuracy of the information that people share in the dialogue/debate

Example: The increasingly informed views of the general public on the implications of tobacco use and the strategies of tobacco companies.

Indicator 3: Supported the people centrally affected by an issue[s] voicing their perspective in the debate and dialogue

Key measurement questions

• Which groups in relation to the issue of concern are most disadvantaged?
  
  
  

• How were they supported to give voice to their perspective?
  
  
  

• What happened?
  

Indicator 4: Increased leadership role by people disadvantaged by the issues of concern

Key measurement questions

• Who makes the major decisions concerning the priorities and activities of the communication intervention?
  
  
  

• How are the people centrally affected by those issues engaged in the decision making process?
  
  
  

• What are some specific examples where the involvement of that group has influenced strategic or fine tuning decisions?
  

Example: The role of people with HIV/AIDS in significantly affecting the nature of some HIV/AIDS communication initiatives and the absence of that perspective from many other HIV/AIDS communication initiatives

Indicator 5: Resonates with the major issues of interest to people's everyday interests

Key measurement questions

• Which were the issues that provided the focus?
  
  
  

• To what extent were people energised by these issues?
  
  
  

• What actions followed?
  

Example: In the civil rights struggle in the U.S.A:

• Buses - getting on a bus, including to go to school were activities to which everyone could relate
  
  
  

• the Dream - captured the essence of the struggle
  

Indicator 6: Linked people and groups with similar interests who might otherwise not be in contact

Key measurement questions

• Which groups are involved?
  
  
  

• What are their interests?
  
  
  

• Have they been linked together?
  
  
  

• How does that linking take place?
  
  
  

• Is there an alliance?
  
  
  

• How does the alliance work?
  

Example: The Child Survival and Development Revolution of the 1980s and early 1990s was essentially based on a communication strategy to draw a number of disparate partners into one focused effort.

Additional supportive material

Some of the “Route Map” Data

Westoff concluded: there is a persistent and frequently strong association between exposure to the mass media and reproductive behaviour in Africa in the expected direction; greater knowledge and use of contraception, intention to use contraception in the future, preferences for fewer children and intention to stop child bearing. In addition, there is evidence that media exposure is also associated with later age at marriage. These conclusions are generalizable to women and men, both married and single. The research controlled for other factors that would make such a difference: age, rural/urban, socio-economic status, educational acheivement etc. It was the linkage that made difference.

Some of the “Route Map” Data [3]

The Jiggasha programme in Bangladesh is a community level family planning programme based on community, small group, women-to-women and family dialogue and discussion. From the JHU-CCP book on health Communication: Over 50% of the participants said that due to the meetings they discussed family planning methods with their husbands and other women. Women who attended the meetings were 3.1 times more likely to discuss family planning with other women than non-participants, and 1.7 times more likely to discuss family planning with their husbands than non-attendants. About 30% of the women adopted a new contraceptive method, 22 percent continued using their current method, 17% visited a family planning clinic, and 7% said they switched to a new contraceptive method as a result of the Jiggasha meetings.

Some of the “Route Map” Data [4]

Twende na Wakati is a radio drama in Tanzania on the broad issues related to sexual health, including HIV/AIDS and sexual health. From the evaluation: 55 per cent of listeners talked to a friend [about issues raised in the radio programme]. 23 per cent of the listeners said they adopted family planning as the result of listening to "Twende na Wakati" Most adopted oral contraceptive pills and condoms.

Some of the “Route Map” Data [2]

In the evaluations of the national scale Soul City programme in South Africa a correlation can be drawn between the promotion and support for interpersonal dialogue on an issue and a resulting positive change. From the first evaluation report: 56% of the sample discussed issues raised in Soul City with friends or family. [And, as an example of positive change] People accessing two components of the Soul City programme are almost twice as likely to disagree with the statement AIDS sufferers should be moved away as people not watching the program [64% to 36%].

Communication opportunities and trends.

The technological, social and political developments of the 1990s create real possibilities for development communication interventions that meet good development principles and imperatives:

From community radio…

…to more participative political systems …

………to the new technologies…

…………..to rapid urbanisation..

……………….to [so many]

The opportunities are immense.