Outline of Presentation

• Background
• Sampling
• Fieldwork
• Preliminary Findings

Background

• 2 national sample household surveys of 2000 African and Coloured respondents.
• Measure the impact of components of the intervention (largely) at an individual level of analysis.
• Baseline (pre-test) survey in June 1999, before the Soul City intervention.
• Evaluation (post-test) survey in Feb 2000.
• Measure the same constructs in the two surveys
• Monitor changes over time.
• Compare responses between Soul City audiences and others

Sampling

• Multi-stage random stratified (by area and province) sample.
• First stage - 500 enumerator areas (EA's).
• Second stage - 5 stands selected within each EA.
• Where necessary a household was randomly selected within a stand.
• Household = group of people who spend at least 4 nights a week together, cook and eat together.
• Respondent randomly selected from all household members between 16 and 65.

Fieldwork

• Baseline survey (1979 respondents) – 10 days for data collection.
• Evaluation survey (1981 respondents) – 19 days for data collection.
• Fieldworkers documented the results of each visit to the household to try to obtain an interview.
• Baseline survey - 2 attempts to complete interview before substitution due to time constraints.
• Evaluation survey - 3 visits before substituting.
• Some rural farms and villages substituted in the baseline survey – access problems.
• Most questionnaires were administered during the day due to concerns about fieldworker safety.
• Questionnaires administered face-to-face in respondents' home language.
• Interviews were conducted in private where possible.

Pragmatics

• Questionnaire developed under time pressure
• Some additions after questionnaires printed.
• Questionnaire was not formally piloted at a large scale.
• Large fieldwork team
• Some inexperienced fieldworkers.
• Larger proportion of women than men in the survey.
• Data had to be collected over short period.
• Long questionnaire - approximately 75 minutes.

Training

• Training took place over one-week period
• Training at four centres around the country.
• 2-day training session.
• 4 research teams conducted training - due to time limitations.

Issues covered in training include:

• gaining access to particular areas
• sampling methodology
• using the random number grid
• recording details of household members
• administering the questionnaire
• translating questions into vernacular languages.
• Role-playing selected sections from the questionnaire.

Research quality

Sampling error = 2.2% at 95% confidence interval.

Check-backs:

• in-field by fieldwork supervisors
• by member of C A S E fieldwork unit, on 10% of the sample in each province.
• In C A S E office by fieldwork staff.

Invalid questionnaires re-administered or excluded.

Data punched and verified by professional data-capturers.

Data cleaned by C A S E research team.

Despite the limitations mentioned we feel that:

• the selection of respondents was unbiased,
• the data collected accurately reflects the views of the respondents interviewed.

Analysis

Stage 1 (Exploratory)

• Cross-tabulations to determine significant differences between the baseline and evaluation surveys.
• Used 5% significance level.

Stage 2 - Changes between baseline and evaluation

• Log-linear analysis using phase, race, sex, age, education and area as input variables.
• Test for significant changes between baseline and evaluation.
• Change could occur generally or within particular subgroups (combinations of particular categories).
• Isolate most important factors or combinations of factors determining responses to question.
• Report on interactions significant at the 5% level.

Stage 3 (Effect of Soul City)

• Constructed variable to measure exposure to Soul City TV and Soul City Radio.
• Loglinear analysis to test effect of exposure to Soul City, with access to Soul City, sex, race, age, education and area as input variables.
• Cross-tabulations in cases where analysis indicates possible influence of Soul City.