Network Impact (Taylor); Whatley (Cause Communications)
"Not all of these results were anticipated in Battlestorm's initial design, but they do demonstrate a pivotal role for youth as initiators of talk about hurricanes and hurricane prep. The significance of conversations prompted by the game cannot be overestimated. Our research shows that, once the impetus to communicate is established, and messages are processed in a social context, many positive outcomes are possible."
Conducted for the Knight Foundation, this report evaluates Battlestorm, a game that engages youth in efforts to keep communities on the Gulf Coast of the United States (US) safe in case of natural disasters, such as Hurricane Katrina. It combines elements of theatrical spectacle, reality game show, physical sport, and an online community (website or Facebook page). Battlestorm requires frequent supervised play sessions and adult-facilitated discussions of hurricane preparedness. It is intended to be integrated into the curriculum of a school or after-school programme which can provide oversight and access to a gymnasium over the course of 8-10 weeks. It was created by Area/Code for the John S. and James L. Knight Foundation, with partners including Boys & Girls Clubs (BGCs) of the Gulf Coast (BGCGC), the United Way, and the American Red Cross,
Conducted from March 10 to May 21 2011, the evaluation set out to document: participation and involvement of youth and other community members in the game; changes in attitudes to hurricane preparedness among parents and youth, as well as knowledge acquired about good preparation habits that can be attributed to the game; changes in behaviour related to hurricane preparedness among parents and youth that can be attributed to the game; and the degree to which the game affects the work of the community partners and becomes embedded in their ongoing programmes.
Evaluation Results: Findings Relevant to Youth
The survey of players confirmed that a large majority of Battlestorm youth enjoyed playing the game; what they enjoyed most was active court play: running, scoring, competition, and teamwork. Furthermore, the youth said that learning a new game was fun in itself, and, since there was no established "playbook", they had an opportunity to invent their own plays.
Evaluators compared the quality of questions that youth asked about hurricanes and hurricane preparation before and after they played the game. Here, they found that players' questions post-game were, on the whole, more sophisticated and detailed than the same players' questions at baseline and more sophisticated and detailed than the questions listed by members of the control group pre- and postgame. This may indicate that youth who played the game spent more time thinking about the topic of hurricanes or hurricane prep and/or took the question more seriously after their experience in the game.
More than half of the parents of Battlestorm players who responded to the survey confirmed that their family was directly affected by Hurricane Katrina. Of these, nearly one in five reported that their children, including those playing the Battlestorm game, continue to experience psychological and emotional problems caused by the storm. The "best evidence of the cathartic effects of the Battlestorm game comes from our research on talk within families. In some families, conversations about hurricanes and hurricane prep prompted by the game led parents and children to process their experiences of Hurricane Katrina..." For instance, at the follow up, Battlestorm players felt safer about a powerful hurricane strike than they did at the baseline, while the control group's feelings of safety decreased. "Based on parents' testimony, it is likely that conversations within families and hurricane preparation that ensued contributed to youth feelings of safety."
Evaluators found that Battlestorm game play had a positive impact on players' sense of self-efficacy (defined as: confidence in ability to i) handle a challenge, ii) cooperate with peers, iii) persist in completing a task, iv) be a good leader, and v) make a difference in own community). For example, in the case of self-reported ability to cooperate with peers, ratings for youth who played the Battlestorm game increased between baseline and follow-up, while ratings in the control group decreased.
Survey research confirmed that players learned little about hurricane characteristics or behaviour from either the Battlestorm website or the court game, which included some relevant cues (For example, a "storm surge" in the game described a situation where the opponent/storm gained extra powers. Related information was provided on the Battlestorm website, which included links to the Harrison County Emergency Management Agency.) However, the evaluation did find evidence of player learning about appropriate items for a hurricane prep kit. "While the number of items and gains were modest, the result is notable since this element was integral to the game's design and directly related to strategic game play. Hurricane prep kit items were deployed by players both as symbolic 'power tokens' on the court and as real life articles that players assembled with their families to boost home team advantage. This is important evidence that 'real life' social impact games built around strategy gaming can produce measurable learning outcomes."
Battlestorm players and members of the control group demonstrated similar knowledge about hurricane prep strategies before and after the workshop. The one exception was the strategy: "finding a safe place" or "shelter".
Survey results show that, between the baseline and follow up, a greater share of Battlestorm players started or continued talking with their parents about how to be prepared for a hurricane (68%) than youth in the control group (38%). Players were also more inclined to talk to friends about hurricane-related subjects:
- At the baseline, 23% of Battlestorm players had spoken with a friend about hurricanes in general during the previous month; at the follow up, 64% of Battlestorm players had done so. For example: "A boy who was severely traumatized by his experience during Hurricane Katrina told his mother about the Battlestorm game and its theme. For the first time, the family began to talk about the devastating effects of the 2005 storm. This player later took a leadership role in explaining some of the details of hurricane prep to his younger siblings."
- At the baseline, 23% of Battlestorm players had spoken with a friend about how to be prepared for a hurricane during the previous month; at the follow up, 44% of Battlestorm players had done so.
- At the baseline, 10% of Battlestorm players had spoken with a friend about what goes into a hurricane prep kit during the previous month; at the follow up, 40% of Battlestorm players had done so.
Evaluation Results: Findings Relevant to Players' Families
- "Parents' testimony is that conversations about hurricanes triggered by the Battlestorm game were cathartic: they allowed family members to process traumatic experiences related to Hurricane Katrina that had been repressed or not adequately addressed in the many years since the storm."
- Consistent with the survey responses of players, results of a matched-sample parent comparison showed that more parents discussed hurricane prep with their children after the game.
- One-third of parents reported that they learned something new from their child who played the game, and 25% reported that they learned something new from watching the Battlestorm game. Only a minority of parents (38%) reported visiting the Battlestorm website and accessing information from that source.
- At the baseline, parents of Battlestorm players had, on average, 13.4 of 22 important items to have in case of a hurricane emergency. At the follow up, parents had, on average, 16.5 important items.
Evaluation Results: Findings Relevant to the Larger Community
Ten weeks after the beginning of training, the Battlestorm game culminated in a final tournament in which BGCGC Club teams competed against an opponent styled as the "world champions" of Battlestorm. The tournament took place in a local high school gym equipped by Area/Code with a light and sound system and a large screen "jumbotron". A local sports announcer provided a running commentary of the game. The programme, with Battlestorm logo and design elements, included an explanation of game rules and a list of recommended items for a hurricane prep kit. In all, materials for 800 prep kits were distributed.
"One of the most important outcomes of the tournament, according to the BGCGC Director of Programs, was that BGCGC youth who participated felt special and valorized. Our focus group with players confirmed that the Big Event was the highlight of their Battlestorm experience. Players especially liked the video replays on the big screen and the media coverage both before and after the Event that showcased their Club teams."
The report concludes with reflections on the role of community partners, followed by reflections and recommendations. For example: "the game introduces a fun, physical activity to youth in a state with one of the highest rates of childhood obesity in the nation. Integrating the game as an alternative to conventional sports activities in public schools might serve to increase the total number of youth in the state who are exposed to regular physical activity."
- "Social Impact Games: Do They Work? BattleStorm and Macon Money Program Evaluation"
- BattleStorm Evaluation Summary PowerPoint
- Battlestorm Information Graphic
- Learn how to bring Battlestorm to your community
Email from Brett Davidson to The Communication Initiative on January 30 2013; and "Social Impact Games: Do They Work? BattleStorm and Macon Money Program Evaluation", Knight Foundation website, February 5 2013.