Communication Currents

Efficient Campaign Evaluation

April 1, 2013
Health Communication

Efficacious communication campaigns require exposure to the campaign’s messages and messages that work. Knowing what messages can be effective without actually running the campaign is an important shortcut to an efficient use of resources. This is especially true for public campaigns that typically do not have the resources of commercial advertisements.

Running a mini-version of a campaign is time consuming, expensive, and highly impractical. If campaign designers could test their messages in some quicker, cheaper but less perfect way than running a mini-version of the campaign itself, then campaigns could be much more efficient.  One option is to find a shortcut to assess the effectiveness of candidate messages. A good deal of previous research has been conducted on such shortcuts such as perceived message effectiveness.  Perceived effectiveness is associated with how believable or realistic the message is, and is linked to learning,attitudes and intentions to act in ways consistent with the message.

One big problem with what is already known about perceived effectiveness is causal direction.  If people who already believe the message evaluate it favorably, then asking about their reactions to the message does not tell us about the message, only about their prior attitudes. For example, maybe smokers who are trying to quit think anti-smoking messages are more effective than smokers who have no interest in quitting. If so, then perceived effectiveness is more a measure of how much someone agrees with the message and less of an assessment of the message itself.  Our primary focus was to find a way to determine whether perceived effectiveness preceded actual effectiveness or vice versa. 

To better understand the relationship between perceived effectiveness and actual effectiveness, we enlisted the help of over 1,000 adult current smokers from all over the U.S. Each person watched four (out of 100) professionally produced anti-smoking television Public Service Announcements (PSAs) and answered questions on his or her own computer. After watching each PSA, individuals provided their assessment of the message’s perceived effectiveness and their emotional responses to the message (e.g. feeling sad or hopeful). However, we had to develop a unique approach in order to unravel the question of causal direction. Instead of using each person’s reactions to the four anti-smoking PSAs to see if they were associated with their intentions to quit, we used everyone’s assessment of each message to get an average effectiveness of each message. This way, we had an average perceived effectiveness of each message that was not restricted to a single person’s evaluation.

The most important result was that the average effectiveness of messages seen increased a person’s desire to quit after watching the four random anti-smoking PSAs. Specifically, messages perceived as effective increased an individual’s desire to reduce the number of cigarettes they smoke a day, talk to someone (a family member or friend) about quitting smoking, and their overall desire to quit smoking. In addition, anti-smoking PSAs that made current smokers feel hopeful or proud about quitting increased their intention to quit smoking completely and permanently and desire to reduce the number of cigarettes smoked daily. All of these actions move current smokers in the direction of cessation, some more so than others. 

Even when advocating a difficult behavior to change, like smoking cessation, persuasive messages can influence individual intentions to change their behavior, above and beyond their desire to change before watching the message. Researchers who are already using these tools in the field are finding that shortcuts to effectiveness do work in predicting tough behavioral outcomes such as smoking cessation, or at least quit attempts, which are a path to permanent smoking cessation. Perceived effectiveness is an efficient way for researchers and practitioners to help construct and identify successful messages for the lab and for the field.

About the author (s)

Elisabeth Bigsby

Northeastern University

Assistant Professor

Holli H. Seitz

University of Pennsylvania

Doctoral Candidate

Joseph N. Cappella

University of Pennsylvania

Gerald R. Miller Professor