It’s difficult to imagine a time before randomized trials and data. But as an article in the December 12 issue of the Economist points out, before the 20th century, “The sick were wise to stay away from doctors. Medical treatments were often worthless and sometimes dangerous.”
Then in the 1920s, English statistician Ronald Fisher used randomized controlled experiments to test the effectiveness of fertilizer on farms, and quality and production escalated. The medical field soon picked up the method. More than any single discovery, controlled experimentation contributed to the 20th century’s miraculous increase in lifespan.
In the realm of public policy and outreach, this technique can be tricky. Changing behavior is often hard to observe and quantify. What’s more, local government programs don’t have the budgets for lengthy experimentation and controls.
But things are changing. Online media provides the ability to conduct small and inexpensive controlled experiments that allow us to determine if people are taking the initial steps towards behavior change. For example, we can test degree of awareness, intent to change, influence of social norms and more.
In 2015, SGA began testing out how online experiments can help build better, more reliable outreach programs. Here are four simple, but important, lessons we’ve learned:
- Reach beyond likes and unique visitors. The biggest problem using social media likes and website visits as a proxy for success is that you don’t know why people took the action. They could have been searching for your content and found it engaging. Or they might have randomly clicked on your website during a Google search and quickly jumped off, never to return again. You just don’t know. While building a strong foundation of fans is important, it’s just the beginning. To build a community, you need to test how engaged your fans are through controlled messaging experiments.
- Embrace A/B testing. If you really want to understand the value of your likes, you need to see if they are repeatable. The key is to run two sets of posts—an A version and a B version. (You can also run multi-variant tests if you want to move more aggressively.) Then see how each performs with your community over time. You can set these up on social media, websites and e-newsletters. For e-newsletters, send the two variations to a random subset of your subscriber list a few days before you intend to send out the newsletter. Then send the one that does best to the rest of your list.
- Turn data into insights. Once you have some data to work with, you can look at the demographics of the people who engaged with your content as well as the content that got the best response. Facebook and Twitter insights and Google Analytics can tell you loads about your followers. Understanding the type of content that resonates with them will help you understand how to inspire them with behavior change messaging. This data allows you to paint a more robust picture of your audience, which you can incorporate into outreach both offline and online.
- Create a call to action. Find the route to engagement by asking your fans to do something. When determining your call to action, be sure it is something you can measure and start small. Ask fans to opt in to receive emails or post their own content showing actions they have taken. One of our projects for the Orange County Stormwater Program asked people to post photos of how they were saving water in their yards. By having residents show their actions, it verifies the behaviors taken and starts to build a social norm for the action—and engagement in general.
In 2016, we will see more and more emphasis on data as online marketing’s balance of art and science continues to demand more of the science. Business and commercial marketing has already shown this. With simple experiments, you can better understand your audience, their motivations and whether they’re buying your program’s message or merely window shopping.