Measure Impact!

Another strategy you can use is to take quantitative data and find a way to dig deeper. Let's say you're tracking media mentions of your campaign. That's great (and something we ask for in our local campaign reporting form), but you could also do media monitoring where you watch for an increase in media coverage of the issue you are addressing, changes in how journalists frame the issue and an increase in audience interest (more comments, tweets, etc.) in such coverage.

 

If you are supporting survivors, remember that each survivor defines success differently and that's often tough to measure in a uniform way. Also, you may not be able to stop the abuse, but you can help people feel safer and stronger despite the abuse. In these cases, measuring changes in knowledge and attitude might be more effective than measuring changes in behaviour. Maybe you helped survivors feel more confident in some aspect of their lives, and that's fantastic even if it was not has high as you were aiming. 

 

You may not meet all of your desired outcomes, and that's all right. You decide what success means for your campaign, and evaluation is a continuous process. We tweak outcomes and measurement tools all the time, and sometimes we find that what we set out to measure wasn't quite right. With some practice, you will start to learn what's realistic for your work and your community. Furthermore, not achieving all of your outcomes does not mean you failed. Learning is a success in itself, and often there are unexpected outcomes. 

 

Once you have figured out what changes you want to measure, you need to determine the baseline (if possible). The baseline is the status before your action to encourage change. If you want to improve knowledge, what is the level of knowledge among your target population right now? You have to find a way to show change, so starting with the baseline gives you a before and after comparison. If you want to do a digital security training, for example, query participants about their level of knowledge or skills prior to starting the training and then do the same thing after. With the telecom example above, our baseline would be that the telecom has done nothing to address online GBV, including expressing any recognition of the problem or developing ways for people to report incidents.

 

You can find baseline by surveying or conducting focus groups with your target audience and by searching government and civil society resources for statistics. If you are holding workshops or other activities where you expect participants to experience a change during the short period of the activity, you can do a pre-test to get the baseline and then do the same test as a post-test at the end of the activity to capture change. Let's say you are holding a digital storytelling workshop. Your pre- and post-test could ask questions about level of knowledge and confidence related to digital storytelling tools and methods.

 

Sometimes you can't get the baseline in advance. As long as you can show a change, that's okay. With the tweet chat example in the section above, you might see the change you need in the content of the tweets themselves. You could also encourage participants to take a Twitter poll that asks them if the chat improved their understanding of online GBV. If you perform a play or do street theatre, you could poll the audience before they leave.

 

For each outcome you want to measure, you will need to determine how you will get the data you need. Conventional tools include surveys, pre- and post-tests, interviews and focus groups, but they may not be feasible for all of your actions, especially those that take place online. Even conventional tools can be used in creative ways. Often, surveys are conducted using a Likert scale (where there are five options ranging from two extremes) One local campaigner uses a range of smiley and frowning faces as answer options in surveys to capture participants' reactions to the activity and feelings about their knowledge and capacity in a way that feels informal and comfortable.  

 

Another method we like is the Most Significant Change technique. It suitable for everyone regardless of level of experience with evaluation, and it works well when outcomes are hard to define, when outcomes are too varied to measure easily or when you want participants to be active  in multiple levels of the evaluation process. The original guide by Rick Davies and Jess Dart is long but fairly easy to understand, and Davies notes that you do not have to follow all ten steps. You can adapt the technique to suit your needs.

 

Social media platforms make some analytics available. For instance, in the upper righthand corner of Twitter, you can find your analytics page, which includes such data as tweet impressions, profile visits, mentions and follower change by month. On Facebook, you can click the insights link at the top of the page to discover analytics that you can define by a date range and export for your records.

 

For more detailed data, you will need to use a social media analysis tool. Unfortunately, most of these tools charge a fee, but you can make the most of their free trials. There are some free tools that are decent and some companies that offer their most basic tools at no cost. Some characteristics to look for: if they have a free trial you can use, how many accounts and hashtags you can measure, what kind of data you have to give them, what timeframes they offer. 

 

Note that we already track #takebackthetech, #dominemoslastic and any additional global hashtags we create for campaigns, but you may want to track hashtags you have created or translated. For all social media use, you will want to tracks metrics around likes, follower increase and reach, but make sure you monitor metrics that reflect more significant engagement, such as shares and conversations. You might also find good qualitative data in social media interactions. We emphasise learning by doing and connecting online and offline, so the success of a campaign cannot be reduced to the number of retweets. Helping one woman stay online is worth more than 100 retweets.  

 

If you want to get serious about measuring impact, here are some more resources on monitoring and evaluation:
 

Community Tool Box: Evaluating the Initiative, University of Kansas

A Guide to Gender Analysis Frameworks, Oxfam

A variety of resources, Clear Horizon