Every organization, big or small, strives to succeed in achieving their mission, and doing it as efficiently as possible. For that to happen, you need know where you are going and chart a clear route to get there. Evaluation can help you make sure you stay on the right path, reach the finish line when and how you intended, and can show others what you’ve achieved.

Why is evaluation important?

• • Your mission matters. When you succeed, you aren’t just meeting organizational goals; you are contributing to the greater good. And we need all hands on deck right now.
  • Your story matters.  Building evaluation into programs allows you to capture your successes and document how you created impact. Not only does this better position your organization, sharing your approach and how you overcome challenges can spark ideas and help others working toward common goals.
  • Insight matters. Taking the time to learn what works – and what doesn’t – makes programs stronger and organizations more efficient and effective. Certain types of evaluation can provide insight on an ongoing basis that allows you to make adjustments along the way, rather than finding out that you didn’t meet your goals at the end.  
  • Results matter. Funders, donors, and other stakeholders want to know that the money and time they are investing in your organization and programs are having measurable impact. You need to show them that you are making a difference.

And here’s how you do it.

Start with the Finish: Evaluation Planning

Set SMART objectives and identify the outcomes that will help you reach your goals. These paint a clear picture of your finish line, so you know where you are going and can assess when you’ve reached it.

Imagine that the (fictitious) Motunui School District (MSD) wants to increase flu vaccination rates to help reduce the number of students that miss school during flu season. Their SMART objective is to educate all 300 families in the district during the month of September about the importance of flu vaccination for their children.

Survey the Terrain: Formative Evaluation

Formative evaluation (also called formative research) tells you what needs to get done and how best to do it.

Take the time to better understand existing strengths and needs, identify relevant and effective strategies, and determine factors that can facilitate success. This insight supports strategy and program development, and can also serve as a baseline to measure against at the end of your program.

There are a number of tools MSD could use for formative evaluation. They chose the following:

• Records review: They gathered vaccination rates from past years.
• Survey:  They surveyed parents on whether they plan to vaccinate their children this coming flu season, the main reasons why they may not, and how they prefer to get health information.
• In-depth Interviews: They interviewed a diverse sample of parents across the district to understand their motivations, barriers, considerations, and questions related to flu vaccination.

MSD found that:

• Only about 50% of children in their school district got flu vaccine in previous years.
• Only about 50% of parents planned to vaccinate their children this coming year.
• Parents who do plan to vaccinate their children want them to be protected from getting sick and to avoid missed school, work, and other activities.
• Parents who don’t plan to vaccinate their children don’t think flu is very serious and have questions about whether it is safe.
• Parents prefer to get health information from medical professionals and vaccination experts.

(For more ideas on gathering formative insight, check out the research section of our blog.)

Chart a Clear Path and Follow it: Process Evaluation

Process evaluation tells you if you did what you set out to do.

Once you have designed your strategy, build in ways to monitor whether activities are being implemented as planned and capture what was done. Some types of evaluation can incorporate monitoring along the way can help identify issues or challenges that may require course correction.

Based on their formative evaluation, MSD decides on the following activities:

1) Featuring a short video from the Centers for Disease Control and Prevention about the importance of flu vaccination on district and school websites

2) Sending home a factsheet from the American Academy of Pediatrics answering common questions about flu vaccination and contact information for the school nurse

3) Putting up posters from the local public health department in school common areas that encourage vaccination and provide details about local flu vaccination clinics

As part of the process evaluation, MSD should then document:

• When each of these strategies were implemented (did they get done in September?)
• How many websites featured the video and where (did district and all schools do this?)
• How many factsheets were sent home (did they get sent to all families?)
• How many posters were displayed at each school and where

Track your Progress: Outcome Evaluation

Outcome evaluation tells you what you accomplished.

Throughout program implementation, document the actions or outputs that occurred as a result of the strategies that were implemented. Measure these against the SMART objectives you set to track your progress and show results.

As part of the outcome evaluation, MSD should track unique views of the video in September, and send out a follow up survey to ask whether parents watched the video, read the factsheet, and saw the posters.

Reaching the Finish Line: Impact Evaluation

Impact evaluation tells you whether you made a difference.

At the end of your program, assess whether your efforts had the impact you intended to support your goal.

As part of the impact evaluation, MSD should ask those parents who did watch, read, or see any of educational products whether those influenced their decision to vaccinate their children this coming flu season. MSD should also compare vaccination rates for the flu season following the program against previous seasons to measure whether rates actually increased.

If the education program was successful in encouraging parents who may not originally have planned to get their children vaccinated to do so, but we don’t see an increase in vaccination rates, there may be issues related to cost or access that need to be addressed.

 

Get in touch with Aparna@devipartners.com to learn more about how we can help you build the types of evaluation into your programs that can demonstrate results and continually improve your programs.  

If you are struggling with how to demonstrate program success with limited resources for evaluation, you are not alone. Fortunately, there are ways to build evaluation into your programs to minimize effort and cost while maximizing insight and impact.

The key to unlocking this puzzle is thinking about evaluation from the start. Not only does this give you a baseline to measure against, it forces you to clearly articulate your objectives, strategy, and intended outcomes (which, as we know, is critical for effective programs). And, if you evaluate along the way rather than at the end, you can course correct if there are any challenges – or opportunities.

Now that you know how to unlock the puzzle, how do you put the pieces together?

What we’ve found most helpful is creating a logic model. It’s a simple yet thorough framework for evaluation. There are many templates and approaches for logic modeling. But, it really boils down to thinking through the following:

1. Goals: Where do you want to be?
2. Intermediate Outcomes: What will it take to get there?
3. Current Context: Where are you now?
4. Inputs: What resources can you put towards your program?
5. Program Activities: What are you going to do?
6. Program Objectives: What will success look like?

Once you have this mapped out, you can determine how to monitor whether programs are being implemented as intended, track progress toward program objectives, and measure movement toward your goals and intended outcomes. Check out this article for a basic overview of evaluation planning.

Your evaluation strategies don’t have to be too resource-intensive.

• Look for ongoing opportunities to collect information and feedback as part of your program activities. Hand out satisfactions surveys, conduct exit interviews, and make use of intake forms.
• Create a system for tracking activities that maintains up to date information on what you’ve done, who you’ve reached, and services/products you’ve provided. If you can’t build a database, a well-designed spreadsheet can do the job. But make it part of of your standard operating procedure.  
• Plan ahead to make sure you have baseline data to measure against so you can demonstrate increases and improvements. Ask participants to fill out quick surveys before and after a training. Gather publicly available data related to your outcomes or add questions to existing surveys if you can’t afford to field your own survey.
• Make sure the information you are gathering is relevant. Don’t waste time and energy collecting data that can’t be used to assess your program or make decisions about it.

BOTTOM LINE: Be clear about where you want to go, and how you plan to get there. Then you can build in ways to monitor whether you are headed in the right direction and on track to reach you destination on time.

Helping organizations (big or small) through this planning process is what we at Devi Partners do best. Contact Aparna@deviparters.com to learn more about our services and approach.

Photo Credit: Ken Teegardin, Creative Commons