Every organization, big or small, strives to succeed in achieving their mission, and doing it as efficiently as possible. For that to happen, you need know where you are going and chart a clear route to get there. Evaluation can help you make sure you stay on the right path, reach the finish line when and how you intended, and can show others what you’ve achieved.
Why is evaluation important?
- Your mission matters. When you succeed, you aren’t just meeting organizational goals; you are contributing to the greater good. And we need all hands on deck right now.
- Your story matters. Building evaluation into programs allows you to capture your successes and document how you created impact. Not only does this better position your organization, sharing your approach and how you overcome challenges can spark ideas and help others working toward common goals.
- Insight matters. Taking the time to learn what works – and what doesn’t – makes programs stronger and organizations more efficient and effective. Certain types of evaluation can provide insight on an ongoing basis that allows you to make adjustments along the way, rather than finding out that you didn’t meet your goals at the end.
- Results matter. Funders, donors, and other stakeholders want to know that the money and time they are investing in your organization and programs are having measurable impact. You need to show them that you are making a difference.
And here’s how you do it.
Start with the Finish: Evaluation Planning
Set SMART objectives and identify the outcomes that will help you reach your goals. These paint a clear picture of your finish line, so you know where you are going and can assess when you’ve reached it.
Imagine that the (fictitious) Motunui School District (MSD) wants to increase flu vaccination rates to help reduce the number of students that miss school during flu season. Their SMART objective is to educate all 300 families in the district during the month of September about the importance of flu vaccination for their children.
Survey the Terrain: Formative Evaluation
Formative evaluation (also called formative research) tells you what needs to get done and how best to do it.
Take the time to better understand existing strengths and needs, identify relevant and effective strategies, and determine factors that can facilitate success. This insight supports strategy and program development, and can also serve as a baseline to measure against at the end of your program.
There are a number of tools MSD could use for formative evaluation. They chose the following:
- Records review: They gathered vaccination rates from past years.
- Survey: They surveyed parents on whether they plan to vaccinate their children this coming flu season, the main reasons why they may not, and how they prefer to get health information.
- In-depth Interviews: They interviewed a diverse sample of parents across the district to understand their motivations, barriers, considerations, and questions related to flu vaccination.
MSD found that:
- Only about 50% of children in their school district got flu vaccine in previous years.
- Only about 50% of parents planned to vaccinate their children this coming year.
- Parents who do plan to vaccinate their children want them to be protected from getting sick and to avoid missed school, work, and other activities.
- Parents who don’t plan to vaccinate their children don’t think flu is very serious and have questions about whether it is safe.
- Parents prefer to get health information from medical professionals and vaccination experts.
(For more ideas on gathering formative insight, check out the research section of our blog.)
Chart a Clear Path and Follow it: Process Evaluation
Process evaluation tells you if you did what you set out to do.
Once you have designed your strategy, build in ways to monitor whether activities are being implemented as planned and capture what was done. Some types of evaluation can incorporate monitoring along the way can help identify issues or challenges that may require course correction.
Based on their formative evaluation, MSD decides on the following activities:
1) Featuring a short video from the Centers for Disease Control and Prevention about the importance of flu vaccination on district and school websites
2) Sending home a factsheet from the American Academy of Pediatrics answering common questions about flu vaccination and contact information for the school nurse
3) Putting up posters from the local public health department in school common areas that encourage vaccination and provide details about local flu vaccination clinics
As part of the process evaluation, MSD should then document:
- When each of these strategies were implemented (did they get done in September?)
- How many websites featured the video and where (did district and all schools do this?)
- How many factsheets were sent home (did they get sent to all families?)
- How many posters were displayed at each school and where
Track your Progress: Outcome Evaluation
Outcome evaluation tells you what you accomplished.
Throughout program implementation, document the actions or outputs that occurred as a result of the strategies that were implemented. Measure these against the SMART objectives you set to track your progress and show results.
As part of the outcome evaluation, MSD should track unique views of the video in September, and send out a follow up survey to ask whether parents watched the video, read the factsheet, and saw the posters.
Reaching the Finish Line: Impact Evaluation
Impact evaluation tells you whether you made a difference.
At the end of your program, assess whether your efforts had the impact you intended to support your goal.
As part of the impact evaluation, MSD should ask those parents who did watch, read, or see any of educational products whether those influenced their decision to vaccinate their children this coming flu season. MSD should also compare vaccination rates for the flu season following the program against previous seasons to measure whether rates actually increased.
If the education program was successful in encouraging parents who may not originally have planned to get their children vaccinated to do so, but we don’t see an increase in vaccination rates, there may be issues related to cost or access that need to be addressed.
Get in touch with Aparna@devipartners.com to learn more about how we can help you build the types of evaluation into your programs that can demonstrate results and continually improve your programs.