By Jessica Tufte, MPH
Assistant Director of Evaluation & Analytics
Does this example match one of your least favorite formats to read at work?
Ours too! There is light at the end of the tunnel because the age of text-heavy reporting continues to come to an end. Over the years, clients have told us and other industry-leading evaluation firms that text-heavy reports are simply not what they want or need.
They didn’t always say it directly (sometimes they did!), and sometimes they didn’t even realize it. So how did we know when a text-heavy report was not what they needed?
Ann Emery of Depict Data Studio nailed it when she pointed out the following hints:
- No response: No response at all after the report is delivered.
- Promise to follow up later. You received a response that says, “Thanks, I’ll let you know if I have any questions.”
- “Compliments.” You receive a response that says, “Thanks we can tell that a really technical team worked on this report, it’s very detailed and thorough.”
- Won’t read it. They ask for another format entirely like a slideshow, one-pager, etc.
We’re adding another hint that we used to come across in our work with clients:
- No Observable Action. We didn’t observe any action taken by the organization based on what was suggested in the evaluation report.
This is where “utilization-focused evaluation” comes into play, and it’s a concept that we’ve fully embraced. What does this term mean?
Micheal Quinn Patton, a founding-figure in the field of evaluation, coined the term “utilization-focused evaluation” with his pivotal book of the same name. The approach of utilization-focused evaluation judges an evaluation, no matter how it’s delivered, on this question, “Did the intended user actually use this evaluation?”
If the answer is no, as communicated explicitly or as implied in the occurrence of any of the hints above, it’s time to rethink the delivery method of the evaluation.
So what can be done? We’ve talked with a number of clients who have shared the formats in which they prefer to have their evaluation results delivered:
- Inclusion of a highly visual one-page summary that can stand alone without the rest of the report. (We now do this on almost every contract.)
- More colors!
- Addition of intentional page breaks.
- Fewer words per page paired with relevant visual cues like icons, pictures, or graphs.
- Shorter reports with appendices that include any additional details.
The switch to these types of reports has also led us to explore a myriad of resources including Ann Emery’s Depict Data Studio, Stephanie Evergreen’s Data Visualization tools, books by people like Michael Quinn Patton, among many others. We encourage you to check them out! If you can’t get enough…let us know and we’ll share our highly visual and usable resource lists.
Could you use a program evaluation that makes it easy to know what to do next? Are you interested in some examples of what that might look like? Visit our website or reach out to someone on our evaluation team (Jessica Tufte at jtufte@ccnyinc.org or David Monroe at dmonroe@ccnyinc.org) – we love thinking about new evaluation designs and collaborating with people in the community who do great work to improve the the lives of others.