John writes about his experience at TARA applying a very creative and effective measurement tool called Measuring Significant Change. The process involves involves collecting significant stories from those involved a program–in this case, from the boys residing at the shelter. Through a process of selecting which ones reveal the most significant change, the impact of a program begins to reveal itself against the rich backdrop of complex variables at play.
September 2012,
On a recent Air India flight to London I watched the movie ‘The Best Exotic Marigold Hotel.’ There wasn’t much else and in fact I had snubbed it on the outward flight. To my surprise I really enjoyed it. Why? It was a profusion of high energy, colour, assorted cultures, and eccentric characters all rolled together in a richly exotic Indian setting. And best of all, the ending wasn’t made in Hollywood.
But what has this got to do with evaluation? Well the answer lies in the energy, enthusiasm, and teamwork that I unexpectedly found in a recent evaluation project that as it happens, also took place in India. It’s not often we get to do what we really love to, much less in an exotic location and with persons who are just as passionate about evaluation as you are.
How it all started
GoPhilanthropic is a US-‐based Foundation that connects donors to carefully selected grassroots organisations in ‘developing’ settings. Tara Homes is one of these organisations, a tiny NGO that provides shelter for vulnerable children in New Delhi, India.
Tara is both pragmatic and forward-‐looking in how it works. Not to be satisfied with the inspiring results it already delivers, Tara wants to more fully understand the impact of its projects and in addition to be truly accountable to its donors and sponsors. Despite the logical worries of the time and energy it might take, Tara not only bought into evaluation but also the need to ‘up-‐ skill’ its staff in ways to conduct evaluation.
Ceres Management through GoPhilanthropic was offered the opportunity to help Tara achieve its evaluation goals. And like the success of the ‘Marigold Hotel, ’ Tara embodied the same critical ingredients: energetic, full of hope, unafraid, collaborative, and willing to ‘have a go’.
Why Most Significant Change (MSC)?
Evaluating in unconventional settings…
Evaluating the impact of conventional training & development programmes is usually fairly straightforward. We are trained to begin with a set of pre-‐determined objectives and success indicators that makes evaluating reasonable straightforward. Programs like TARA however are not conventional. They operate within loose and sometimes chaotic boundaries where unintentional, complex and diverse individual outcomes are the norm. Obviously, this poses a huge challenge to traditional evaluation techniques.
To measure unpredictable and potentially hidden impact we selected the Most Significant Change (MSC) technique. Although MSC is a radical departure from conventional evaluation methods it was chosen for Tara specifically because it is story-‐based, is easy to understand, requires no special skills, and can deliver a rich picture of what’s really happening.
So what happened?
conducting the 2-day MSC workshop |
Stay tuned for the results!