Evaluation: Finding the Value
Program evaluations are often undertaken to prove to external stakeholders (usually funders, partners, authorities) that a program or service is “worth” their support. This means providing external stakeholders with evidence of “value” for their investment, whether that investment was made in money, time, energy, or dedication.
In social purpose initiatives, “value” means being able to move the needle on a social challenge that is negatively affecting individuals, families, communities, and social infrastructure. This may include social and environmental, as well as financial or economic components. But how do we describe that value?
Unfortunately, many evaluation methods have been concerned primarily with gathering outputs. Outputs are quite easily counted, like numbers of people, hours, dollars, resources, or social media “likes”. Outputs are like apple seeds, but they do not tell the fuller story of what has actually changed as those seeds begin to grow. For example, we may know how many people participated in a particular program, but do we know the ways in which their knowledge, actions, capacities or lives may have changed as a result of that program? Those kinds of outcomes may tell us more about how families, communities, and social infrastructure are being affected. And what about the wider ripple-effect? To truly understand the impact and value of an initiative, we need to understand how the metaphorical orchard has changed.
The Difference between Outputs, Outcomes, and Impacts
To tell the fuller story of outcomes and impact, we need to go beyond outputs. We need to use a wider lens to look at what has changed not just for individuals but also for their families, community, associated agencies, or society as a whole. This gives us a way to explore the broader social ripple-effect that has occurred because of a program or service. These kinds of discussions can show us a more complex picture about the role of social purpose organizations in our society.
The Power of Prevention
Discussions about outcomes and impacts may also point us to the preventative nature of social support initiatives. Social programs and services are often designed to address specific challenges that, as society, we are trying to reduce or avoid. Often, those challenges are harmful to individuals and families but also very costly to our social infrastructure. Some of the “value” in a program or service may therefore be in the way that it prevents certain problems from happening or from getting worse. Imagine our society if social support organizations didn’t exist! This is an indication of the social value they create. This also means that evaluations need to describe not just the gains made but also the greater challenges prevented, because both are valuable to us as a society.
When the right interventions are available at a critical crossroad, trajectories can change for individuals, families, communities, or societies. To some degree, the change in trajectory is therefore related to what happened at the crossroads; some portion of the value of the changed outcomes and impacts can be attributed to the investment made at the crossroad.
If we look at the outcomes and impacts related to an initiative (both contributions and attributions), we can identify the positives that have been gained by making “pre” and “post” comparisons. However, the avoidance or reduction of the negative is also value-able. The impact of a program or service can also be defined by the difference between what happened because of the intervention (the solid line in the diagram above) and what more commonly happens (the dotted line). Society changes based on these individual, family, community, and societal trajectories.
Think of the efforts to encourage people to stop smoking or to exercise more. Some of those efforts are motivated by concerns about people’s health, but they are also informed by the desire to save the healthcare system money. “Expensive” can be defined in all sorts of ways and our principles and values will inform which costs (social, environmental, financial, economic) we consider to be most important.
Looking for Social Value
Social impact studies therefore look at the following question: What is the impact of this organization’s activities in the social realm? What social value is being created by this program or service?
If we look beyond the immediate outcomes of a program or service, we can see that social programs have a significant ripple-effect that can be witnessed or experienced in organizations, communities, and social infrastructure. Social purpose initiatives play an essential role in society, as much for what they do as for what they help prevent. Social impact studies give us a broader picture of change and can help us find data that supports the development of better policies and planning because we’re looking not just at the program as an object, but as an agent of social change within a larger context.
In social impact studies, it is critical to identify and talk to different kinds of partners and stakeholders. It is critical to ask different kinds of questions, and to do academic research to see whether other studies have come to similar conclusions. It thereby becomes possible to put both monetary and non-monetary investments on the scale alongside a more accurate inventory of outcomes and impacts, not just outputs, that have resulted from an initiative so that social value can be described more fully. If necessary, the financial cost savings, economic benefits, or “return on investment” can be calculated.
Of course, doing a social impact study will also point out ways of improving the program or service, and will help organizations explore ways of improving the type and amount of impact that they have. Social impact studies even make it possible to work towards greater social innovation through collective impact. As a result, it is possible to get a better understanding of individual and collective benefits, and to make improved plans for the future.
- Stakeholder involvement, including broad scoping to identify primary and secondary beneficiaries
- Investigation of “witnessed and experienced changes” using case study methodology, focus groups, unstructured and semi-structured interviews, and participatory research approaches
- Emergent process development to integrate new information as it arises (constructivist grounded theory approach)
- Identification of Crossroads Challenges and alternate story-mapping that uses different kinds of questions to get at social impact
- Creation of an Impact Inventory summarizing key individual, group, community and/or systems changes
- Description of Social Impact Snapshots to illustrate witnessed and experienced social impacts
- Assistance with development of data-gathering strategies to support selected indicators of change
- In some cases, Social Return on Investment calculations may be applied
- Ongoing capacity building for the client through collaborative work, staff presentations and workshops that consciously seek out opportunities for on-going knowledge exchange