Posts tagged ‘non-profit measurement’

March 24, 2010

Shared Measurement Systems for Non-Profits

Attended a great webinar this morning hosted by FSG Social Impact Advisors. They are a non-profit consultancy focusing on social impact measures.

The webinar talked about some of the research they have done in the sector. One of the fundamental challenges for non-profits and funders is that funders naturally want to see the results of their grants and investments. The problem arises that non-profits then need to report back to every funder for every grant in different formats. The FSG research looked at groups that are using shared measures to minimize this impact, save money and learn from each other.

They have identified three main categories of sharing measurment

1. Shared Measurement Systems – A common platform to report different goals and measures. www.successmeasures.org is an example of this type of system for community impact measures with over 200 orgs using the system.

2. Comparative Performance Systems – A system that uses identical measures to compare performance. The Cultural Data Project is an example of this type of system used by thousands of non-profit cultural organizations across the U.S.  And an example of one useful by very small organizations in the cultural sector.

3. Adaptive Learning Systems – A system that uses identical measures to align efforts and goals. Strive is an example of this type of system to connect leaders across sectors around a common vision of education support. The cool thing about this project is that the participating groups really get to learn from each other, identify missing pieces and develop new programs.

What is interesting about all of these categories is that they are sector driven. The benefits of the measurement and data accrue not just to funders (the traditional beneficiaries) but to the sector and the non-profit organizations themselves.

I believe this model is the way of the future for non-profit organizations and measurement. The bottom-up development approach, the focus on “usefulness” of the measures for organizations and the drive for collaboration around measure development are all critical in creating effective measurement systems. I believe this is also a great example of the trend toward openness and collaboration in the non-profit space.

Finally, there is an amazing opportunity here to connect this movement with the open data and open government movements. Shared measurement systems like this can benefit from benchmark and other data held in government systems. For example, Strive is looking to access data on school performance stored in State data systems for comparison.

I’m looking forward to seeing the evolution of these strategies as well as integrating this thinking into the Demonstrating Value Initiative and our own social sustainability reporting.

Share

February 16, 2010

Demonstrating Value – Building a Dashboard for Potluck Cafe

Last year I worked with Heather O’Hara, the Executive Director of Potluck Cafe to put into practice some of the techniques I have been writing about. As you can see from the story below, experience taught both of us a lot about the value of this work and how to make it more effective in the future.

The Potluck Cafe & Catering Society in Vancouver, Canada, one of the city’s most successful social enterprises, operates a cafe and corporate catering service generating revenue of approximately $1 million per year. With this revenue, Potluck is able to provide over 26,000 free meals to residents of the Portland Hotel Society – residents with severe addiction, mental and physical health issues. In addition, Potluck hires and trains local residents, providing meal support, bus passes and life skills coaching.  Even with these successes, however, Potluck was hard-pressed to quantify and articulate the value it was delivering to funders and community stakeholders, and therefore having a difficult time engaging their Board of Directors in effective strategic planning.

Like Potluck, most nonprofit organizations face heavy demands for data reports, studies and proposals showcasing their mission and social cause. Unfortunately, most of these organizations are often stuck needing to spend vast amounts of their scarce time and resources trying to identify, locate and portray the necessary data in a meaningful way on a case-by-case basis. As a result, much reporting is solely anecdotal, hindering an organization’s ability to effectively demonstrate their value and engage in effective planning.

To address this need locally, Vancouver’s Vancity Community Foundation and other funders came together with Potluck and other select local organizations to form the “Demonstrating Value Project”. This initiative was designed to explore frameworks that could better enable these organizations to understand, communicate and assess their financial performance, organizational sustainability and mission-related impact. SAP is a major funder of the project, contributing financial resources, strategic advice, technical resources and software.

Having witnessed the benefit of SAP Business Object solutions in the corporate sector, Potluck’s Executive Director Heather O’Hara suggested exploring SAP Business Objects’ Xcelcius technology to see how it could meet the needs of the Demonstrating Value Project. Heather recognized that introducing visualizations into data reporting could help enable nonprofits to communicate issues and progress towards goals in a “simple, non-technical and engaging format”.

We knew that without skilled support, the technical capability to effectively leverage a solution of this nature was often out of reach for organizations like Potluck, and partnered with them to provide skilled volunteers to establish the needed direction and scope of the project.  This engagement involved an approximate total of 6 weeks of direct collaboration time and was a very iterative process. Because you are telling a story through communicating data, because it’s visual, there naturally has to be a back-and-forth with any client to make sure you  have the right data components in place to tell this right story, and are using the best visual components to paint the desired picture.

Working directly with the Potluck Executive Director, I began with the important non-technical first step in the process: helping the organization understand and flesh out what it really needs to demonstrate, what it wants to measure and what it wants to ultimately demonstrate to its Board of Directors. I then worked with the client to do an inventory of data – gathering and assessing what is often disperse financial information from a variety of accounting, customer service and sales software, to identify where the needed data is stored, in this case QuickBooks, Excel and Survey Monkey, among others. Where the organization did not have ready access to supporting data, I helped Potluck identify categories of information it could gather to demonstrate impact, and put together a plan to ensure that they could easily get that data going forward.  In the case of Potluck, that included helping them show the improvement of job satisfaction and life skills of the local residents employed in their program, in addition to the more direct outcomes of revenue generated and free meals provided. Being able to demonstrate this ‘extended value’ provided by its programs is a critical component of demonstrating Potluck’s overall community value.

Once the Potluck staff worked to export the identified data from the appropriate sources, we were able to then import the resulting spreadsheet into Xcelcius and build a customized, graphic dashboard.

As a result of the skill and tools provided by SAP, Potluck is now able to generate data-driven, graphic, high-impact snapshots of the organization’s financial, organizational and mission-related metrics, and what-if analysis for funding and program decisions. Potluck had entered this project in the hopes of better informing and engaging its Board of Directors, and Heather now laughs to recall that upon seeing the dashboard for the first time, the immediate reaction of her Board of Directors was “Wow! We do all that?” She elaborates that “the dashboard [has become] a great way to communicate to the board both our breadth of programs and depth of impact.” Heather was also happy to see that this tool has become an internal reporting mechanism by which Potluck can measure its accountability.  “The dashboard provides insight to the detailed aspects of our operations. [Having this information] tells people that our organization is innovative, progressive, and a leader in terms of taking on new initiatives like technology and new business practices like dashboarding.”

In addition to the direct benefits of the Xcelcius tool itself, the exercise of identifying, gathering and assessing existing data and the additional support provided by SAP in setting up effective, integrated processes for gathering that data on an ongoing basis has been key for Potluck. From Heather’s perspective, “to not just invest money or even products but your core business skills is incredibly beneficial support for a company to provide” and helps make their other forms of support “more meaningful” as a result of that assistance.

Performing this type of service for nonprofit organizations is an approach for meaningful community investment that also builds on SAP’s core competencies.  Employees have the opportunity to are asked to step outside of their daily tasks to leverage the company’s product expertise while working in a fresh environment, with new and more intimate client interaction.  This type of engagement also offers the company an opportunity for employee growth and product development through the learnings gather on-the-ground from these clients, as well as business development by creating a new pipeline of future clients and references.

With what we have learned, SAP volunteers are currently working on three parallel projects in Vancouver implementing dashboards for non-profits. I’ll post the stories on those as the projects move along.

The Demonstrating Value Project is moving ahead as well with more social enterprises going through the process and training and speaking in Toronto, Scotland and coming up at the Social Enterprise Alliance Summit in San Francisco in April. I will be speaking about our work with my colleagues Bryn Sadownik and Elizabeth Lougheed Green from the Vancity Community Foundation.

Share

February 4, 2010

Question Zero Part 6: Next steps

In the last 5 posts, we have covered a number of components that help you answer, and communicate  your answer, to Question Zero “What, exactly, are we trying to accomplish?”:

  • Choosing the type of story you are trying to tell
  • Ensuring you are speaking the right language for your audience
  • Making sure you are measuring things that support your mission, not just capturing your activities
  • Checking that you are balancing your measures across time and space

As you can see from these items and the previous posts, there is no simple answer to the “perfect” measures that illustrate your answer to Question Zero and your progress to achieving that goal. However, this process and series of questions can  help you get there and ensure you are measuring the “right” things – things that will help you better manage your organization, achieve your goals, communicate your impact to your audience and ultimately make the change you are driving for.

So now, over to you! When I’m doing this presentation in person, I usually split people into groups and we do a short run through this process. I would encourage you to try this on your own. In “real life”, this process needs to include representation from your board, your staff, your volunteers, your clients and your ED. As an exercise though, try this:

  • Choose a project to focus on
  • Decide if you are going to:
  • Communicate
  • Demonstrate
  • Engage
  • Decide on your audience
  • Customers and potential customer
  • Volunteers
  • Potential donors
  • Internal managers and staff
  • Board of directors
  • Check for balance
  • Time and space
  • Financial, mission, organization
  • Find 3 things to measure that support your answer to Question Zero

I would love to hear your thoughts and your answers to this question either by email or commenting here.

February 3, 2010

Question Zero Part 5: Measurement in Time and Space

I am very grateful to Morgan MacDonald from Offsetters for introducing me to the concept of balancing measurements in time and space. This is a very simple idea that provides a powerful check on your reporting.

When looking at your selected measures, you can plot them in a series of circles (you can do this literally or figuratively) related to time period and spatial relationship to your organization.

Measures in time and space

On the time dimension, are you measuring things that are happening right now, this quarter, this month? Or are you also measuring things that are happening soon – over a period of months. And are you attempting to measure things that happen over a longer time period – perhaps years. This is often where the real impact of your work is felt. Phoenix Print Shop in Toronto sends out email questionnaires to the alumni of their youth program six months, one year and two years after leaving the program. This provides a valuable source of data on the success of the program and is useful not only for reporting but in fine tuning the details of the program.

For the space dimension, it is useful to check whether you are reporting on things happening purely inside your organization, which is often the case. When communicating your impact, the relationships you have across the boundary of your organization are important to capture – for example the partnerships you build, the referrals, the collaborations and the joint successes. It is also important to illustrate how you are linking your internal goals to external impacts. Be careful here as it is easy to swing too far in the opposite direction and only concentrate on what is happening outside and ignore your own environmental footprint or hiring processes or efficiency targets.

Next time a final wrap-up and some practical steps for you to take with your own organization.

February 2, 2010

Question Zero Part 4: Goals, Activities and Outcomes

The question of measuring outcomes, defining goals and managing activities can be quite complex and many people have written on this topic. I don’t want to rehash that work or get into a debate about the exact terminology, I don’t think that is important.

What I tell groups I work with is that you do need to be clear on a few key pieces and you can label them however you want.

First – there should be a hierarchy of detail.

  • Start at a high-level mission or vision for an organization such as “strengthen the economy by raising the incomes of poor families” [from A Living Wage for Families]. While not a single measure, this sets the overall framework for your organization.
  • Then getting into specific goals and objectives
  • Next define the specific activities to achieve each goal
  • Finally think about the outcomes of your activities – how is this linked back to your mission

Where most organizations get into trouble with communications is that they focus on measuring activities or outputs not outcomes. And in fairness, this is often easier. It is simpler to measure how many kids went through your after-school reading program than it is to measure the impact this had on family cohesiveness. This is the critical piece of tying your measurement back to supporting your mission using simple measures that can be easily captured.

As an example, I worked with a group of high-school students that organized a conference by high-school students and for high-school students. The conference was designed to get kids involved in the community by volunteering, creating non-profits, working on environmental projects, etc. When we talked about measurement, they immediately focused on the low-level details like how many tonnes of carbon were reduced by the various environmental projects. While useful, what is really interesting is what the group is actually doing is building a community of engaged youth leaders. And the tonnes of carbon doesn’t measure that! Instead, I suggested looking at how many kids came back year after year, how many were still engaged with their projects after two years, how many “alumni” returned to the conference to donate their time. These measures start to get at the social and community value being built.

The process of defining measures is often an iterative one, starting with that high-level and dropping down to the details.  You need to remember to use your mission as a check on the measures you choose – do they really communicate the value you are delivering to the community? Or are you simply measuring your activities without linking that to a meaningful impact?

February 1, 2010

Question Zero Part 3: Speaking the Right Language for your Audience

Last time we spoke about the importance of being clear on the type of story you are telling – to communicate issues, to demonstrate impact or to engage stakeholders.

This time we will talk about getting the language right for your audience. No matter how well you understand your issue, if you are not speaking the right language, your audience just won’t get it. I see this most often with environmental organizations that are heavily research based. All the data is there to prove a point regarding their issue but framed in a way that average citizens cannot understand. Another chronic failing is with financial information that is presented purely for the understanding of accountants.

The Demonstrating Value Project has been an attempt to help social enterprises better deal with this issue – not just in measuring success but communicating and using that information to better manage the organization. I have been very happy to help out the team based a the Vancity Community Foundation with this project over the past few years. There is a wealth of information on the project website, but I want to focus on one important concept: the Lenses

Demonstrating Value Lenses

The Lens concept helps frame the problem of how to best communicate information by asking two important questions: What information is important to show and how will that information be used?

For social enterprises, and in fact most non-profits and social purpose businesses, information can be grouped into three clusters:

  • Business performance looking at the financial health of the organization and revenue (for social enterprises) or donations (for non-profits), costs and expenditures
  • Mission performance demonstrating how the organization is delivering on its mission whether that is around the environment, social justice, food security, etc.
  • Organizational sustainability to illustrate how the organization is training staff, building a board and creating a plan to survive into the future

For each of those categories of information, there are three potential audiences with very different types of information needs:

  • Operational for day-to-day management of the organization. Information at this level needs to be very detailed for example showing inventory levels, staff planning and cash flow
  • Strategic information is most useful for governance of the organization, for example working with boards. Rather than the granular detail needed for making operational decisions, here the focus should be on illustrating progress toward strategic and mission goals.
  • Accountability to stakeholders is a third category of information. Depending on the stakeholder, this information could be for a funding agency showing the success of a project, to a government entity or to the general public. With this type of information, one cannot assume familiarity with the organization so you must ensure to include relevant explanatory context with the information you publish.

While these are not hard-and-fast categories for every piece of data, these lenses provide a very useful framework for presenting your data. Many organizations struggle to find THE perfect number that will meet the needs of all audiences and for all types of data. In fact this is a fruitless quest – THE number does not exist that will satisfy everyone. But if you start with an understanding of who your audience is and what their interests are, you will be able to speak their language and get your point across more effectively.

Next post we’ll sort through the confusing mix of outcomes, objectives, activities, missions, strategies and goals.

[Updated Feb 3:  Replaced lens image with better resolution]

January 30, 2010

Question Zero Part 2: Communicating, Demonstrating and Engaging with Information

Continuing from last post, let’s get into the details of how to measure and communicate your success.

The question to think through is “What kind of story are you trying to tell?I use the word story deliberately – many people forget that any time you are presenting a number or report there is a reason behind that, a story you are trying to tell.

Are you trying to justify additional budget for a new project? Are you trying to show your board how well you are doing? Do you want buy-in to launch a new and innovative service? It is critical to think through these issues before jumping straight to looking at measures, metrics and dashboards.
There is a telling Dilbert that illustrates the perils of NOT doing this:
Dilbert.com

So let’s assume you want your audience to hear your story, not just be seduced by pie.

There are three genres of stories that non-profit organizations tell:

  • Communicating issues
  • Demonstrating impact
  • Engaging stakeholders

Communicating issues

Some organizations have a primary focus of raising awareness of issues such as child poverty, violence against women or food security issues. Many of these organizations come from a policy and research background, investing time and energy in creating white papers, policy documents and background briefings. The challenge is that if the information on issues is buried in PDF documents, the only people that read them are the people that already agree with the issue – missing the people that don’t yet know about or understand your cause.

In these cases, what is best is to pull out simple metrics that illustrate the issue in real terms, preferably comparable to examples that resonate with your audience (more on audiences next time).  The Living Wage campaign is attempting to do this with their stats on poverty in BC:

  • BC has the highest child poverty rate in Canada
  • BC is the only province where child poverty rates were actually higher in 2006 than in 1997
  • BC is one of only two Canadian provinces where median earnings for individuals fell between 2000 and 2005
  • More than half of BC’s poor children live in families where at least one person has a full-time job

I like the last point best as it has an element of shock behind it and a “I didn’t know that” element which also directly supports the message of the campaign that poverty is not just about unemployment and welfare, it’s about wages. Personally I would recommend moving these stats higher (i.e. above the fold) on the page but that’s another topic.

Demonstrating Impact
The second category of stories happens when you (or others in your space) have succeeded in communicating the importance (or at least the existence) of your issue and now you need to demonstrate why your organization in particular is best suited to do something about it.

As with metrics and measures around communicating impact, best choices are simple and direct. It is often tempting to show an extended chain of impact measures but that is usually too confusing for your stakeholders to grasp immediately. An example from United We Can illustrates some good ways to do this, along with some areas to improve the communications.

From a Vancity presentation, here are some amazing impact measures from United We Can in Vancouver:

Results for 2008:

-150 jobs created
-20,000,000 containers recycled
-$2,000,000 put back in the hands of local binners
Now these are great number, very clear and the sheer scale is impressive. What would make this even more compelling is the addition of some comparisons (e.g. what proportion of containers in Vancouver was this? 20%? 50%?) and some performance to target measures (e.g. $2m was 15% our 2008 target and an increase of 30% over 2007).
Unfortunately, as with Living Wage, these numbers are hidden from the web site and therefore not used in communications vehicles. However, they do now have the Vancity video posted which is a great start!
Engaging Stakeholders
Once you have succeeded in communicating your issue and demonstrating why your organization is best able to do something about it, the final step is to engage your stakeholders. At the end of the day, you want your constituents to take action – to write a letter, to volunteer, to donate, to join a march, to change their lightbulbs – not just listen.
This is often the most challenging part of a non-profit’s work, especially in the environmental sector. Ordinary citizens are often overwhelmed with the scope of the issue and skeptical that their own individual actions can make a difference. The key here is to make a concrete link to an individual action and collective impact. This can be done by highlight the impact if 100 of that individual’s friends also took action or to show the impact of actions over time.
FINCA has done a good job of this in the microfinance space with their Village Bank Donation Calculator. This calculator shows some simple impacts such as the total amount of money generated from small loans, the cumulative value over time and most importantly, the lives impacted. I find this to be a great example of helping an individual understand that even a small action can have a large impact.
Next time, we’ll address the audience for these stories – what type of information they are looking for and how to make sure you are speaking the right language.
January 29, 2010

Answering Question Zero: What is success and how do we measure it?

Herman Leonard, a professor at Harvard Business School, has a concept called Question Zero. This asks “What exactly are we trying to accomplish?” The idea being if you can’t answer Question Zero, then don’t bother with questions 1 through 99!

For social ventures and non-profits this can be a difficult question to answer simply and communicate succinctly. Especially challenging is the need to manage, measure and balance financial success, mission or community impact and ongoing organizational sustainability.

I have been writing and speaking about this topic for a while – at Social Venture Institute at Hollyhock and Social Tech Training in Toronto, with the Demonstrating Value Project – and most recently at the Vancouver Net Tuesday event in November.

Over the years  I have developed some techniques, and incorporated the best I have found, to help non-profit, social enterprises and social purpose businesses answer this question. Ironically, through another series of questions! Questions like:

  • What kind of story are you trying to tell?
  • Who is the audience?
  • Are you speaking the right language for that audience?
  • Are you measuring your mission or your activities?
  • Do you have the right balance between time and space in your measures?

Over the next few posts I will be going into more detail on each of these questions. For now I’m very grateful to Guacira Naves from Online Strategy for posting a video recording of the presentation here. Many, many thanks Guacira!

Part 1: http://onlinestrategy.ca/2009/11/05/vancouver-net-tuesday-november-2009/
Part 2: http://onlinestrategy.ca/2009/11/06/vancouver-net-tuesday-november-2009-part-ii/
Part 3: http://onlinestrategy.ca/2009/11/06/vancouver-net-tuesday-november-2009-–-part-iii/
Part4: http://onlinestrategy.ca/2009/11/09/final-video-segment-steve-williams-at-net-tuesday-vancouver/

Follow

Get every new post delivered to your Inbox.