RU R A L D E V E LO PM E N T I N S T I T U T E K N OW L E D G E B R I E F FA L L 2016 KNOWLEDGE BRIEF Poverty Reduction Part II: Using Indicators to Support Poverty Reduction INTRODUCTION WHY ARE INDICATORS IMPORTANT? Poverty reduction efforts are increasingly reliant on the use of indicators to measure the complex dimensions of poverty and evaluate the results of actions taken. The use of indicators is explained in part by the growing need to make a compelling case to policy-makers, funders, and other investors, as well as to demonstrate accountability for the effective use of funds invested in social initiatives.1 Additionally, poverty reduction advocates highlight a number of other reasons to measure poverty, such as raising awareness and maintaining focus on the issue, dispelling myths about poverty and its origins, and informing communities about relevant trends to support actions that are targeted and appropriate.2,3 Though the term “indicator” is used widely, as are “measurement” or “metric”, it can be used to refer to multiple different, but related ways to track variables: monitoring particular social or economic conditions; identifying trends within a population; setting targets or goals for specific programs or policies or evaluating their success; or establishing benchmarks for comparison with other communities. See Poverty Reduction Part III: Identifying Key Indicators for Poverty Reduction for a more detailed discussion about the different scale and focus of indicators. MODELS & FRAMEWORKS FOR USING INDICATORS TO MEASURE POVERTY REDUCTION Many communities and agencies are drawing on established frameworks to support incorporating indicators into their poverty reduction efforts. One such example is the Social Return on Investment (SROI) model, which is used to measure value in a broader sense, beyond purely financial terms, using monetary values to represent social, environmental, or economic outcomes.4 An SROI calculation gives a community or agency the ability to reflect the costs (e.g., investment in a social program) as a ratio of the benefits (e.g., the social value of the program). When the ratio shows an outcome value higher than investment value it supports the case for the positive impact of an initiative. As an example, the Region of Waterloo used SROI to calculate the potential benefits of a regional living wage policy, finding that the total social value of the policy would be $1.80 for every dollar invested.⁵ Like the SROI, the Genuine Progress Indicators (GPI) approach looks at a broad range of economic, social, and environmental indicators, and represents their total value, using monetary figures.6 The strength of this approach is its ability to bring together diverse indicators into a single, aggregate measure that can be used to compare overall well-being in different regions. Although the GPI approach was designed primarily for large-scale use (e.g., at the national level), 1.888.953.1133 www.cbrdi.ca KNOWLEDGE BRIEF P OV E R T Y R E D U C T I O N PA R T I I recently interest has turned to ways the GPI may be applied at the local level,⁷ with examples of local level GPI assessments completed in Edmonton and Atlantic Canada.8,9 One challenge of the approach is that it requires access to a large number of indicators across many dimensions of community well-being and this data may not be available for rural communities.10,11 However, it may still be of value for communities including poverty reduction as a component in a comprehensive community planning process. The Vibrant Communities Canada: Cities Reducing Poverty approach is increasingly popular, with dozens of Canadian municipalities now participating in the collective impact movement.12 Shared measurement, or the use of consistent indicators across partners, is one of the essential tenets of the collective impact model, highlighting the importance of measuring progress against reliable benchmarks and communicating these results regularly to the community.13 Additionally, Vibrant Communities draws many of its principles from the Developmental Evaluation approach. Developmental evaluation recognizes that social initiatives are continually developing and adapting to unpredictable social conditions, and relies on a continuous and simultaneous process of planning, action, and evaluation to support innovative thinking in complex, shifting contexts.14,15 The Health Equity approach taken by the Saskatoon Health Region (SHR) has attracted attention as one of the top inspirational approaches to poverty reduction in Canada.16 A ground-breaking study, released in 2008, described the extent of disparity in health outcomes by socio-economic status in the Saskatoon community, which was in some cases shocking in its magnitude for a city in the western world.17 The report also identified evidence-based policy options to prevent health disparities, resulting in a new approach by the SHR in which “health equity became both a tool to build programs and a means to measure program outcomes.” 18 New research is available to support a health equity approach to poverty reduction in other communities. The Canadian Institute for Health Information (CIHI) recently released a detailed report that examines trends in 16 health indicators over time and across five income levels,19 and provides an interactive e-tool that allows users to explore these trends at the provincial level.20 Within British Columbia, the Provincial Health Services Authority recently released an updated analysis of 52 priority health equity indicators, which in most cases includes a breakdown by Local Health Area (LHA) or Health Service Delivery Area (HSDA).21 The Sustainable Livelihoods Framework uses assetmapping as a measurement tool, identifying individual or community assets in five categories (i.e., financial, social, 2 Rural Development In s t i t u t e K N OW L E D G E B R I E F Fa ll 2 0 1 6 personal, physical, and human), determining a baseline, and regularly reassessing identified assets for change relative to the baseline over the length of a program.22 The Sustainable Livelihoods model was applied by Thunder Bay’s PARO Centre for Women’s Enterprise, a not-for-profit organization that provides multi-faceted, holistic programming focused on enhancing the economic independence of women and their families across Northwestern Ontario. The Canadian Community Economic Development Network’s (CCEDNet) PlaceBased Poverty Reduction (PBPR) initiative studied the effectiveness of PARO’s approach to measurement, finding that in addition to demonstrating that all research participants had increases in all asset categories, the approach also gave the participants a quantifiable way of assessing changes in their own lives.23 CRITERIA FOR SELECTING INDICATORS Given the importance of integrating indicators into poverty reduction efforts, communities and agencies need to give attention to selecting appropriate indicators. Many may choose to follow an existing framework, like those detailed above, to structure their approach to defining indicators. However, the specific needs of each community are also important considerations. A research study conducted with rural women living in poverty in Ontario’s Grey, Bruce, and Huron Counties developed a Community Report Card approach that emphasized the importance of defining the information needs of the community. The toolkit recommends asking questions such as: “What information would influence new and more positive attitudes in the community? What information would help the community develop strategies for change?... What information would help to engage people and support action?... What do we have to do to ensure that we involve the full diversity of our community in the information gathering process?”24 The literature suggests that combining poverty indicators can improve the validity of results when evaluating programs.25 A recent survey of approaches to measuring poverty highlighted that while most approaches incorporated income measures, none used income as the sole measure.26 Other important similarities across these approaches include measurement of multiple indicators across a number of dimensions of poverty; a combination of local data collection and national statistics; community specific measures relevant to unique programs; and measurement of change from the perspective of the community and individuals living in poverty.27 The Canadian Centre for Policy Alternatives argues that a comprehensive poverty reduction plan for British Columbia must include carefully selected indicators, suggesting: “No one measure/indicator is enough to establish and monitor a comprehensive poverty reduction plan. An effective plan must track the breadth, depth and duration of poverty; it must have indicators that focus on those populations where poverty is most acute and persistent; and it requires multiple indicators of hardship and financial stress (covering homelessness, housing insecurity, food insecurity, etc.)."28 For the Region of Waterloo, the following criteria were used to select indicators to monitor their comprehensive approach to poverty reduction: “relevancy to poverty; how measurable the indicator; whether the indicator was evidence-based; whether the data and data source were reliable; whether the indicator was statistically sound; whether the data were available in a timely manner; whether a provincial and national comparator was available; and whether the data were accessible at the Regional level.”29 Similarly, the Saskatoon Poverty Reduction Partnership defined their indicator selection criteria as follows: “that reporting will occur at least annually; that indicators relate to the theme areas we're monitoring and evaluating; and that they are crucial to the well-being of the community.”30 LIMITATIONS & CHALLENGES One concern with the increasing reliance on indicators is the risk of selecting false or misleading measures. A prime example of a misleading indicator is a reduction in the number of individuals receiving income assistance because the indicator does not track the reasons people may be leaving or denied access. It also does not measure whether or not those individuals remain poor.31 Food bank usage is another such example, often used as an indicator of food security levels in a community. Yet, research shows that food bank usage reflects only a subset of the food insecure population and is not representative of the food insecure population.32,33 Additionally, there is a concern that the use of indicators may negatively impact the way programs are selected, favouring programs that have easily quantifiable outcomes.34 Particularly in rural communities, the availability of locallevel data and resources for data collection may pose serious concerns and challenges. The time and expense of collecting data may draw resources away from time spent on direct poverty reduction work. One participating organization in the CCEDNet PBPR initiative observed that embedding data collection procedures in programs can make quantitative measurement “relatively painless”, yet this did not address the concern that pre-intervention indicators were absent, making it challenging to establish appropriate baselines.35 However, recent research examines twenty promising efforts to develop shared approaches to measuring outcomes in the non-profit sector, such as new technologically enabled shared measurement platforms, arguing that these innovative collaborations may create significant changes in effectiveness for the entire sector.36 REFERENCES & RESOURCES 1. Paul Chamberlain, “Place-Based Poverty Reduction Initiative: How Community Economic Development Is Reducing Poverty in Canada and How It Could Be Doing More” (Victoria, BC: The Canadian CED Network, 2008). 2. Region of Waterloo Social Services, “Poverty Indicators in Waterloo Region: Status Report 2014” (Waterloo, ON: Social Planning, Policy and Program Administration, Region of Waterloo Social Services, 2014), http://communityservices. regionofwaterloo.ca/en/communityPlanningPartnerships/ resources/PovertyIndicatorsinWaterlooRegion.pdf. 3. Colleen Purdon, “Counting Women In: A Toolkit for Rural Action on Poverty” (Rural Women Take Action on Poverty Committee, June 2009), http://vibrantcanada.ca/files/ counting_women_in_toolkit_final.pdf. 4. Tim Goodspeed et al., “A Guide to Social Return on Investment” (London: New Economics Foundation, 2009), http://www.neweconomics.org/sites/neweconomics.org/ files/A_guide_to_Social_Return_on_Investment_1.pdf. 5. SiMPACT Strategy Group, “Social Return on Investment (SROI): A Living Wage Policy for the Regional Municipality of Waterloo” (Waterloo, ON: Region of Waterloo Social Planning, Policy and Program Administration, 2009), http://communityservices.regionofwaterloo.ca/en/ communityPlanningPartnerships/resources/DOCS_ADMIN726143-v2-FINAL_LIVING_WAGE_SROI_CASE_STUDY_. pdf#Final Living Wage SROI Case Study. 6. Kenneth Bagstad and Marta Ceroni, “Opportunities and Challenges in Applying the Genuine Progress Indicator/ Index of Sustainable Economic Welfare at Local Scales,” International Journal of Environment, Workplace and Employment 3, no. 2 (2007): 132–53. 7. Jeffrey Wilson and Peter Tyedmers, “Rethinking What Counts. Perspectives on Wellbeing and Genuine Progress Indicator Metrics from a Canadian Viewpoint,” Sustainability 5, no. 1 (January 14, 2013): 187–202. 8. Mark Anielski and Heather Johannessen, “The Edmonton 2008 Genuine Progress Indicator Report: The State of Economic, Social and Environmental Wellbeing for the City of Edmonton” (Edmonton, AB: Anielski Management Inc., November 11, 2009), http://www.anielski.com/wp-content/ docuploads/Edmonton%20GPI%202009%20Report.pdf. 9. GPIAtlantic, “Community GPI,” GPIAtlantic: Genuine Progress Indicators for Atlantic Canada, 2014, http://www. gpiatlantic.org/community.htm. 10. Wilson and Tyedmers, 2013. 11. Bagstad and Ceroni, 2007. 12. Brock Carlton and Paul Born, “10 - A Guide to Cities Reducing Poverty” (Tamarack Institute, 2016). KNOWLEDGE BRIEF U S I N G I N D I C ATO R S TO S U P P O R T P OV E R T Y R E D U C T I O N Ru ral D e v e l op m e n t I n stitu te K N O W L E D G E B R I E F Fall 2 0 16 3 P OV E R T Y R E D U C T I O N PA R T I I : U S I N G I N D I C ATO R S TO S U P P O R T P OV E R T Y R E D U C T I O N REFERENCES & RESOURCES CONTINUED 13. 14. 15. Brian Reid, Mark Cabaj, and Louise Kearney, “Convening a Comprehensive, Multi-Sector Effort to Reduce Poverty: A Primer 2nd Edition” (Waterloo, ON: Tamarack Institute, 2013), http://www.tamarackcommunity.ca/library/ convening-a-comprehensive-multi-sector-effort-toreduce-poverty. Jamie A.A. Gamble, “A Developmental Evaluation Primer” (Montreal, QC: The J. W. McConnell Family Foundation, 2008), http://tamarackcci.ca/files/developmental_ evaluation_primer.pdf. Liz Weaver, Paul Born, and Denise Whaley, “Approaches to Measuring: Community Change Indicators” (Waterloo, ON: Tamarack Institute, 2010), http://www. tamarackcommunity.ca/library/approaches-to-measuringcommunity-change-indicators. 22. Weaver, Born, and Whaley, 2010. 23. Chamberlain, 2008. 24. Purdon, 2009. p.33-34 25. Geranda Notten, “How Poverty Indicators Confound Poverty Reduction Evaluations: The Targeting Performance of Income Transfers in Europe,” Social Indicators Research 127, no. 3 (June 24, 2015): 1039–56. 26. Weaver, Born, and Whaley, 2010. 27. Weaver, Born, and Whaley, 2010. 28. Seth Klein et al., “A Poverty Reduction Plan for BC” (Vancouver, BC: Canadian Centre for Policy Alternatives, BC Office, 2008), 30. 29. Region of Waterloo Social Services, “Poverty Indicators in Waterloo Region: Status Report 2014.” 16. Carlton and Born, 2016. 17. Mark Lemstra and Cory Neudorf, “Health Disparity in Saskatoon: Analysis to Intervention” (Saskatoon, SK: Saskatoon Health Region, 2008), https://www. saskatoonhealthregion.ca/locations_services/Services/ Health-Observatory/Documents/Reports-Publications/ HealthDisparityRept-complete.pdf. 30. Saskatoon Poverty Reduction Partnership, “Indicators of Outcome Progress,” accessed August 23, 2016, http:// www.saskatoonpoverty2possibility.ca/outcomesprogress.html. 31. Klein et al., 2008. 18. Carlton and Born, 2016, p.25. 32. 19. Canadian Population Health Initiative, “Trends in IncomeRelated Health Inequalities in Canada: Technical Report” (Ottawa, ON: Canadian Institute for Health Information, 2015), https://secure.cihi.ca/free_products/trends_in_ income_related_inequalities_in_canada_2015_en.pdf. Rachel Loopstra and Valerie Tarasuk, “Food Bank Usage Is a Poor Indicator of Food Insecurity: Insights from Canada,” Social Policy and Society 14, no. 3 (July 2015): 443–55. 33. Na Li et al., “Priority Health Equity Indicators for British Columbia: Household Food Insecurity Indicator Report” (Toronto, ON: Research to Identify Policy Options to Reduce Food Insecurity (PROOF), 2016), http://proof. utoronto.ca/. 34. Chamberlain, 2008. 35. Chamberlain, 2008. 36. Mark Kramer, Marcie Parkhurst, and Lalitha Vaidyanathan, “Breakthroughs in Shared Measurement and Social Impact” (FSG Social Impact Advisors, 2009), http:// vibrantcanada.ca/files/breakthroughs_in_shared_ measurement_complete.pdf. 20. 21. Canadian Institute for Health Information, “Health Inequalities Interactive Tool,” Canadian Institute for Health Information, June 2, 2016, https://www.cihi.ca/en/factorsinfluencing-health/socio-economic/health-inequalitiesinteractive-tool. Provincial Health Services Authority, “Priority Health Equity Indicators for British Columbia: Selected Indicators Report” (Vancouver, BC: Provincial Health Services Authority, Population and Public Health Program, 2016), http:// www.phsa.ca/population-public-health-site/Documents/ Priority%20health%20equity%20indicators%20for%20 BC_selected%20indicators%20report_2016.pdf. The Columbia Basin Rural Development Institute, at Selkirk College, is a regional research centre with a mandate to support informed decision-making by Columbia Basin-Boundary communities through the provision of information, applied research and related outreach and extension support. www.cbrdi.ca 1.888.953.1133