Shaping evaluation of policies and programs impacting First Nations people –Consultation listening series

Limited progress in Closing the Gap over the past decade has led to increasing calls for government to partner with First Nations people and communities to build an evidence base around ‘what works’ in achieving outcomes. As a step towards building this evidence base, NSW Treasury has committed to develop a culturally responsive First Nations policy and investment framework by June 2024. Evaluation is one pillar of this framework1 and forms part of the NSW Implementation Plan for Closing the Gap 2022-24. 

To inform this work, NSW Treasury consulted with a broad range of stakeholders to gain a deeper understanding of current evaluation practice relating to First Nations policies and programs across both government and non-government sectors. While the consultation series focused on evaluation of policies and programs impacting First Nations people and communities, much of the feedback was found to be applicable to evaluation practice generally. This paper reports back on what NSW Treasury heard through this process.

Overall, feedback from the consultations highlight that there are a number of capacity, capability and cultural factors currently limiting both the number of evaluations and the effectiveness of evaluation practice in New South Wales. This includes a number of unique factors which are impacting the effectiveness and appropriateness of evaluation for First Nations people and communities. At a high level, the consultation series found that: 

• There is large variation in evaluation capacity and capability across the NSW Government sector.
• Very few agencies within the sector expressed high levels of confidence or expertise in evaluation of First Nations policies and programs.
• Only a limited number of agencies conduct evaluation in-house, with most outsourcing in whole or in part to third parties through the tendering process.
• There is a significant opportunity and growing demand for First Nations owned research and evaluation services.
• There is a lack of consistent evaluation culture across government and a need to improve systems for accountability and transparency.
• Creating avenues for embedding evaluation into program design from the beginning represents a significant opportunity to improving evaluation quality and creating a culture of evaluation.
• Current evaluation practice often does not effectively or meaningfully include First Nations people in its design or in the assessment of what success looks like.
• There is no consistent approach to how evaluations are funded across government, often leading to lack of resources to perform evaluation and to perform high quality evaluation.
• Evaluation across much of the sector continues to be focused on outputs rather than outcomes.
• A set of foundational principles to guide evaluation of programs that impact First Nations people and communities would help to centre evaluation through a First Nations lens.
• Cultural capability has been identified as an area for improvement across the public sector, including the recognition of the cultural and structural biases that are embedded into mainstream evaluation practice.

What we have heard will be used to inform the development of a First Nations policy and investment framework and will complement the existing suite of NSW Treasury investment frameworks and policy guidelines.