NSW Treasury Evaluation Guidelines

• Monitoring and evaluation framework (PDF 732.96KB): A structured approach that plans for and guides monitoring activities and supports evaluation over the life of an initiative.
• Evaluation management and resourcing (PDF 535.43KB): Supporting guidance on practical elements of delivering an evaluation, including how to manage stakeholders, set up a governance structure and source the right expertise.
• Logic models (PDF 1.62MB): Guidance across the investment lifecycle, from building a logic model for business case development to refinement for evaluations.
• Evidence in evaluation (PDF 375.96KB): Guidance on types of data, sources, collection tools and methods that support creating evaluation evidence.
• Outcome evaluation design (PDF 619.91KB): Key features, uses, strengths and limitations of different outcome evaluation designs.
• Sampling strategy (PDF 419.51KB): Key considerations when developing a sampling approach and sample size.
• Ex-post cost benefit analysis (PDF 603.57KB): Guidance on cost-benefit analysis during implementation or after a project has completed.
• Glossary of terms (PDF 146KB).

• Logic models templates (PPTX 4.53MB)
• Evaluation plan template (PDF 501.45KB)
• Evaluation report template (PDF 458.77KB)

E-learning

The Investment Framework e-learning modules are designed to provide users with a basic understanding of monitoring and evaluation, along with related Investment Framework policies. 

Quality assessment tools

The process and outcome evaluation and ex-post CBA quality assessment tools are additional resources that may be used to assess and improve the quality of evaluations. 

Discover our quality assessment tools.

The Evaluation Guidelines TPG (PDF 1.66MB) outlines mandatory requirements and recommendations for evaluating government programs. These evaluations should follow the guidelines to ensure they are high quality and transparent. Evaluations of government resourced ongoing and new initiatives should be prioritised, planned, conducted and reported with reference to the requirements, recommendations and guidance set out in the guidelines.  

The guidelines are supported by technical notes and templates to help with evaluations and provide information to support monitoring and evaluation. The resources provide detailed guidance on evaluation methods and best practices. The evaluation plan and report templates can be adapted to fit different programs and contexts.

A logic model shows how an initiative is supposed to work. It highlights the main inputs and activities, the outputs they produce, and the intended outcomes and benefits. It also shows how these parts are linked. As the project is developed, the logic model can be updated with more details. For large or complex projects with many parts, it can be helpful to have a main logic model that summarizes everything, with supporting logic models for each part if necessary.

Refer to Technical note: Logic models (PDF 1.62MB) for more information.

Outputs are the products, services and infrastructure that result from the project's activities. 

Outcomes are the changes that happen because of these outputs. These changes can be in areas like:

• economy
• social
• environmental
• culture. 

They can happen in the short, medium or long term. Outcomes may include changes in people's: 

• lives
• status
• health
• surroundings
• knowledge
• attitudes
• values
• behaviours
• satisfaction levels.

For more information, refer to Technical note: Logic models (PDF 1.62MB).

The impacts of an initiative will include the outcomes and the benefits that come with them. Outcomes are the changes that happen because of the project, and benefits are the improvements in people's well-being from these changes. Benefits are a measure of the value of the outcomes to the NSW community which can be monetary or non-monetary.

Monitoring is the ongoing process of collecting and analysing information about how the project is going and its impacts. It's important for gathering data needed for evaluations. 

Evaluation is a detailed and clear process to check if the project is appropriate, effective, efficient, and beneficial to the community. Evaluation works best when supported by monitoring. It looks at more information than monitoring and answers specific questions about the project's performance. Monitoring and evaluation should work together to show how well the project is doing. 

A monitoring and evaluation framework plans and guides these activities over the project's life. It can be based on a logic model that shows the project's inputs, activities, outputs, and impacts.

For more information, refer to Technical note: Monitoring and evaluation framework (PDF 732.96KB).

Benefits management can inform evaluation, it is not to be used instead of evaluation. 

Benefits management helps make sure the project achieves its intended outcomes and benefits. Evaluation is an independent process to check if the project is efficient and effective.

You can find more information is the Benefits Management Guide (PDF 2.34MB).

Cost-benefit analysis (CBA) is used to investigate the actual costs, outcomes and benefits of an implemented initiative and assess its net impact on social welfare.

Ex-post CBA may be performed during the life of an initiative, this is also known as ‘in medias res’, or at the end of an initiative’s life.  

When performed during the life of an initiative, an ex-post CBA can help to:

• review the total expected costs and benefits of an initiative in progress
• update an ex-ante CBA with new data collected
• assess whether outcomes and benefits will continue
• inform adaptive management of a current initiative to maximise the benefits.

An ex-post CBA completed at the end of an initiative can be used to:

• assess whether the initiative was a good investment, and  
• assess whether the ex-ante CBA, where performed, was broadly accurate.  

Ex-post CBA should be performed at the stages that are useful to inform decision-making.  

For more information on the ex-post CBA, go to the technical note, Ex-post CBA (PDF 603.57KB).

Outcome evaluation:

• Checks if and how a project is making the changes it was meant to.
• Measures how much change has happened and how much the project contributed to the changes.
• Can also find unexpected outcomes.
• Can ask about fairness and distributions of outcomes.

Outcome evaluation can be performed when the project is expected to achieve results with enough data. It will provide an important base for economic evaluation, like cost-benefit analysis (CBA).

Economic evaluation:

• Looks at the project's impacts compared to its costs.
• Decides if the project is worth the money or brings social benefits.
• Uses data from outcome evaluations and project monitoring.

Cost-Benefit Analysis (CBA):

• Examines all the project's impacts, including economic, social, environmental and cultural.
• Checks if the project brought a net social benefit to people of NSW.
• Preferred method by NSW Treasury for economic evaluation.
• Recommended for large, complex, or risky projects.

For more information about outcome and economic evaluations, review the Evaluation TPG (PDF 1.66MB). 

The counterfactual is a consideration of what would likely have happened in the absence of an initiative, usually under a “do minimum” or “business as usual” (BAU) scenario as set out in the initiative’s business case. In ex-post CBA, the ‘counterfactual’ includes projection of costs and benefits ‘without’ the initiative. The use of a counterfactual is to ensure that the assessment of the initiative is based on ‘additionality’ - that is, the ‘extra’ amount of costs and benefits over what would have been the case in the absence of the initiative.

External expertise to the business and delivery team can be used to perform the evaluation or aspects of the evaluation. This may be an evaluation unit or evaluators within the agency, independent consultancy teams or academic teams. The need for external expertise may depend on existing evaluation capability within the team, the technical expertise required for the evaluation, and any requirements for an independent perspective.

Review Table 2 in the Evaluation Guidelines (PDF 1.66MB) and Technical note: Evaluation management and resourcing (PDF 535.43KB).

Best-practice is to include resources for monitoring and evaluation in the initiative budget. All proposals looking for government resources need to incorporate resourcing requirements for monitoring and evaluation into the business case funding submission (or set funding aside) or include a plan for when resourcing will be determined and obtained. The monitoring and evaluation budget should be appropriate to the evaluation requirements and scope, as well as the size of the initiative.