Data sharing considerations

Data sharing can be between public offices, with a private sector organisation or community or interest group, or published as open data. In all cases, it must be governed to ensure that the data is usable, trustworthy and understandable, and the sharing is done in a manner aligned with legislative requirements and community expectations.

Open Data

The NSW Government Open Data Publishing Guidelines outline the processes and safeguards to support open data publishing. In addition to those safeguards, agencies should capture records about the data they share.

Benefits of recordkeeping

Keeping records of open data sharing facilitates faster decision-making in relation to requests for access to information under the GIPA Act and removes the risk of duplicated effort. If requested information is already published as open data, applicants may be directed to the relevant portal.

Recordkeeping requirements

Public offices should record details of the data they share as open data, including:

• what they have shared – including data, metadata and any data dictionaries
• where they have published that data (e.g. the Data.NSW open data portal)
• when the data was shared
• any data quality statement that was provided with the data
• the licence under which it was shared
• decisions and approvals to publish the data.

Public offices should also update their Information Asset Register with details of data sharing.

Different recordkeeping requirements will apply to source data (i.e. the records held in the public office’s business systems) and data published as open data. For example:

• a database of Council development applications would be retained by the public office as evidence of development decisions.
• the data may be published as open data in the form of de-identified application data, used to understand trends about applications, decisions and application types in different regions.

These different business contexts will result in different recordkeeping requirements and retention and disposal outcomes. Public offices must retain and dispose of records in line with their business and recordkeeping requirements and the authorised records retention and disposal authority.

Disposal

Public offices should have a disposal plan in place for data published as open data. The Open Data Publishing Guidelines outline roles and responsibilities for managing the publication of open data. Public offices should have a process by which to remove datasets from open data portals and, where appropriate, destroy them. Such processes should take account of any applicable Data.NSW requirements relating to removal.

Data sharing

The Data Sharing Act requires public offices to have data sharing agreements in place before sharing data. Data sharing agreements should also be established for sharing data with private sector organisations.

Data sharing agreements are contracts which ensure data is handled in compliance with legislative and other requirements. Data sharing agreements support good recordkeeping by clarifying the recordkeeping responsibilities of each party involved in sharing, receiving or otherwise using the data.

Data sharing agreements document the purpose, requirements and limits of data sharing. In general, a data sharing agreement should include:

• the purpose, scope and lawful basis for sharing (as required by the Data Sharing Act)
• relevant legislation and frameworks or standards for the access, handling or use of the data – for example, the Privacy and Personal Information Protection Act 1998, or the Framework for the Governance of Indigenous Data (which supports implementation of Priority Reform 4 of the National Agreement on Closing the Gap)
• who owns the data, or who is responsible for determining the permissible use, sharing and disclosure of the information, and how the integrity of information is to be maintained and protected
• who is responsible for accuracy, updates and access control
• roles of the data provider, data recipient and any accredited data service providers
• conditions for use, disclosure, retention and destruction
• security and privacy controls
• roles and responsibilities for monitoring, identifying and reporting on data breaches (including notifying State Records NSW if records are compromised)
• agreed systems for data sharing and storage and long-term maintenance planning
• approvals and delegations
• any restrictions on on‑sharing or secondary use
• whether personal information is included
• consent conditions (if applicable)
• privacy safeguards and restrictions.

Benefits of recordkeeping

Keeping records of data sharing, such as data sharing agreements, supports good data management. Retaining data sharing agreements as records:

• facilitates oversight over shared data (both data that has been shared, and data that has been received by the public office)
• clarifies the roles and responsibilities of the public office with regards to data in its custody
• clarifies what the public office can do with data in its custody.

Data sharing agreements also provide evidence that the public office has done its due diligence with regards to the appropriate protection, handling and use of data.

Responsibility for shared data and records

The NSW Data and Information Custodianship Policy requires that public offices formally assign roles and responsibilities with regards to data. A data sharing agreement should define which public office has responsibility for the data, and how new access, use and sharing is authorised.  For example, who has responsibility for the data, and any datasets or products created by combining data? Is responsibility for any of the data being transferred or shared?

A data sharing agreement provides evidence and clarity about responsibilities with regards to shared records. The agreement should clarify whether responsibility for a set of records is being transferred when records are shared and, for example, who is responsible for protecting them, approving access to them, and for their disposal.

Retention and disposal considerations

Each party should be aware of applicable records retention and disposal requirements, and any impacts data sharing may have on their ability to comply. Whether a public office is sharing or receiving records, each public office should ensure that the data sharing agreement includes requirements which support their recordkeeping obligations.

For example, with regards to retention and disposal, a data sharing agreement should define data ownership, including roles and responsibilities with regards to retention and disposal and consider:

• Is approval required from a participating party to dispose of information?
• Can data be disposed of at each public office’s discretion once no longer required?
• Is return of data or records required at the end of an agreement? Is destruction required? If so, what evidence is required?

Metadata requirements

A data sharing agreement should define metadata requirements. Defining metadata requirements is critical to ensure that shared data does not lose its reliability, trustworthiness or usefulness in the process, especially where records are concerned.

Each party should consider what data they need about the dataset to make it useful and permit each party to fulfil their recordkeeping requirements. For example,

• Records are evidence of transactions, activities or decisions. Contextual metadata reflecting, for example, when a transaction happened, who was involved, or who made the decision may be critical to preserving the function of the record when sharing or receiving records.
• Metadata which supports disposal decisions in line with applicable records retention and disposal authorities should also be included. For example, date of creation or last update, or other business context metadata.
• Other metadata requirements may include, for example, data quality metadata, technical metadata, data dictionaries, or metadata mappings from source system fields to destination system fields to ensure data interoperability and preserve the data’s integrity and trustworthiness.

Monitoring and assurance

Even when data sharing is automated, regular reporting, monitoring and other assurance processes may be required to safeguard the integrity and reliability of the data. The data sharing agreement should outline data quality requirements and assign relevant roles and responsibilities to ensure those requirements are satisfied.