Frequently asked questions on digitisation
This page describes the conditions under which original paper records can be destroyed after digitisation, provided they meet the requirements of the General retention and disposal authority: Original or source records that have been copied (GA45). It outlines the need for quality control and proper documentation.
Disposal of original paper records
This page describes the conditions under which original paper records can be destroyed after digitisation, provided they meet the requirements of the General retention and disposal authority: Original or source records that have been copied (GA45). It outlines the need for quality control and proper documentation. If you lack a recordkeeping system, destruction is allowed if digital copies meet authority standards.
Can I destroy original paper records that were digitized before GA45 was issued?
If the originals are eligible for destruction under the general retention and disposal authority: Original or source records that have been copied (GA45), and they meet all conditions, they may be destroyed.
You should check the documentation related to the digitisation process and quality control measures to ensure the digital records are authentic, complete, and accessible.
If adequate documentation is not available, it may be prudent to retain the originals.
Certain records are excluded from GA45 and cannot be destroyed after digitisation. These include:
- State archives created prior to 1980.
- Audiovisual records required as State archives.
- Records of high personal value to individuals, such as care leavers’ records.
- Records subject to legal or policy requirements, like letters patent signed by the Governor of New South Wales.
- Records with intrinsic value, such as those with cultural, iconic, heritage, or aesthetic importance.
How long should I keep original paper records after digitisation before destroying them?
Original paper records should be retained for a period to allow for quality control checks on the digitised images. Your organisation should assess factors such as:
- Assurance that the digitised record is accurate and complete
- Assurance that the digital image is managed properly in a recordkeeping system
- Robustness of the digitisation process
- Authenticity of the digital record
- Need for access to the original records for legal purposes
Note that it is rare that original records are required for legal purposes. As long as you can demonstrate that your processes result in true and exact copies, this is sufficient for most legal purposes.
Can day boxes be destroyed?
Day boxing, a practice in business process digitisation, involves scanning records as they are received and placing the originals in a 'day box' or batch. You may destroy these records if they are covered by the retention and disposal authorities.
However, day boxing should not be used if the originals are to be retained. After digitisation, original records should be kept for a period of time for quality control before they can be destroyed. This period should be chosen and applied consistently across the organisation.
Can I destroy originals if I do not have recordkeeping system?
If your digitisation meets the conditions outlined in GA45, you can destroy original records even without a sophisticated recordkeeping system.
The key is to ensure the digital records are authentic, complete, and accessible, and that all other requirements (such as retaining originals for quality control) are met.
If you cannot meet these requirements, you are not authorised to destroy any original records.
Admissibility of digital images in court
Can digital images be admissible in court?
In most cases there is no barrier to organisations tendering digital images of records as evidence. They can be considered suitable to submit in legal proceedings in response to Government Information (Public Access) Act 2009 (GIPA Act) applications and for other evidentiary purposes.
However, the value or credibility of the digital image as evidence can still be questioned.
The authenticity of a presented record may be challenged or a judge may be given some other reason to doubt the reliability of a digital image.
In these cases, your documentation regarding how the digitisation was conducted and the digital images created and kept may help to demonstrate that the digital image is an authentic and credible representation of the original.
See legal admissibility of digital records in the relevant guideline for more information.
Technical standards
What technical standards should I use?
Technical specifications ensure the digital image is clear and of sufficient quality for its intended use, remaining legible and usable over time. For long-term digital records, they should withstand time and multiple migrations.
The technical specifications guidelines explain file formats, resolution, compression, and other relevant factors, along with a table listing recommended technical specifications for records and considerations when deviating from these standards.
What is the difference between PPI and DPI in relation to resolution?
Resolution measures the ability to capture detail in the original work. It is commonly measured in pixels per inch (PPI) for displays and dots per inch (DPI) for printers.
Higher PPI or DPI results in better resolution and clearer images. While PPI and DPI are often used interchangeably, PPI applies to digital displays, while DPI refers specifically to printers. In most cases, PPI will equate roughly to DPI.
Do I need to digitise in colour?
Digitising in colour is not always necessary. Consider whether the essential characteristics of the records need to be preserved in the digital image.
If colour is important for understanding or preserving the record's evidential value, such as in court documents with coloured annotations, then colour should be included.
However, if colour is not critical, it may be omitted. For example, colour in a letterhead is not essential for understanding the content of the record.
Can I use enhancement techniques for the digital images?
Image enhancement techniques can be used to improve the image quality to closely resemble the original. However, changes should be documented and routinely applied to avoid altering the record's evidential value.
If enhancements are not properly managed, the authenticity of the digital images may be questioned.
Should I use watermarking or fingerprinting for records?
Watermarking and fingerprinting are techniques that modify digital images by adding information such as the organisation name or user details. While useful, these modifications make the image no longer an exact copy of the original paper record.
To maintain the authenticity of the record, it is recommended to keep the digital image unmodified and store additional information as metadata rather than embedded within the image itself.
What is the difference between PDF and PDF/A?
PDF/A is an International Organization for Standardization (ISO) standardised version of the Portable Document Format (PDF) designed for long-term digital preservation. Unlike standard PDFs, PDF/A embeds all information required for display, such as fonts and images, and does not rely on external sources like font programs or hyperlinks.
Although PDF/A files are larger than regular PDFs, they are more suitable for long-term access. However, PDF/A files cannot include encryption, JavaScript, or multimedia content. If security or authenticity is a priority, it is recommended to store PDF/A records in a recordkeeping system as PDF/A files can be edited.
Equipment
Should I use optical character recognition (OCR) software?
Optical character recognition software extracts text from scanned documents, enabling it to be saved as a text file or word processor document. OCR recognises text within images and often includes search and export features. Additionally, OCR can automate metadata capture by identifying document definitions.
Whether to use OCR depends on the objectives of your digitisation project or program and the nature of the records. Consider whether OCR aligns with your organisation's needs, as well as the cost of the software compared to your budget. Note that some documents may not be suitable for OCR processing.
Benchmarks and quality assurance
Quality assurance in digitisation is important to ensure accurate and authentic digital records. Key points include performing checks on enhancements and OCR accuracy, preventing common faults, and considering pagination (pagination allows users to use links and page numbers to navigate between pages) for maintaining order. Robust processes help ensure long-term usability and evidential integrity.
What quality checks should I perform?
The level of quality control required depends on the risks associated with your digitisation program or project. Refer to benchmarks and quality assurance for more details.
Should enhancements like sharpening, blurring, or de-speckling be checked?
Yes, all enhancements should be reviewed. These adjustments may improve the image but can unintentionally remove critical details if applied too strongly. For example, if the tolerances are set too high the dots above the letter ‘i' may be removed.
Quality assurance should ensure that:
- no information is lost during adjustments
- changes align with documented standards to protect authenticity
Documenting these processes helps maintain the integrity of the digital records.
If I use optical character recognition, do I need to check for the accuracy of text?
Yes, optical character recognition often needs manual checks to ensure accuracy. Factors such as handwriting, damaged pages, or unusual fonts can reduce reliability.
What are common quality faults in digitistaion and how can they be prevented?
Quality faults fall into three categories:
- implementation faults
- dirty originals
- incorrect file sizes or formats
- inappropriate compression
- process faults
- incomplete specifications
- incorrect colour management systems
- faulty hardware or software
- low-quality originals or metadata
- operator faults
- incorrect file names or metadata including incorrect use of controlled vocabulary
- uncalibrated equipment
- File optimisation faults such as image orientation, exposure, focus, colour, contract of the image during processing
- cropping errors
- placing digital images into incorrect folders, files or classification structures
Prevent faults with regular equipment checks, proper training, and clear procedures.
Should papers be paginated before digitsation?
Your organisation should consider whether it is necessary to add page numbers to a file or volume prior to digitisation if not numbered already. It does require additional effort and potentially cost if digitisation is outsourced, but this should be balanced with your organisation’s need to have evidence of the exact order of the original papers on the file. A risk assessment should be conducted when planning the project. If records are high risk or long term/archival and/or tend to be requested in court, your organisation may well decide that the additional effort is justified.
Pagination can assist you with quality assurance checking. Where original paper records are to be retained after digitisation, it will also assist you to reconstruct the original paper records.
If page numbers are to be added your organisation’s procedures for digitisation should indicate acceptable ways this must be done. Pagination of archives is generally done in soft pencil so that it can be removed if necessary. Stamping is not generally recommended for archives as the stamps alter the original, and may obscure text or negatively impact on pictures or images. In some cases you may be able to add page numbers to the metadata of digital images.
If your organisation decides not to paginate, you should (at least) determine the amount of pages digitised and compare these to the number of papers in the original records (this can be sampled where relevant). If pages are removed to be digitised in separate batches, for example, non-standard materials that require a different scanner, flags should be added to ensure pages are returned to the correct order.
Outsourcing
Outsourcing digitisation requires clear responsibilities and thorough planning to ensure compliance with laws, standards, and organisational needs.
What are my organisations responsibilities if digitation is outsourced?
Public offices remain responsible for meeting the requirements of the State Records Act 1998, even if digitisation is outsourced. This includes managing both the source records and the digital images. Contracts with providers must specify all relevant requirements.
What should I include in contracts with outsourced providers?
Contracts should ensure that digitisation projects are managed effectively. They must include:
- clear guidance on the records to be digitised
- timeframes, costs, and expectations, including that records must not be altered
- roles and responsibilities of the organisation and the provider
- requirements for sensitive or personal records and urgent requests
- benchmarks such as technical and metadata standards
- quality assurance measures, including sample checks and remediation for unmet benchmarks
- a monitoring framework for oversight
- a statement ensuring all records remain in NSW unless approved by State Records.
Providers must also comply with the following standards:
- Standard on the physical storage of State records, including careful handling and conversion to recognised standards
- Standard on records management, which specifies metadata requirements for reliability and trustworthiness
- General retention and disposal authority: Original or source records that have been copied, detailing conditions for destroying original records after digitisation.
If records required as State archives are being digitised, consult with Museums of History NSW to discuss archival requirements and agree on parameters for the process.
Planning
A large-scale back-capture digitisation project requires careful planning and efficient resource management.
What should I consider for large-scale back capture projects?
Breaking large-scale projects into smaller, manageable components can improve efficiency and allow for adjustments based on lessons learned. Each component should have clear and measurable parameters.
Consider the following:
- Different formats or ages of records: Records requiring different equipment or methods can form separate parts of the project.
- Frequently used records: Assess how long records can be unavailable and whether some can be digitised on request, even if out of sequence.
- Simultaneous project components: Ensure you have efficient monitoring mechanisms if multiple parts are conducted at the same time.