Quality assurance and benchmarking

Quality assurance and benchmarking in digitisation is important to ensure accurate and authentic digital records. Quality assurance and quality control measures can help ensure that:

• source records are handled correctly and are undamaged during digitisation processes
• source records are lawfully destroyed (when appropriate)
• digital files are validated against technical specifications 
• digital files are accurate, complete and legible and of sufficient quality to be ‘fit for purpose’ with their essential characteristics preserved
• metadata to control and manage digital files is adequate
• your organisation can prove that robust and trusted systems and processes were used to produce digital files if their authenticity as evidence is ever questioned.

Benchmarks

Benchmarks are the quality standards that a digitisation project must meet. They describe quality that the digital copies need to be and what other steps are needed to keep the records safe and usable.

Benchmarks should be planned at the start of a project and agreed upon by everyone involved. They can change depending on the project’s purpose and the records being digitised. The digital copies must be good enough to replace the originals, especially if the records are high-risk (for example, required as evidence) or high-value (for example, must be kept long-term).

Ensure that:

• benchmarks related to the quality of digitisation have been developed in liaison with stakeholders, documented, approved by senior management and communicated to relevant stakeholders.
• the organisation established, documented and implemented quality assurance measures to ensure that benchmarks can be met
• benchmarks make it clear what and when image enhancement techniques can be used
• processes been devised to expedite checking of digital files
• quality checking is completed before the digital files are used as part of business processes
• quality checking is completed before any original paper records are disposed of
• the results of quality checks are documented
• procedures are in place for when re-digitisation is required under quality checking
• reviews of benchmarks and quality assurance measures are conducted regularly to ensure processes to meet business requirements
• all staff members understand their roles and responsibilities in relation to benchmarking and quality assurance including:
  • setting benchmarks, testing them and defining and testing processes that will enable benchmarks to be met, undertaken by the project manager in liaison with relevant stakeholders, service providers etc.
  • undertaking defined processes so that benchmarks are met by digitisation staff members 
  • undertaking quality assurance checking of digital images by project staff members
  • undertaking additional quality assurance checking of digital images to promote consistency by supervisory staff
  • checking that other program benchmarks are met – undertaken by the project manager. 

See digitisation specifications for technical specification and metadata requirement benchmarks.

Quality Assurance

Quality assurance measures often include:

• setting, implementing and monitoring digitisation processes, policy and procedures 
• acquiring equipment that is fit-for-purpose
• implementing a testing process and the management of quality fails, etc. 
• identification and assessment of quality criteria for benchmarking.

The degree and nature of quality assurance should be defined early in a project. Quality assurance must be carried out periodically during a project in accordance with defined procedures.

Original paper records should be retained for an appropriate period determined by the organisation to allow for quality control checks on the digitised files. Your organisation should assess factors such as:

• assurance that the digitised record is accurate and complete with no information lost
• assurance that the digital file is managed properly in a recordkeeping system
• robustness of the digitisation process
• authenticity of the digital record
• need for access to the original records for legal purposes

It is rare that original records are required for legal purposes. As long as you can demonstrate that your processes result in true and exact copies, this is sufficient for most legal purposes.

Some quality assurance may be undertaken by service providers in accordance with agreed parameters. It is essential that some quality checking is also undertaken by your organisation at regular intervals to ensure quality standards are maintained.

Quality control and checks

Quality checks should be planned early and tested in pilot projects to make sure everything works well. Regular checks should also be done throughout the project. All quality checks must be finished before using the digital files or destroying the original paper records.

Quality control involves checking the scanned file during or after digitisation to identify and correct any defects or errors. The goal of the quality control/check is to ensure that all the essential characteristics of the original paper records, defined during the benchmarking phase, are faithfully represented in the digital format. Representational accuracy is especially important if the hard-copy records will be destroyed once digitisation and verification is complete. 

Examples of what quality control should check for, include:

Quality checks for metadata

Quality checks for metadata must be clearly planned, written into procedures or contracts, and implemented. 

Checks for manually entered metadata should be stricter, but automatic metadata also needs some level of review.

Quality checks should ensure:

• Completeness: All required metadata has been collected.
• Standards compliance: Metadata follows the standards set for the project.
• Correct naming conventions: File names match the required naming conventions.
• Relevant metadata: Metadata accurately describes the content.
• Accuracy: Grammar, spelling, and punctuation are correct, especially for manual entries.
• Consistency: Metadata is created and interpreted in the same way across the project.
• Synchronisation: Metadata stored in different locations (e.g., in a TIFF header, management system, or database) is consistent.

Organisational procedures should explain how to handle errors found during quality checks. Over time, it is also helpful to review how useful the collected metadata is and adjust processes or systems to improve future collection.

Important considerations

• File identifiers: File names must uniquely and consistently identify the digital image and its metadata. Information like page number, date, or institution ID in file names should match what is stored in database records or headers. File identifiers link metadata stored in different systems and must be accurate.

• Sequence and completeness of multi-page items: Pages should be in the right order, with no pages missing. Metadata should reflect the content, including attachments, chapters, or multi-page records. Descriptions of these parts must match the actual images. Choosing to digitise only what is ‘important’ from a file destroys the authenticity, accuracy, trustworthiness and integrity of that file and must be avoided.

   

Sampling

Sampling for quality checking of digital formats

When faced with time and financial constraints, it may be necessary to use sampling as a method for quality checking digital records. Sampling options and their advantages and disadvantages:

Checking all digital images and metadata:

• Pros: All images will meet the minimum required quality baselines.
• Cons: Time- and resource-intensive.

Checking only random samples of digital images and metadata:

• Pros: Requires less time and fewer resources.
• Cons: There is a lower degree of certainty that all images meet the quality baselines.

If sampling is used, the following should be considered:

• Benchmark frequency: The frequency of sampling should be based on system usage and anticipated periods of deterioration. It’s crucial to include both the images and their metadata in the sampling process. System vendors may provide guidance on the appropriate frequency. ³
• Frequency of sampling: Initially, more frequent sampling may be required, but as benchmarks, equipment, and processes stabilise, random sampling of 5-10% may be sufficient. ⁴
• Representative samples: Ensure that the samples represent the full range of digitised records, including those with poor quality in the original paper records. If new staff, equipment repairs, or service providers are involved, every image may need to be checked until standards are fully met.

Environment for quality checking

• A controlled environment is necessary for consistent quality assurance. Factors like excessive glare, reflections, or improper setup of computer systems could lead to misjudging the quality of a high-quality image.
• Output device: The quality assurance checks should be conducted using the same output device intended for the image (for example, if the image is meant for printing, it should be printed and checked against the quality baselines for printed images).
• Checking may need to be conducted on a variety of printers and monitors to detect any variations. 

User fault reporting

• Users should report any faults to the digitisation team so these can be corrected. Faults should be included in quality assurance reports, as they may help identify common issues that need to be addressed.

Re-digitising

• If images do not meet the required benchmarks, the organisation or service provider must re-digitise them. Approaches include:
  • If more than 1% of images and metadata in a randomly selected sample are defective, the entire batch must be re-inspected and corrected.
  • If less than 1% are defective, only the faulty images and metadata should be redone.

While some organisations may set an ‘acceptable margin of error,’ the goal should be to achieve 100% accuracy. If the essential characteristics of the original records are compromised, the images may no longer be fit for their intended purpose and should not be retained.

Pagination 

Pagination before digitisation is optional but should be weighed against how important it is for your organisation to prove the original order of documents. Adding page numbers takes extra time and may increase outsourcing costs, so a risk assessment is recommended. For high‑risk, long‑term, archival, or legally sensitive records, the extra effort is often justified.

Pagination supports quality assurance and makes it easier to reconstruct the original paper files if they are kept after digitisation. If you choose to paginate, your procedures should specify acceptable methods. Archives are usually numbered in soft pencil so markings can be removed; stamps are discouraged because they alter or obscure the original. In some cases, page numbers can instead be added to digital metadata.

If you decide not to paginate, you should still confirm that the number of digitised pages matches the number of original pages, at least through sampling. When pages are removed for separate scanning—such as oversized or non‑standard items—flags or markers should be used to ensure they are returned to the correct order.

Monitoring 

As your digitisation program progresses, monitor and review activities for effectiveness. Adjust plans, business rules and procedures if and as needed. Periodic reviews of benchmarks and quality assurance measures are essential to ensure they remain relevant to the purpose of the records and align with emerging technologies, legislative changes, and industry trends.

When a program is in its infancy, it is good practise to put safeguards in place. Implementing a day boxing procedure for BAU scanning or a more robust quality assurance process for outsourced work are common examples of applied safeguards.