Review of benchmarks and quality assurance measures

Periodic reviews of benchmarks and quality assurance measures are essential to ensure they remain relevant to the purpose of the records and align with emerging technologies, legislative changes, and industry trends.

Regular updates help maintain the integrity and effectiveness of digitisation projects over time.

The benchmark and quality assurance checklist

The most important points from this checklist are:

• organisations should define the necessary benchmarks
• clear procedures and training must be in place for staff who are responsible for meeting these benchmarks
• quality checks should be conducted to assess how well the benchmarks are being met, and remedial action should be taken when necessary.

Establishing benchmarks and quality assurance standards

• Have benchmarks been developed in relation to:
  • digital image quality, including technical specifications, metadata requirements and requirements for equipment (for example, calibration and output viewing device)
  • storage and controls for digital images
  • management of original paper records including when they can be destroyed (if relevant)?
• Have benchmarks been
  • developed in liaison with stakeholders
  • documented
  • approved by senior management
  • communicated to relevant stakeholders?
• Do the processes determined and documented in digitisation procedures enable benchmarks to be met?
• Have benchmarks and quality assurance measures been communicated to relevant staff through procedures?
• Do procedures address:
  • roles and responsibilities, including sign-off at an appropriate level
  • the method and frequency of calibration testing of equipment
  • the quantity of images and metadata to be checked and how frequently checks should occur
  • how quality checking is to be carried out
  • what to do if checks reveal poor image or metadata capture, including when re-imaging is required and how it is to be conducted
  • when image enhancement techniques can be used
  • whether images should be enlarged for quality checking?
• Have benchmarks and quality assurance procedures been tested before digitisation  commences, to ensure they can be implemented and produce acceptable results?  
• Has advice been sought from Museums of History NSW regarding the digitisation of original paper records required as State archives?  

Staffing

• Are staff and managers with responsibility for benchmarking and quality assurance sufficiently trained and supported to meet their responsibilities in a consistent way?  
• Are changes to procedures documented and communicated to staff?

Equipment

• Is equipment regularly cleaned, serviced and calibrated?  

Checking of digital images

• Are digital images quality checked as regularly as required by organisational procedures?      
• Are a sample of digital images checked for such aspects  as:
  • legibility (for example, smallest type size for text, clarity of punctuation marks, and including decimal points)
  • completeness of detail (for example, acceptability of broken characters, missing segments of lines or pixels, missing information at the edges of the image area and images cropped or incomplete)
  • whether dimensions accurately compare with the original
  • whether scanner-generated speckle has been removed ( when speckle is not present on the original)
  • whether colours or tones accurately compare with the original  
  • is the density of solid black areas too light or too dark?
  • has colour been realistically captured ?  
  • is brightness and contrast correctly captured?
  • the sharpness of the image compared to the original (neither too sharp or too blurry and with no halos around dark edges)
  • the accuracy of the text captured by optical character recognition (OCR) software (where relevant)
  • whether images are in the correct file formats
  • whether compression ratios are correct (where appropriate) whether the correct bit-depths and resolutions are used?
• Have all the essential characteristics defined in the planning stage of a project been reproduced successfully?
• Have all of the records identified for digitisation and all of the pages in multi-page items been digitised? 
• Have benchmarks for the output viewing device been met?

Metadata

• Has all required metadata been collected?        
• Is the captured metadata relevant, accurate and linked to the correct records? For example:
  • have the appropriate levels of security been applied
  • has the accurate creation date been captured
  • have the correct document author and scanner operator identified?      
• Can metadata be interpreted consistently?
• Is metadata that is stored in multiple locations synchronised?
• Are all mandatory metadata fields complete?      
• Has the usefulness of the metadata been assessed over time?  
• Are digital records being re-imaged and metadata re-captured in line with procedures when they do not meet quality standards?

Random sampling

• Is any sampling conducted in line with documented procedures, for example, in line with the frequency specified?      
• Do the agreed samples for quality checking represent the range and quality of records digitised?  

Image enhancement

• Are the scope and extent of use of any image enhancement techniques documented?
• Have these been checked so they do not result in the loss of information?

Quality failure and re-imaging

• Are failures in the reliability of storage logged, analysed and addressed?      
• Are quality failures logged, analysed and addressed?
• Have records been re-imaged and/or metadata reassigned according to procedures when initial digitisation does not meet quality standards?

Management of original paper records

• Are all original paper records being carefully handled during retrieval, preparation for digitisation and the digitisation process?    
• Where original paper records are not destroyed, are they returned to their original order and storage after digitisation?      
• If original paper records are destroyed, are they kept for the predetermined retention period after digitisation (for quality assurance purposes) before being destroyed?        
• If original paper records are destroyed, is this in accordance with the General retention and disposal authority: original or source records that have been copied?
• Management of digital images where original paper records are destroyed
• Are digital images stored in accordance with the organisation’s documented requirements, for example, in a mandated recordkeeping system?
• Is the storage system tested to ensure digital images remain protected?  

Documentation

Is quality control data (such as logs, reports, decisions) documented and captured as records and managed as part of the digital images’ metadata?

Review

Are benchmarks and quality assurance procedures implemented and regularly reviewed?