Digitisation specifications
It is important to determine technical specifications before commencing digitisation or purchasing equipment. Setting technical specifications ensures quality, consistency, and guarantees that the digitised records are fit for purpose.
The primary goal is to capture sufficient data so that resulting images or files are of appropriate quality for their purpose and can remain legible or useable for as long as required. Where records have long term or archival value, the resulting digitised files may need to withstand time and numerous migrations.
Adopting the technical specifications defined below will assist your organisation in meeting the copying provisions outlined in Original or source records that have been copied (GA51). GA51 allows for the authorised destruction of original or source records that have been copied, provided that certain conditions are met.
Determining the right specifications
The appropriate specifications depend on project needs:
- For short-term use, lower-quality formats may suffice.
- For archival records, create a high-quality 'master' format and lower-quality derivatives for access to maximise the return on digitisation efforts and support long-term accessibility.
Consult guidelines like the State Library NSW's Digital Practice Guidelines for advice on specifications for master copies and web delivery.
As a general rule, use the highest technical specifications that your organisation can realistically support, especially for records that are required as State archives and serve as evidence of business activity if originals are destroyed.
Analyse these factors to determine suitable specifications:
- criticality of records (for example, use in court)
- essential characteristics to reproduce
- whether original paper records or physical video or audio formats will be destroyed
- the record version to serve as the official business record
- retention and accessibility periods
- current and future uses of images, audio and video.
Documenting image, audio and video capture specifications is essential to maintain a clear record of the digitisation process. This ensures consistency and meets the requirements of ongoing or back-capture digitisation projects.
Enhancements
Only use image enhancements, such as sharpening or background removal, when necessary and document the changes. Avoid enhancements if high-quality images can be obtained without affecting the original record’s authenticity.
Watermarking and fingerprinting are techniques that modify digital files by adding information such as the organisation name or user details. While useful, these modifications make the file no longer an exact copy of the original paper record. To maintain the authenticity of the record, it is recommended to keep the digital file unmodified and store additional information as metadata rather than embedded within the file itself.
Technical components
The highest technical specifications possible and supportable should be selected.
If your organisation chooses to vary these technical specifications, they should conduct an assessment of all factors and document this along with the reasons for choosing alternative specifications. The primary considerations should always be to ensure:
- the legibility of the digital file
- the reproduction of the original records' essential characteristics
- that the file is fit for purpose.
It is important to test your technical specifications and settings for suitability. Check that:
- the smallest detail is legibly captured (e.g. smallest type size for text, clarity of punctuation, marks including decimal points)
- details are complete (look for broken characters, missing segments of lines, glitches in the video or audio etc.)
- file sharpness is sufficient compared to the source (if halos appear around the text/characters, it is likely due to over compression).
When testing specifications, view digitised files at scale (i.e. 100%).
Overall technical components should be:
| Document type | Resolution* | Bit Depth | File Format | Compression |
|---|---|---|---|---|
| Documents with text only, black and white | Minimum 300ppi | 1 bit (bi-tonal) | TIFF, PDF/A† containing TIFF or JPEG 2000‡ | Lossless compression |
| Documents with watermarks, grey shading, grey graphics | Minimum 600ppi | 8 bit greyscale | TIFF, JPEG2000, PDF/A containing TIFF or JPEG 2000 | Lossless compression |
| Documents with discrete colour used in text or diagrams | Minimum 600ppi | Minimum 8 bit colour | TIFF, JPEG2000, PDF/A containing TIFF or JPEG 2000 | Lossless compression |
| Black and white photographs | Sufficient to provide >3000 pixels across long dimensions | 8 bit greyscale | TIFF, JPEG2000, PDF/A containing TIFF or JPEG 2000 | Lossless compression |
| Colour photographs | Sufficient to provide >3000 pixels across long dimensions | 24 bit colour | TIFF, JPEG2000, PDF/A containing TIFF or JPEG 2000 | Lossless compression |
| Black and white negatives | Sufficient to provide >3000 pixels across long dimensions | 8 bit greyscale or 24 bit colour | TIFF, JPEG2000, PDF/A containing TIFF or JPEG 2000 | Lossless compression |
| Colour negatives and transparencies | Sufficient to provide >3000 pixels across long dimensions | 24 bit colour | TIFF, JPEG2000, PDF/A containing TIFF or JPEG 2000 | Lossless compression |
*The scale/ratio for resolution here is 1:1.
† PDF/A is a constrained version of PDF version 1.4 with various proprietary fonts and formats removed, issued as ISO 19005-1:2004.
‡ JPEG 2000 is defined in ISO 15444-1:2000.
The size and quality of a digital file is a product of its technical elements. Image resolution, colour properties, compression, output file format and embedded metadata all contribute to the quality and size of a digital file.
There is no way to reduce file size while also maintaining quality – it is a compromise. To reduce size, one or more of the technical elements will need to be decreased. Establishing technical specifications is about striking an acceptable balance.
If records have longer retention periods or are required as State archives, create a high-quality master file for preservation. Where files sizes need to be smaller for delivery purposes (e.g. via email), a lower quality version can be derived from the master file.
File size is a valid consideration but should not override the need for:
- legible images
- fitness for purpose
- retention of essential characteristics.
File formats must be suited for long-term sustainability and accessibility and be capable of meeting compression requirements. For a list of sustainable formats, consult the guidance on sustainable file formats.
To determine which file formats are most appropriate for purpose, consider:
- compression method (lossy, lossless, or uncompressed)
- ability to hold metadata
- ability for text/optical character recognition (OCR)
- page display (single vs multipage page)
- compatibility with software programs
- opensource formats with published technical specifications available in the public domain – avoid proprietary formats where there are risks of obsolescence if the vendor stops supporting, increases costs of access or cannot be supported long term
For digitisation programs, it is recommended to use:
- PDF/A (lossless) – for documents with long-term retention requirements or those required as State archives.
- Standard PDF (high-quality lossy) – for modern/word processed records with short-term retention requirements.
- TIFF or JPEG2000 (lossless) – for photographic and illustrative material.
- WAV or BWF - for audio as these are widely accepted for preservation and use lossless Linear Pulse Code Modulation (LPCM) codec.
- FLAC – for audio as stable, free and open-source with lossless compression. While suitable for preservation, FLAC may not be supported by the audio industry.
- MOV or AVI with uncompressed 8- or 10-bit encoding for small projects. If storage capacity allows, use uncompressed 8- or 10-bit encoding. Lower-quality, compressed copies (for example, MP3) can be used for web delivery or other non-archival purposes.
What is the difference between PDF and PDF/A?
PDF/A is an International Organization for Standardization (ISO) standardised version of the Portable Document Format (PDF) designed for long-term digital preservation. Unlike standard PDFs, PDF/A embeds all information required for display, such as fonts and images, and does not rely on external sources like font programs or hyperlinks.
Although PDF/A files are larger than regular PDFs, they are more suitable for long-term access. However, PDF/A files cannot include encryption, JavaScript, or multimedia content. If security or authenticity is a priority, it is recommended to store PDF/A records in a recordkeeping system as PDF/A files can be edited.
Most common file formats use compression techniques to reduce the size of the resulting digital file for ease of storage and transmission. Depending on the file format, compression could occur by default or be (to various degrees) controlled through settings/preferences. Compression reduces the size of a digital file to facilitate storage or transmission. It can be categorised into two types:
- Lossy: Removes data during compression, meaning some information is lost (irreversible).
- Lossless: No data is lost, and the original file can be exactly recreated when decompressed.
Lossy compression algorithms reduce file size by discarding less important information (such as slight tonal differences). Some lossy formats such as JPEG have threshold setting – the impact of the threshold can vary image to image as it is dependent on pixel content. Too much lossy compression will degrade image quality. The effects of lossy compression are irreversible and compact with every save.
Lossy compression can be applied to short-term records and records digitised for reference purposes.
Lossless compression algorithms reduce file size through simplifying the representation of the code. No data is irretrievably lost, and the decompressed object is of the same quality as the original file. Examples include LZW or ZIP lossless compression within TIFF files.
If the records have long-term retention requirements or are required as State archives, it is recommended to use lossless compression for master files and to create an access copy with lossy compression for everyday use.
Note: Smart compression techniques may employ additional reduction techniques such as resizing, lowering resolution, bit depth/colour reduction, etc. Any use should be after thorough testing.
Resolution is a measure of the ability to capture detail (or samples) from the hardcopy record. It is most frequently quantified in pixels per inch (ppi), which describes pixel density over a given scale (ideally 1:1 – original size). The higher the ppi at the same scale/size, the better the resolution and the clearer the image.
Choosing the correct resolution from the start is essential as it is impossible to increase an image’s true resolution after it has been digitised, as software can only invent or redistribute pixels rather than capture new detail. If higher resolution is needed, the original item would need to be digitised again.
Consider:
- The nature of the records to be digitised. Photographs, drawings and cursive handwriting require greater resolution than typed documents.
- How the digital files will be used. Hardcopy records that are enlarged or require fine detail for viewing and printing should be digitised at a higher resolution. Hardcopy records that are reduced for viewing may be digitised at lower resolutions. If text/optical character recognition (OCR) will be applied, consider resolution requirements for the OCR software.
Note regarding resolution for photographs, negatives and transparencies: For photographs, negatives, and transparencies, the required resolution will vary according to the size of the photograph or negative. In these cases, measure the longest side of the photograph in inches then calculate the required resolution by dividing 3000 by the length of that long side.
For example:
If you have a photograph that is 5 inches by 8 inches, then 8 inches is the longest side. 3000 divided by 8 = minimum 375ppi.
As rough rules:
- For photographs with a longest side measuring 15 inches or greater for the longest side, use at least 200ppi.
- Between 10 and 15 inches, use at least 300ppi.
- Between 5 and 10 inches, use at least 600ppi.
Bit depth refers to the number of bits (zeros and ones) used to represent the smallest unit of information (such as a sound byte or image pixel). For digitised images, bit depth governs the range of colours or shades of grey (tonal range) represented in the resulting image.
Bit depth also determines the image’s colour mode – i.e. if the resulting image will be bi-tonal, greyscale or full colour.
Like resolution, if a higher bit depth is required, the record will need to be re-digitised.
| Common colour modes and when to use them: | ||
|---|---|---|
| Bit depth | Colour mode | When to use it |
| 1 bit | Bi-tonal (Pure black and white – no grey) | Black and white, clear high contrast word processed documents and art line only. |
| 8 bit | Greyscale | Greyscale or black and white documents. Includes those that contain watermarks, grey shading and grey graphics. |
| 24 bit | Colour | Documents with discrete colour used in text or diagrams and coloured documents. |
| 36-48 bit | RGB Colour | An extended colour space, resulting in a larger file size and requiring specific formats like TIFF or PNG. |
Note: 8 bit colour (RGB) is not suitable for digitisation.
Specialist equipment may also offer greater bit depths such as 16 bit Greyscale and 48 bit Colour. Larger depths provide greater tolerance for colour/tone correction during post-production editing. This is particularly necessary for preservation purposes and when reproducing images at scale.
A colour space defines the range of colours within an image. Assigning a colour space will help software know how to render colour. Commonly used colour spaces include sRGB and AdobeRGB.
A colour space is just one component of colour management. Scanners, monitors and printers, etc. typically use different colour spectra. Colour management uses device profiling (ICC profiles) in aims of ensuring that a file looks the same across a range of different devices. Colour management is best left to experts due to its technical complexity.
Digitising in colour is not always necessary. Consider whether the essential characteristics of the records need to be preserved in the digital file. If colour is important for understanding or preserving the record's evidential value, such as in court documents with coloured annotations, then colour should be included. However, if colour is not critical, it may be omitted. For example, colour in a letterhead is not essential for understanding the content of the record.
Equipment
Equipment for digitisation should not be selected (or procured) until you understand:
- the records you intend to digitise
- your technical specifications for file quality.
You can then assess the hardware and software to see if it can deliver the quality you require as vendor specifications may differ from real-world performance.
Digitising paper records as part of your business-as-usual process (for example, day boxes) can have lower quality specification than records that will be required as State archives.
High level considerations for equipment selection may include:
| Category | Considerations |
|---|---|
| Format |
|
| Technical |
|
| Records handling |
|
| Output/Configuration |
|
| Software |
Image enhancement (for example, sharpening, clipping of highlights or shadows, blurring to eliminate scratches, spotting or de-speckling) should be documented and follow consistent procedures to ensure authenticity. Annotations, such as redactions or sticky notes, should be managed as overlays, not altering the original image. |
| Operational |
|
Technical infrastructure considerations
Ensure your organisation has the technical capacity to handle increased digital storage needs. Key considerations include adequate storage for projected file sizes, sufficient bandwidth for real-time access and backup or offsite storage solutions.
For example, a small project might require 1.56TB over 3 years, while larger projects could require 10–15TB annually.
Health and safety considerations
Consider the health and safety of staff when setting up digitisation equipment. Test all equipment before full rollout to ensure it meets operational and safety requirements.
| Characteristic | Explanation |
|---|---|
| Style |
|
| Resolution |
|
| Automatic document feed | This is the ability to process original documents in bulk, sequentially digitising rather than hand feeding individual pages. Varieties of automatic document feed (ADF) are:
|
| Original size | This is the maximum page size a particular scanner can accommodate. Typically scanners are A3 or A4 enabled, but consideration for non-standard paper sizes may be needed. |
| Speed of scanning | Digitising speeds depend on the resolution required and whether the digitisation is in colour or black and white (greyscale). |
| Capability | Scanners support multiple requirements. One scale to assess the capability of scanners is:
|
| Ports and image transfer supported | Interfaces between scanners, printers and computers for storage are required. Typical interface protocols are:
|
Multifunction devices (MFDs), such as photocopiers, can be used as scanners for business process digitisation, and in some cases their built‑in functionality is sufficient. If using an existing MFD, organisations must confirm that it meets required technical specifications.
As with any equipment purchase, it’s important to review the vendor’s reputation, service agreements, and support levels. Procurement must also comply with NSW Government Contract 9827 - Print and Imaging Devices and Services.
Useful MFD features include the ability to send scanned images directly to a designated network location for automatic import into an EDRMS and allow users to add metadata at the point of digitisation.
Staff must also be trained in correct device use, quality assurance responsibilities, digital filing procedures, and how to handle original paper records after they have been digitised.
Metadata requirements for digitisation
Why collect metadata?
Metadata is data that provides information about other data. Whether descriptive or technical, metadata can help people understand digitised records.
In digitisation projects, metadata may be used to:
- find and use digital files
- link files to the business processes they document
- demonstrate that files are accurate and reliable renditions of the hardcopy records
- document the digitisation process
- document formats and dependencies to help manage files over time
- transfer records to the State Archives Collection with Museums of History NSW.
When planning your digitisation program, consider requirements for metadata as early as possible as without proper metadata, digital files risk becoming inaccessible, reducing the project's long-term benefits. For more information, see Metadata for records and information.
Organisations may already have pre-existing metadata sources such as:
- existing systems like metadata from an electronic document and records management system (EDRMS) or similar system
- optical character recognition (OCR) as automated metadata capture from standardised document formats
- inherited metadata that is extracted metadata from linked business systems via XML schemas.
Manual metadata collection is a costly last resort, prone to errors. Automating metadata capture is strongly recommended.
File titling conventions
File titling rules should be determined to name digitised files consistently and to promote discovery.
If appropriate, file titling can follow the source records’ existing file titling or numbering system. Alternatively, you can develop a standardised naming convention to identify the records, and/or employ unique identifiers consisting of alpha-numerical sequences (i.e. with a prefix and running number).
In general, file and image names should:
- be unique
- be consistently structured
- include the use of leading zeros to facilitate sorting in numerical order
- avoid special characters (e.g. tabs or symbols)
- avoid spaces (as they can cause problems across operating platforms).
File titling conventions should include provisions for each object level used (files and folders) and distinguish between master and deviate copies.
Be sure to document file titling conventions to inform future interpretation and use.
Descriptive metadata
Descriptive metadata is commonly managed alongside the digitised records, such as in business systems or EDRMS where metadata is connected to the digital files. Depending on the file format used, metadata can also be embedded into digitised files themselves for assurance.
You will need to determine which descriptive fields are relevant to your project.
Generally, this would include:
- title - facilitates retrieval with meaningful names like “Minutes 2024-02-24.”
- creator - indicates the creator of the original paper record.
- original file number - distinguishes each digital image or file
- original creation date - refers to the original paper record's creation date.
- document form - see advice on Document form metadata scheme
- custom fields - including business function
If the records are required as State archives, it is recommended to include the required fields for transfer. This will save effort later. See Museum of History NSW’s Transfer Fact Sheet 6 Metadata.
Often descriptive metadata comes from existing systems or lists, but in other cases it may be necessary to data enter or extract descriptive metadata from the records themselves. If hardcopy indexes and/or registers are required to make these records accessible, you will need to consider how these can be incorporated into the program.
Searchable text / Optical Character Recognition (OCR)
If your program involves word-processed documents, determine if optical character recognition (OCR) is a requirement of your program. This may be subject to the type and clarity of the source documents.
OCR may enable the automation of some metadata capture, particularly if your documents follow a standardised structure.
Technical metadata
Technical metadata is a by-product of the digitisation process.
Data such as:
- date and time of file creation
- imaging capture device
- software used
- operator/creator
- technical properties (file format, resolution, bit-depth, colour space, and compression)
- number of revisions or edits, etc.
Can be automatically recorded during digitisation, however, equipment and software may need adjustment to ensure technical data is collected and embedded into digitised files.
Technical metadata is useful in documenting the digitisation process and can also help inform quality assurance activities.
Request an accessible format of this publication.