How Wollongong's Digital Archive Problem Built Up Over Decades, and What Happens Now

The Wollongong City Council's library and heritage services division is sitting on tens of thousands of digital image files, an unknown but substantial portion of which are duplicates, some triplicated, some appearing in five or more versions across different storage systems. It is a problem that did not arrive overnight.

The reckoning matters now because the council's current digital asset management contract is due for renewal in late 2026, and regional cultural bodies funded through the Illawarra Shoalhaven Joint Organisation are under pressure to demonstrate proper stewardship of public records ahead of a State Archives NSW compliance audit scheduled for the second half of the year.

How the clutter accumulated

The root cause is not negligence so much as it is the predictable consequence of three decades of uncoordinated digitisation. Wollongong City Library on Crown Street began scanning local history photographs in the mid-1990s. The Illawarra Museum in Wollongong's CBD ran its own parallel digitisation program. The University of Wollongong's Special Collections, based at the main Northfields Avenue campus, added a third stream. Each institution used different file-naming conventions, different resolution standards and, critically, different software platforms with no shared deduplication logic.

When cloud migration became standard practice after 2015, organisations bulk-uploaded everything they had. Files that existed in three separate local folders were uploaded three times. A single 1953 photograph of the Port Kembla steelworks, for example, might now exist as a TIFF master, two JPEG derivatives at different resolutions, a web-optimised PNG, and a copy embedded inside a PDF catalogue, all treated by the system as distinct assets. Multiply that pattern across thousands of images and the storage overhead, and the indexing confusion, becomes significant.

The issue is not unique to the Illawarra. The NSW Government's digital records guidance, updated in March 2024 under the State Records Act 1998, specifically flagged duplicate digital assets as a compliance risk that regional councils had chronically underestimated. Wollongong was among more than a dozen councils named in background correspondence as requiring closer attention, according to documents circulated to local government bodies and reported in sector publications at the time.

The local cost of inaction

Storage is not free. Commercial cloud storage for cultural and government institutions in NSW is typically priced on a per-gigabyte basis, with enterprise contracts for local government running in the range of tens of thousands of dollars annually depending on volume. A duplicate rate of even 30 percent, a conservative estimate cited in general digital asset management literature, means councils may be paying for roughly a third of their storage unnecessarily.

For Wollongong, the practical consequences go beyond cost. The Illawarra Performing Arts Centre on Burelli Street digitised its production archive in 2019. The WIN Entertainment Centre contributed event photography going back to 2002. Neither collection was cross-referenced against existing council holdings before upload. Researchers and journalists requesting images through the library's local studies service regularly receive multiple versions of the same asset with inconsistent metadata, making reliable attribution difficult.

The BlueScope Steel industrial archive, which documents Port Kembla's transformation from a mid-century heavy industry site into the proposed renewable energy precinct it is becoming, is among the collections most affected. Images of the same infrastructure at different stages of development have been catalogued separately rather than linked, fragmenting a visual record that has genuine historical and planning value as the Port Kembla Renewable Energy Zone project progresses.

What comes next depends partly on what the council decides to do before its contract renewal deadline. Digital asset management specialists recommend a deduplication audit as the first step, a process that uses hash-matching software to identify byte-identical files and near-duplicate detection algorithms for visually similar images. For an institution the size of Wollongong City Library's digital holdings, such an audit typically takes between three and six months. Once duplicates are identified, a retention decision needs to be made: which version is the master, which metadata set is authoritative, and which copies can be safely deleted. None of that is technically complex. All of it requires someone to be clearly responsible, which, until now, has been the missing piece.