The ABC have a piece from National Library of Australia web archiving manager Paul Koerbin, about the importance of digital records preservation.
Of equal importance, how can we be sure that we can actually read those archives in the future? Literacy of Egyptian Hieroglyphs was long-gone by the 18th century, and it took the discovery of the Rosetta Stone for them to start making sense again.
It’s difficult enough deciphering human language. Understanding machine language is another thing entirely.
I’ve written about this in the past, contrasting the thousand-year-old Domesday Book (which is still legible) with the BBC Domesday Project (which was rendered virtually unreadable a mere sixteen years after production).
The means of preserving our culture for digital preservation is to use open standards. If the means for ‘reading’ the information is widely documented and understood, without any encumbrances, we stand a much greater chance of being able to interpret it in a couple of hundred years.
I’ve got essays from school written only ten years ago, and I can’t read them any more as they’re stored in a proprietary file format that is no longer supported.
Imagine you ran a company that had important and valuable written records stretching back for decades. Storing vast libraries of paper is expensive and inefficient, so you decide to digitise them all. That’s great — you now have a system that is easy to manage and search. Ten years later, you want to migrate your now-ageing data management system to something more modern. Only, you can’t — it’s all stored in a proprietary format that cannot be accessed by anything else.
If you had kept those paper records, you would have still had access to that information. Your choices now are to continue with your old, obsolete system for all eternity, or hire some clever hacker to decipher the file format. With no equivalent of a Rosetta Stone, that’s no mean task. After spending buckets of money on this avoidable problem, and losing even more due to inefficiencies and competitive disadvantage from the old system, you’d be wise to make sure it cannot happen again.
This is a very common kind of scenario. If our information can’t even last ten years, how can it last a thousand?
From a business perspective, open standards protect the independence of a company. It means no vendor lock-in, so you are not stuck paying monopoly prices. Through the creation of a free market surrounding a method/technology, open standards give you the freedom to select the vendors, products, methods and technologies that suit your requirements best, or you can even create your own. They are the ultimate in risk mitigation, and through their flexibility can also open avenues for competitive advantage. They just make good business sense.
LotD: Vioxx maker Merck and Co drew up doctor hit list and Merck Makes Phony Peer-Review Journal
May 10th, 2009 at 2:07 pm
Sridhar, you’ll be pleased to know that the Digital Preservation crew at the National Archives of Australia are also firm believers in Open Standards for long term access to data. So much so that we are engaged in FOSS development to make it possible for all to have access to an easy means of getting stuff into open formats.
Our Xena software is a good start http://xena.sourceforge.net and for those doing this stuff on an industrial scale and looking for a digital preservation workflow management and audit tool, there’s our Digital Preservation Recorder http://dpr.sourceforge.net
All released under the GPL and all cross platform (Java).
We’re committed to Open Standards based formats and as an organisation we get involved in standards processes through Standards Australia. That we can implement all this through Free and Open Source Software is a bonus and in my opinion essential if this kind of work is to be trustworthy.
Cheers,
Michael Carden (with my NAA hat on)