Sure, you can write your data, but will you be able to read it?
There was a time in my professional life when I spent lots of time and energy helping people implement database, file and messaging servers.
Inevitably, folks would want to talk about some form of archiving.
They’d have been the recipient of some pitch or seen some spec sheet on some wild optical media device, and then been seduced by the dark side.
“25 years! 50 year media life!”
“Ha!”, I would say, usually projecting some subtle degree of smugness, “and how are you going to read it, 50 years from now?”
“What application, what file system, what hardware, what technology are you planning to have around then that will give you some confidence of your ability to read this stuff?”
That good advice was, at the time, largely theoretical, a matter of informed conjecture.
Just last week though, the news provided me with not one, but two examples of that specific problem playing out.
The first was a case of the Andy Warhol Museum coming on a group of old diskettes in their collection, as reported in the Washington Post. The Pittsburg museum was fortunate enough to be able to work with the Carnegie Mellon Computer Club — remember Mach Unix? — who had the curiosity and the skills to piece together what computer and application created them. Ever more fortunately, The Computer Club actually had a running example of a Commodore Amiga in their collection which allowed them to read the disks and open the files.
The files, in this case comprised at least 4 recognizable signature original Warhol works, along with several more bits of Andy just mucking about with a new computer.
Andy is an icon of 20th century style — These might not be irreplaceable masterworks, but Warhol’s whole approach to art was that all of his work was conceived to be eminently replaceable.
The second current example of getting back the data is a subject of somewhat more gravity.
It seems that back before the Apollo moon landings, NASA sent several missions to map the moon so we would be able to identify safe landing spots for the planned manned missions. Doug Bierend at Wired has written a fascinating story about a project to recover the information.
It seems that the photographic images — including the first ever of the dark side of the moon — were sent back as analog data. The pictures that were used and published at the time were made by transmitting single scanned screens of information from a photograph made by the satellite and printed out on thermal printers in 20 foot long strips which were then taped together into a whole image. That resulting montage was then photographed and passed to the wire services. If you’ve ever seen contemporary published photographs they’re notable for an weird, banded appearance that are artifacts of that oddball process.
No one, actually, has ever seen the original photographs, because the satellites that took them crashed into the moon.
I used to work for a company that was headquartered within sight of the impressive blimp hangar at Moffett Field in Silicon Valley. Up and away! At NASA’s Ames Research Center, located there, there is a closed McDonald’s restaurant filled with Analog Tape Drives and a crew of techs who describe themselves as “digital archeologists” and refer to their Jolly Roger-flying former fast food joint as “McMoon’s”.
These talented folks started from scratch and had to reverse engineer the data formats and how the information was encoded. Nothing was known, and the nature of the problems that needed to be solved ranged from the sublime to the ridiculous.
Do you know that the specifications for a 1966 Ampex tape drive call for whale oil to be used as the lubricant? Like they have that down at the local Radio Shack.
To cut to results, these guys succeeded in cracking the encoding, hacking a bridge into Adobe Photoshop, and recovering and rendering the information at better resolution than is possible with our current digital cameras.
The resulting photographs are staggering — breathtaking.
The information was there, but no one had ever devised a way to see it until now. We sent rockets to the dark side of the moon, but never really saw the full results of that work — just dim shadows.
If there’s a Nobel prize for Geekery, these guys have my vote.
So if you find yourself in the position of making decisions about a new information system, my old good advice about considering how you intend to access that information in the future is doubly good now. The life cycles of data storage devices have shortened dramatically. Operating systems, applications and file systems have gotten more and more fragmented. If you’re planning on storing large amounts of data with a cloud provider, you’d don’t want to be surprised if it turns out they haven’t really considered the portability and tools they’ll use to move your data if you need them to later.
In system design, you need to spend nearly as much time considering how to manage the access life cycle of the data as you do figuring out what data you will write.
Cause it does you no good whatsoever to write if you can’t read.