We are renovating our house, so, to get ready I was packing up closets and career memorabilia that hadn’t been touched in 20+ years. As I discovered stuff I had long forgotten about, I was struck by the perils of relying on technology when it comes to preserving our history.
There, on a high shelf in a back closet, were original boxes of System 7 and System 9 for the Mac – those date from the mid-to-late 1990’s. Next to it were retail boxes of Adobe FrameMaker (1994) and Ventura Publisher (1990). All of that software, by the way, shipped on 400 KB hard shell 3.5” floppy disks.
NOTE: Floppy disks, for those under the age of 22, were small, flat plastic or cardboard cases with a thin piece of magnetic plastic inside. They stored anywhere from 48 KB to 800 KB of data and were the principle method of distributing, selling and installing software, as well as the most commonly used system for storing personal files. Please note that they were measured in KiloBytes, which is 1,000 times smaller than MegaBytes. And 20 years ago, they were everywhere.
After I put that software into boxes, and moved the reel-to-reel audio deck down into a box next to the slide projector, I moved over to my collection of VHS tapes and DVDs. Yup, right next to the collection of Beta-SP and DigiBeta tapes.
As I looked at all this stuff, I was struck that my broadcast career started almost 50 years ago (49 to be exact) and during that time, there hasn’t been a five year period where the media we store our programs on remained the same.
During my career, and in my various closets, I have programs, highlights or personal images stored on:
Sheesh! That list was even bigger than I expected.
Then, when we include hard drives and RAIDs, these have more ports and connectors that I can count; all with incompatible connectors:
And I haven’t even begun to explore the codecs and file formats we’ve used for digital files all these files were created with. For example, all my kids earliest drawings were done using MacPaint, a format which can’t be opened today. I stored them in Extensis Portfolio, the iPhoto – both which are now dead.
As we’ve learned over the years, the nature of technology is that it has to change and evolve in order to grow. But we are in terrible danger of losing our history and collective memories.
There will always be a way for large studios to preserve highly-popular media assets, such as the “Wizard of Oz” or “Casablanca.” Large studios have the resources to spend millions of dollars preserving assets they will make even more millions on.
But that leaves the rest of us wondering what to do.
I have travelogues on 3/4″ video tape of Maryland in the early 1970’s, mini-docs of historical sites that no longer exist on 8 mm film, original music from Don McLean and John Denver on 2” tape, Vincent Price talking about art history on 1” tape; the list goes on and on. I even have Oprah’s resume reel on 3/4″ that got her “that job” in Chicago from which her career exploded. (Yeah, it was a show that I directed.)
Each of these programs capture a way of life that no longer exists. I won’t make any more money on them, but shouldn’t they be preserved as a part of the larger historical record of our society?
Years ago, seeing the writing on the wall, I dubbed all my 2” and 1” tapes to DigiBeta. Then, later, I dubbed DigiBeta and Beta-SP tapes to DV tape. I was worried that if I didn’t move quickly, my programs would be lost. However, I was not expecting DV to die as quickly as it did.
How do I preserve the past? Our preservation options are dwindling. Hard disks are not reliable over time. A dual-layer DVD only holds 8.3 GB of data, assuming you even own a DVD burner and player, while a single DV tape is 13 GB.
Blu-ray Discs hold more, but Blu-ray is not natively supported on the Mac. This is worrisome because even native technology dies quickly; consider DVDs or FireWire.
The Cloud, the technology titans would say. Move everything to The Cloud. Except, even if we ignore the security issues of The Cloud – and everyday the news reminds us that we can’t ignore security – there are still major problems with The Cloud:
Oh, the technology titans would say, that would never happen. Somehow, I’m much less sure.
It seems to me that there is a huge market here for historical preservation that an enterprising company could make a fortune in – providing safe, secure and high-quality preservation of media assets.
But it will take some work because what we have today is insufficient:
Even worse, there is no consistent archive media codec or storage format that works equally on Windows and Mac. Nor is there any reassurance that any company that provides a codec today will continue to support it tomorrow. (QuickTime on Windows, for example. And Apple can not be considered a small start-up with limited resources.)
I, like many readers who send me emails each week, are really struggling to figure out how to preserve history in the face of the relentless onslaught of technological change with using software and hardware that is both affordable and lasts for the long-term. Surely, even the titans of technology have families and projects that they would like to preserve for the future.
NOTE: I define long-term as the ability to access and play media that is more than 10 years old. (Archivists would suggest 50 years is a more reasonable horizon, but I think that is totally unattainable today.)
This is not an easy problem to solve – it requires a different kind of thinking and a focus on a different slice of the market. More importantly, it requires a way for a company to develop, create and support hardware long after the original sale, otherwise, this becomes yet another format that dies just when we start to rely on it.
As I was putting yet another VHS tape into the box, I realized that it was the future that scares me. What happens in ten years when my grand-daughter looks at me and asks: “What did you do when you were growing up?” only to realize that I have nothing to show her?
A generation’s worth of history – lost in the mist – simply because we felt change – and newer/better/faster – was more important than preserving the memories of who we are and what we did.
That’s more than sad, its terrifying.
As always, let me know what you think.
37 Responses to The Difficulty of Preserving Our Past
Newer Comments →-
Elvis Cruz says:
January 16, 2017 at 5:17 am
-
Larry says:
January 16, 2017 at 10:16 am
-
Michael says:
January 16, 2017 at 6:44 am
-
Larry says:
January 16, 2017 at 10:17 am
-
Patrick Ira DonEgan says:
January 17, 2017 at 2:00 pm
-
Larry says:
January 17, 2017 at 9:00 pm
-
Donald Davis says:
January 16, 2017 at 6:52 am
-
stu aull says:
January 16, 2017 at 7:01 am
-
Larry says:
January 16, 2017 at 10:17 am
-
Frank Maxwell says:
January 16, 2017 at 7:11 am
-
Larry says:
January 16, 2017 at 10:18 am
-
Jsnice Edens says:
January 16, 2017 at 7:25 am
-
Christina Fox says:
January 16, 2017 at 7:26 am
-
Larry says:
January 16, 2017 at 10:19 am
-
Greg Knollmeyer says:
January 16, 2017 at 7:47 am
-
Larry says:
January 16, 2017 at 10:20 am
-
Patrick Ira DonEgan says:
January 17, 2017 at 2:03 pm
-
C. Park Seward says:
January 16, 2017 at 9:57 am
-
Paul Jordan says:
January 16, 2017 at 10:18 am
-
Larry says:
January 16, 2017 at 10:27 am
-
Larry says:
January 16, 2017 at 10:20 am
-
Elvis Cruz says:
January 16, 2017 at 10:12 am
-
Patrick Ira DonEgan says:
January 17, 2017 at 2:04 pm
-
Larry says:
January 17, 2017 at 9:01 pm
Newer Comments →Larry, great article. I’ve been dealing with the same thing, trying to revive / recover a feature film I backed up to hard drives in 2011. Those hard drives are now almost unreadable.
As I read your article I was hoping it would end with an answer. When even the great Larry Jordan is stumped, we’re all up a creek!
Elvis:
It is a really complex problem of hardware, software, codecs and designing to include both the past and the future.
I wish there was an easy solution but I don’t think it exists.
Larry
Larry,
I know that you have talked about this subject before in the professional area and come to the conclusion of LTO as the all-around best “bet” for archiving. So, it’s nice to see you address the issues on the consumer and personal side. I, too, think it is an issue that many people don’t realize or don’t care much about. Today, thanks to the smart phone, many personal historical moments have become disposable after captured and shared. There was an article that one of your inter-planetary-internet training videos subjects had written a little while ago about a lost generation as it relates to this subject.
In the time periods that you wrote about, we have had many forms of physical media. Moreover, different types of media had different storage formats (audio, video, photos, etc.) Photos probably had the least variance (film) until the dawn of digital. Now the primary storage format for all of those for many consumers is their mobile phone. Sure, they can be backed up (albeit in many cases to the cloud) and they capture in fairly common formats (jpg, mov, etc), but you’ll never be able to take grandpa’s iPhone 7 out of a shoe box you find in the attic in 2047 and be able to experience the media it once captured and stored. There are already business services for transferring VHS and 8/16mm to DVD (a half measure, imho), so soon we may see something like that for phones. Of course, if you lose your phone or it’s damaged, then what? Especially if it’s not backed up. I’ve heard many a person in an Apple store lament the loss of all the phone’s media either from physical damage or a software issue requiring a re-format.
There have always been local services for media transfers, but I have heard advertisements recently for a service called Legacy Box. They say that even Disney trusts them to archive their stuff. You put all of your old media in a box and ship it off to them where they convert it and send you back a DVD or USB thumb drive. While this is great for space saving and bringing media up to date for viewing now, how much harder is a DVD or thumb drive to lose than a box of VHS tapes or film reels? Moreover, the comments you made about being able to play the DVD or read the files on the USB drive still apply. In the case of the USB, the physical plug may not work if you have a most recent generation (and presumably beyond) Mac where it’s all USB-C. I’ve still kept every dongle from FW400 til the most recent batch needed with the new Mac Book Pro “just in case.”
This subject fascinates me as I’m sure it does many of your readers. While not at a paranoid level, I’m certainly proactive about keeping up my own personal digital media both in current readable formats (the files and the storage types) and safeguarded with pretty sufficient back up schemes (one of which includes the cloud). The life of my two kids is pretty well captured and archived (my family would chuckle at the ‘pretty well’ part) and is completely digital as they are ages ten and under. I’ll be diligent about keeping it up for them so that I’ll be able to show it to their kids (assuming I’m around). Hopefully it will be easily recoverable by them or someone else out of the digital shoebox if I’m not around. For the photos, printing and shoe boxing may still be the better bet. I still enjoy the physical and tangible act of flipping thru photos individually or in an album/book that one can’t get from hitting the arrow keys or swiping looking at a screen.
My own personal history is a bit more shaky. My parents did some of the typical media transfers. We had an 8mm type camcorder, a bulky Betacam camera system and later a mini DV camcorder. The cameras were, of course, great for to play with as a kid which has continued into adult hood. Media that was even older was transferred first to VHS and then maybe DVD. My parents moved four times in twelve years and had storage units, etc., along the way. So, the whereabouts of any of that media is still a mystery to me. My mother thinks she might know, but I’ve never been able to find it. So, while I would love to personally digitize and archive it in some way to be able to share it with my kids, first I have to find it. People who at least have the media are one step ahead. Even though today’s media is largely contained all in one format (the phone) you first need the physical item before you can even start to think about how to read it and preserve it.
Thanks for writing about this. I very much look forward to yours and other’s comments. As for me, I’m waiting to see what Apple’s next move is around the iMac and Mac Pro so that I can then move everything to USB-C/Tbolt3 storage units leaving the Tbolt2 as a backup and shelving the FW800.
Cheers,
Michael
Michael:
Great comments. Thanks!
Larry
Is this the place to complain about the quality of “DVD-Video”
compared to VHS and Beta (consumer) and Hi8?
Patrick:
Sure – though it probably won’t do any good. Feel free to complain if it helps you feel better.
(Though, personally, assuming the bit rate is reasonably high, I’ve found DVDs consistently higher quality than VHS tape.)
Larry
Great intro to the preservation challenge all of us with family and professional media face. I inherited my dad’s 8mm film collection that he began in 1942 (Kodachrome). Having lost film of our family trip to Disney World that I sent to Kodak’s Atlanta Lab for developing: a tornado hit the Atlanta Airport’s mail facility where our film was a casualty, I didn’t want to mail Dad’s films to a digitizing facility. Fortunately, I discovered a company here in the Tampa area that washes, cleans, then digitally captures the film frame by frame to a hard drive. I then imported those clips into FCP X and edited them into formats (DVD’s, and Private YouTube files) for my family. Of course, I have the files backed up on hard drives and in the cloud. 50 years from now, will they survive? I can only hope, but for now, it’s the best I can do.
Now, on to scanning decades of family photos.
This blog the very week I am pulling 1999 interviews off client Beta SP reels to update their Historical video! Happy to report that the shots (I ended up dubbing the entire 12-reel collection for them to have at least a ProRes copy of it all. Yeah, to a hard drive…) all pulled off flawlessly! No tape failure, dropouts or anything! I was pleasantly surprised. Or damn lucky? Still have an ol’ faithful Beta SP player for just these circumstances. Next time? who knows?
And then there is the whole Back-ups conundrum. Gahh!
Stu:
I had both VHS and Beta-SP players for years – until they died.
Larry
Xmas I got my Fuji Single 8 films out the cupboard and had a films show. To my horror in the 1970 trip to Disneyland and Hawaii the colour had slightly faded and the bought 8mm films from Disneyland had gone a bit brown. Even cine film is loosing its crips colour.
What can we do regards Larry’s article?????
Kodak are bringing back film. Why i ask myself. We need to preserve our history for future generation.
Frank:
Color film dyes fade – especially blue. Time to get those films transferred to digital before you lose everything.
Larry
About a month ago I was able to pull videos on 2 different tape-based camcorders off to my computer to store there. Obviously this is only a temporary solution, but I was glad to find video footage I didn’t even recall taking. Some of it is priceless to me.
Maybe we need to move all the video onto film!
Think about it – 35mm film has been around way longer than all these digital formats and codecs and is still accessible.
I was in the London metropolital Archives a few weeks ago and it seems the best storage there was paper and microfiche (i.e. film).
Maybe going backwards is the way forward.
Christina:
Film is a great preservation media – especially black and white. But, can you imagine the expense – at multiple dollars per film foot – of archiving personal or small-budget projects?
Totally unaffordable.
Larry
I have both a question and answer. I think your article is an excellent picture of the dilemma. I have gotten all of my old media into ProRes masters that I’m storing on a hard drive (& it’s cloned copy). My thought is that if I keep my media in its nomadic journey across hard drives about every five years, I won’t lose anything.
Essentially if I copy the media forward before the drives fail, would there be any significant degradation over the years? I realize that I might have to transcode if ProRes becomes outdated or inaccessible. But the pain of transcoding is far less than digitizing film or tape.
So my answer for me has been to digitize and keep the footage on a nomadic journey across hard drives. But my question is will there be any degradation over time with this approach?
Greg:
Copying a digital file does not degrade image or audio quality, even if you do it one hundred times.
Larry
That is what “the big guys” with “9-track tape” do:
copy all the tapes every 5 years or what ever schedule.
Larry,
Found this on the M-Disk web site.
100 GB Disc
We are excited to announce that our 100 GB capacity M-Disc will be available within the next few weeks. It has all the incredible features of our original M-Disc, with over 20x times the capacity of the standard DVD. Archiving large data sets is now much more convenient.
Sure, that’s nice. But who’s going to have the drives to read them in ten years when optical drives are getting phased out everywhere? The storage media is only one part of the problem. You also need to be able to read the media, and then you have to play/access them. I still have those Mac Paint files Larry was talking about, on a folder on my computer I’ve moved around for thirty years. It doesn’t mean I can open them.
Paul:
(smile…) I have several copies here, as well. Every so often, I try to open them – just to see if, by magic, they now work.
Larry
Park:
Good to know, thanks.
Larry
Larry, or anyone,
Any experience with M-Disc? It supposedly has a 1,000 year life.
http://www.mdisc.com/mdisc-technology/
How did they test the “1000 year” claim?
Patrick:
There are recognized, standardized tests that will accurately estimate lifespans of equipment. While I don’t know the specific tests, I have worked with engineers who have used them enough that I am comfortable with the underlying theory.
Larry