We are renovating our house, so, to get ready I was packing up closets and career memorabilia that hadn’t been touched in 20+ years. As I discovered stuff I had long forgotten about, I was struck by the perils of relying on technology when it comes to preserving our history.
There, on a high shelf in a back closet, were original boxes of System 7 and System 9 for the Mac – those date from the mid-to-late 1990’s. Next to it were retail boxes of Adobe FrameMaker (1994) and Ventura Publisher (1990). All of that software, by the way, shipped on 400 KB hard shell 3.5” floppy disks.
NOTE: Floppy disks, for those under the age of 22, were small, flat plastic or cardboard cases with a thin piece of magnetic plastic inside. They stored anywhere from 48 KB to 800 KB of data and were the principle method of distributing, selling and installing software, as well as the most commonly used system for storing personal files. Please note that they were measured in KiloBytes, which is 1,000 times smaller than MegaBytes. And 20 years ago, they were everywhere.
After I put that software into boxes, and moved the reel-to-reel audio deck down into a box next to the slide projector, I moved over to my collection of VHS tapes and DVDs. Yup, right next to the collection of Beta-SP and DigiBeta tapes.
As I looked at all this stuff, I was struck that my broadcast career started almost 50 years ago (49 to be exact) and during that time, there hasn’t been a five year period where the media we store our programs on remained the same.
During my career, and in my various closets, I have programs, highlights or personal images stored on:
Sheesh! That list was even bigger than I expected.
Then, when we include hard drives and RAIDs, these have more ports and connectors that I can count; all with incompatible connectors:
And I haven’t even begun to explore the codecs and file formats we’ve used for digital files all these files were created with. For example, all my kids earliest drawings were done using MacPaint, a format which can’t be opened today. I stored them in Extensis Portfolio, the iPhoto – both which are now dead.
As we’ve learned over the years, the nature of technology is that it has to change and evolve in order to grow. But we are in terrible danger of losing our history and collective memories.
There will always be a way for large studios to preserve highly-popular media assets, such as the “Wizard of Oz” or “Casablanca.” Large studios have the resources to spend millions of dollars preserving assets they will make even more millions on.
But that leaves the rest of us wondering what to do.
I have travelogues on 3/4″ video tape of Maryland in the early 1970’s, mini-docs of historical sites that no longer exist on 8 mm film, original music from Don McLean and John Denver on 2” tape, Vincent Price talking about art history on 1” tape; the list goes on and on. I even have Oprah’s resume reel on 3/4″ that got her “that job” in Chicago from which her career exploded. (Yeah, it was a show that I directed.)
Each of these programs capture a way of life that no longer exists. I won’t make any more money on them, but shouldn’t they be preserved as a part of the larger historical record of our society?
Years ago, seeing the writing on the wall, I dubbed all my 2” and 1” tapes to DigiBeta. Then, later, I dubbed DigiBeta and Beta-SP tapes to DV tape. I was worried that if I didn’t move quickly, my programs would be lost. However, I was not expecting DV to die as quickly as it did.
How do I preserve the past? Our preservation options are dwindling. Hard disks are not reliable over time. A dual-layer DVD only holds 8.3 GB of data, assuming you even own a DVD burner and player, while a single DV tape is 13 GB.
Blu-ray Discs hold more, but Blu-ray is not natively supported on the Mac. This is worrisome because even native technology dies quickly; consider DVDs or FireWire.
The Cloud, the technology titans would say. Move everything to The Cloud. Except, even if we ignore the security issues of The Cloud – and everyday the news reminds us that we can’t ignore security – there are still major problems with The Cloud:
Oh, the technology titans would say, that would never happen. Somehow, I’m much less sure.
It seems to me that there is a huge market here for historical preservation that an enterprising company could make a fortune in – providing safe, secure and high-quality preservation of media assets.
But it will take some work because what we have today is insufficient:
Even worse, there is no consistent archive media codec or storage format that works equally on Windows and Mac. Nor is there any reassurance that any company that provides a codec today will continue to support it tomorrow. (QuickTime on Windows, for example. And Apple can not be considered a small start-up with limited resources.)
I, like many readers who send me emails each week, are really struggling to figure out how to preserve history in the face of the relentless onslaught of technological change with using software and hardware that is both affordable and lasts for the long-term. Surely, even the titans of technology have families and projects that they would like to preserve for the future.
NOTE: I define long-term as the ability to access and play media that is more than 10 years old. (Archivists would suggest 50 years is a more reasonable horizon, but I think that is totally unattainable today.)
This is not an easy problem to solve – it requires a different kind of thinking and a focus on a different slice of the market. More importantly, it requires a way for a company to develop, create and support hardware long after the original sale, otherwise, this becomes yet another format that dies just when we start to rely on it.
As I was putting yet another VHS tape into the box, I realized that it was the future that scares me. What happens in ten years when my grand-daughter looks at me and asks: “What did you do when you were growing up?” only to realize that I have nothing to show her?
A generation’s worth of history – lost in the mist – simply because we felt change – and newer/better/faster – was more important than preserving the memories of who we are and what we did.
That’s more than sad, its terrifying.
As always, let me know what you think.
37 Responses to The Difficulty of Preserving Our Past
← Older Comments-
Tonyfleming says:
January 16, 2017 at 10:12 am
-
Richard says:
January 17, 2017 at 8:33 am
-
tony Fleming says:
January 20, 2017 at 3:36 pm
-
Kevin Murphy says:
January 16, 2017 at 10:42 am
-
Alan Halfhill says:
January 16, 2017 at 10:59 am
-
Brad Horne says:
January 16, 2017 at 12:12 pm
-
Mike Janowski says:
January 16, 2017 at 2:03 pm
-
Corey Bailey says:
January 16, 2017 at 7:39 pm
-
Bob White says:
January 18, 2017 at 3:47 pm
-
Larry says:
January 18, 2017 at 6:42 pm
-
Gregory Shaw says:
January 19, 2017 at 9:44 am
-
Larry says:
January 19, 2017 at 10:45 am
-
wayne says:
January 21, 2017 at 10:21 am
← Older CommentsI completely agree both on the need to have a reliable means of backing up and also the ongoing need for discs or some other cheap and portable way of distributing media.
A few months ago, I posted a video on Vimeo using black & white still photos plus some 16 mm film shot in Hong Kong in the early 1960’s. Within 20 minutes of uploading it on a Sunday evening, I was receiving e-mails from people in HK saying how much they appreciated the images. More photos are taken by more people today than ever before but how many are going to survive into the future?
I also posted a video documenting a journey across the Sahara desert in 1955 using color slides shot at the time. Most of these were Kodachrome which have survived the last 60 years remarkably well. Modern technology has allowed thousands of people to share the pictures but that would not have been possible had the original images not survived.
A real gripe of mine has been the arrogance of major companies deciding that optical media (discs) are obsolete. This is absolutely ridiculous. I agree that streaming provides good quality images but only to those people who have absolutely reliable fast internet. I show my work at functions where it would be sheer lunacy and irresponsible to rely on streaming to show the main event to a large audience. Now that most of us are shooting in 4K it is very depressing to show it from a SD DVD so it is more or less imperative to be able to go to BluRay.
Steve Jobs may well have distained it and described it as a “world of hurt”. As far as I am concerned, that is a phrase I would use to describe the current situation where we have all been abandoned. I currently use Roxio Toast which has its limitations. It could be greatly improved with, it seems to me, the minimum of effort but they don’t bother to respond to feedback. I know – I tried.
Most of the world does not enjoy decent internet connection including in this country. I spend a good percentage of each year in places where there is absolutely no internet or TV coverage. That suits me just fine given the quality of the content – but cheap and portable media is important and necessary!
Those sound like wonderful videos. Could you please provide the links to the Sahara and the Hong Kong films.
The link to the HK video is:
https://vimeo.com/121733138 and to the Sahara crossing the last number is: 38473412. If you are interested in my other Vimeo stuff it’s best to go to
http://www.vimeo.com/tonyfleming.
Great timing! I have just resumed work on both VHS and DV tape-copying into my current system (MacBook Pro based, with FCPX as the emergency ward for restoring the stuff and polishing what can be polished), and that will be both possible, and valuable, for I am rediscovering videos I had forgotten existed (meaning they are not in my current YouTube collection).
Older text files are still recoverable in a couple of ways for me, but those ways are clunky, and fading fast, too: I have an old 1992 Packard Bell with early Windows on it that is still running old word processors that I can use to modify things like AmiPro and WordStar documents to rtf for copying into the Mac. But it is old, hiding in an attic work space that is not convenient, and inconvenient to connect to anything that will connect to the Mac — and its hardware isn’t getting younger. I’ve already replaced the B-drive, and have nothing left to cannibalize for any more replacement parts. And I have software that works in Snow Leopard that can translate many earlier personal computer word processing formats accurately. But, all of it is TIME-CONSUMING! And, as you point out, these formats will also fade away, in all probability.
Yet, the salvaging of the tape-based videos has been encouraging, at east for the near future. Sadly, I have nothing with which to salvage my 1″ cassette (or was it ¾”?) recording of a critical video from 40+ years ago, nor the ½” magnetic tape video reels from even earlier. Yet, I refuse to toss them . . .
Our historical society lives in a constant state of frustration over barely decipherable handwritten and print fragments that find their way into its hands from the “distant past” (19th and early 20th century). What we are discussing here is the digital equivalent of that frustration.
Like you, I would like to preserve family history for children, grandchildren, etc. and, even more importantly, to preserve community history for our community’s archives, but industry has not yet seemed to take such matters seriously, at least in an affordable manner.
Just dubbed 20 Hi8 tapes to Hard Drive and DVD. No problems playing them back.
Hello Larry and friends,
I also agree this is a problem which needs resolution. There are many working to solve this and many others who have made attempts. Most of which are business options and doesn’t solve archival problem on a personal level.
A smart migration is the theoretical approach which most will recommend; but who then has time for that in the course or normal life. So yes, using the ‘cloud’ will be a good option for some with a good line to their home. Otherwise, my limited recommendation is to use the format that has proven the longest life from my standpoint (being born in ’81), use an optical disc format to back up all personal data.
Disc formats such as CD-ROM and DVD-ROM have been around for over 20 years and are easy to access with most computers. A Blu-ray drive can hold between 25GB – 100GB. This is a respectable amount of personal data, many copies can be affordably made and stored in a small drawer in different locations around the house.
Coming forward through time, I doubt it is going to be difficult to find an external optical disc drive that will support your media. In short, from what I can see is until there is another option out there, stick with the optical discs.
(Of course this requires, all the tape formats to be digitized first [hard work and time consuming!] and find a codec you are happy with for putting the media onto the discs; most of the standard MPEG formats should suffice in my opinion)
This is a very timely article for me, as I’m in the process of re-making my entire family photo and video archive, as well as my personal professional archive.
I’ve followed you closely over the years on this topic, and have taken your advice (vis-a-vis hard drives) to heart. I have hard drive backups, certainly, but EVERYTHING is going to LTO-6 tape. I’m renting a unit from a company in Columbus, OH called MagStor, which markets a couple of neat LTO boxes with Thunderbolt 2 connectivity. So for the short term I feel somewhat confident that I’ll be preserving my precious assets in the best and most archival way possible.
But there are many other issues to deal with here besides the physical storage of media assets, the largest one being metadata; the cataloging of said assets so future generations can figure out WTF they’re looking at. For instance, my earliest venture into archiving was getting my family’s 8mm films transferred…which was great, but resulted in a bunch of files called “JanowskiFilm_xxx”, whether they were actual pictures of the Janowskis, the Hartmans, the Guinns, or any other branch of the family. I know who they are, but my kids certainly won’t, so I need to create a methodology of naming and cataloging the files so that they’re not . Similarly, when I began to digitize all the SVHS family movies (via FCP), I ended up with a bunch of files named “UntitledX”; not a very useful moniker, you’ll agree. Slowly, I’m recovering from these early gaffes, attempting to organize and name the files with consistent terminology understandable on the desktop level that isn’t tied to an editing system project file.
There seems to be a majority of Apple product users here. I would suggest that you pick a medium that is cross-platform compatible. It’s important that, in the future, your data can be read by the platform that will be in use.
Beyond that:
Be ready to migrate, migrate, migrate, regardless of the medium you choose today.
I believe that it is the arrogance and greed of the film studios, TV companies and apple who are trying to force us to use their preferred technologies (Like iTunes if you want to buy music) They may say it is for your convenience but really it will make them more money in the future. When you can not give your customers cd’s & DVD’s then you are forced to put your footage online for remote recording for remote display. There you pay the studios and companies money for that service. That inflates their income stream but does nothing for the small company who believe it or not do not want to steal their precious copyright., TV companies have to accept that we copy their programs for later viewing (The government allows the mass sale of consumer recording equipment) but at least we also copy the adds in the process.
I think a solution to availability of equipment could be that once a company stops producing old technology like Hi 8, or Audio Tapes then their patents and designs should become public property then any enterprising company could use the designs to fill a gap in the market.
Bob:
Open sourcing the hardware is an interesting idea. Thanks for sharing your thoughts.
Larry
Not that it changes the importance of finding durable preservation tools, but this is not a new problem. It has impacted humankind from the day communication and culture was invented. Whether scratches on a wall or clay tablets or the Sistine Chapel, only the tiniest percentage of creative work has survived the millennia.
The visual cultural materials that do survive only need an eyeball to access (That is one technology that likely won’t be changing soon). No proprietary encryption/encoding involved. Coded documents (text) have always created an access issue and become inaccessible without translation meta data (Rosetta stones).
Gregory:
All true, though the Rosetta Stone did not require constant power and a clean environment to survive…
Larry
Unless there is a forced incentive for companies to adopt a single archiving format (not likely), then we are stuck not archiving ourselves (if we can do so and choose to do so). But even so, as technology changes content has to be moved from one media to another. This is where service companies get involved, for example Legacy Box. (I am not affiliated with them). They will take all your different media and condense them and put on a DVD, or flash. But in 5 to 10 years my guess is you need to do it all over again. Will this work? (good timing as archiving my family stuff is moving up on my “to do” list)