Thinking About Storage: Why Don't Hard Disks Hold As Much As They Say?

Posted on by Larry

[ This article was first published in the March, 2010, issue of
Larry’s Final Cut Pro Newsletter. Click here to subscribe. ]


Have you ever wondered why a 1 TB drive never has 1 TB of free space? Or why a 500 GB drive stores much less than 500 GB?

Well, it isn’t your imagination, it’s that engineers and marketers use two different numbers – but the same words – to measure the storage capacity of a hard drive.

A marketer wants to keep things clear and simple, and make their hard drives seem as big as possible. So they define:

This is all well and good and simple… but wrong.

Because computers don’t think using multiple of ten, like we do. Instead, they think using multiples of two. (Remember, computers store everything as 1’s or 0’s, called “bits.”)

Surprisingly, 1,000 (which is a power of 10) is not a natural number for a computer. Instead, they work with powers of 2. See if this string of numbers looks familiar:

1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024…

Yup. Each number to the right is the number to its left multiplied by two. So, when a computer formats a hard drive, it does so using these power-of-two numbers, which results in storage sizes like this:

So, when you take a drive and format it, about 1% of total storage capacity is used by the drive for file directories and other technical stuff. The operating system, as it structures the hard drive for use (the process of formatting) also reduces the total available size.

RAIDs also reserve a significant amount of storage for data protection — roughly 25% of total storage capacity.

Finally, you lose storage due to the differences in defining the numbers between the simplicity of marketing and the technical realities of how computers actually work.

This is why a 1 TB drive formats to about 931 GB and a 500 GB drive formats to about 465 GB.

For the technically inclined, here’s a very nice Wikipedia article discussing disk capacity and access speed you may find interesting.


UPDATE — MARCH 23, 2010

Robin Harris provides some additional information:

Saw your piece in the newsletter about disk size reporting. Right on – as far as it goes. In Snow Leopard Apple fixed the base2 reporting bug by going to base10.

 

Now Snow Leopard reports the same size that your disk drive says it supports. See blogs.zdnet.com/storage/?p=589

 

Another key point: the percentage difference between reported size and actual size keeps growing with capacity; i.e. the disconnect gets worse. [Robin then included a table, which I am summarizing here. For instance, using two different units of measurement — 1024 vs. 1000, there was about a 2% difference in storage when total storage was measured in Kilobytes, but a 10% difference when storage increased to Terabytes.]

 

The fact is that kilo, mega, giga, tera and peta are formal international standards that have been in use for over 200 years. Kilo=1,000 and so on.

 

I think the confusion arose because DRAM is measured in the powers of 2 because that is how CPUs access it. Since RAM was very expensive and disks were small, computer folks didn’t differentiate between kilo and kibi (the official power of 2 prefix for 1024).

 

Apple did all Mac users a favor by fixing this error.

Larry replies: Thanks for this clarification, Robin.

 


Bookmark the permalink.

One Response to Thinking About Storage: Why Don't Hard Disks Hold As Much As They Say?

  1. Michael Cox says:

    How LITTLE free space should I leave on an external, non-boot drive which is being used solely to archive quicktime media files? I am trying to free up other drives, and shuffle as much as I can onto two external drives, but I know that I cannot fill them to their rated capacity.

Leave a Reply

Your email address will not be published. Required fields are marked *

Larry Recommends:

FCPX Complete

NEW & Updated!

Edit smarter with Larry’s latest training, all available in our store.

Access over 1,900 on-demand video editing courses. Become a member of our Video Training Library today!

JOIN NOW

Subscribe to Larry's FREE weekly newsletter and save 10%
on your first purchase.