Posts tagged with "storage"


Using UUIDs in Fedora's fstab file

Fedora icon

As with my beloved FreeBSD, Fedora has a /etc/fstab file that lists partitions to be automatically mounted on boot, but with one important difference: Fedora uses a partition's UUID and not its label.

Why?

I was all ready to pose a question in a newsgroup myself, but fortunately Bill Nottingham from this old thread from the Fedora 9 days put it simply:

UUIDs are unique. (In theory, anyway.) Labels aren't.

Bill

Fair enough, I suppose you could have unintentionally labelled two of your drives the same thing. I never have because I like to use unique labels that match their mountpoints to keep things simple, but I suppose if Sarah Palin or Stephen Conroy ever installed Linux they'd probably try a stunt like that.

Anyway so it seemed if I wanted my brand new formatted drive to be mounted when Fedora booted, I needed to find out what the new drive's partition UUID was instead of just using the label I'd just assigned to it.

How do I find out a partition's UUID?

Good question. A cursory Google search returned this page from ServerFault which lists a dizzying array of options with plenty of justifications. blkid worked for me:

# blkid /dev/sda
> LABEL="moe" UUID="#-#-#-#-#" TYPE="ext4"

If this doesn't work, you may not have /sbin in your $PATH. In that case, just run it from that folder, no worries.

Once you've got the UUID, you can finally add it to /etc/fstab/ along with the file system and mountpoint. One extra step compared to FreeBSD, but not too much of a biggie, to use the technical lingo.


Some Western Digital drives have EARS

So I want to get a new 2TB drive for my FreeBSD and Fedora tower. Looking at the Cybermind pricelist most of their Western Digital drives referenced EARS, an acronym which I'm assuming has nothing to do with the drives being able to detect audible messages from their operators, or allow them to operate a European aerospace conglomerate. Oh wait that's EADS, never mind.

Now hear this, sorry, really bad pun

It turns out EARS is (amongst other things) a new formatting method for Western Digital drives that uses 4KiB blocks instead of the regular 512 bytes. It seems to me like its a cost cutting measure and would decrease drive density due to greater file size rounding. As far as I can tell no other manufacturer is doing this yet.

Potential performance and density issues aside though, what I'm more interested in is compatibly. Scouring FreeBSD and Fedora web forums it seems a lot of people have been having issues with these drives using any OSs other than Windows Vista or Windows Vista Second Edition/7. Whether it's something FLOSS OSs will adapt to in the future I can't say, but for the time being it makes these drives useless for my needs.

My new Logitech powered USB hub!

Insert required personal experience here

On the whole I've had good experiences with Western Digital drives; I was burned quite literally by a Seagate a few years ago when it irreversibly destroyed the desperately needed FireWire port on my MacBook Pro and Maxtor are really just a cheapie brand for Seagate now. Then again I've also heard plenty of horror stories from people about WD drives, including ones that WD use refurbished drives in some of their external enclosures.

I suppose hard drives are universally bad and it all comes down to user preference. It's a clever ploy because drive manufacturers can skimp on more stringent QC and lower tolerances, and then tell people they need to buy more drives for backups for when their crappy drives fail. It's one of the few consumer goods that are terrible but can be fixed by buying even more, it's genius!

Is there an economic model that shows a correlation between the demand for a good and a decrease in quality at the same price? It wouldn't be an elastic or Giffin because price isn't the reason people buy more, but it wouldn't be inelastic because that shows a sustained amount of consumption regardless of price. Or perhaps it would be then? Its been a while since I've studied economics ;).


An arm, a leg and a Drobo

So after hearing Scott Bourne rave about the Drobo intelligent redundant data backup system on MacBreak Weekly so many times I decided to finally look into it. Drobo doesn't sell machines outside the US or Canada but they do have links to foreign distributors, one of which is Streetwise Australia:

Drobo: The world's first storage robot. Fully automated storage you don’t have to manage.

Chances are, you get passionate about creating or collecting digital content, not about managing the storage where it lives. For you we’ve created Drobo, the first robotic storage device that takes the pain out of keeping your content safe.

AU $779

My data is extremely valuable to me, but even value must give in to economic realities. It may be an amazing device, but spending almost AU$800 on such a device when it doesn't even come with any drives is completely out of the question. I respect Scott Bourne, but that's not to say I can ever afford his recommendations.

AU $779 ???

Guess it's back to Rsync between my internal MacBook Pro and ThinkPad X40 drives and my Stonehenge of individual external hard drives each with their own power supply and data cables. Its a messy, seething mass of twisted electronics that takes up an entire tabletop and an almost two full powerboards, but it works.


The optical and magnetic media tag race

Photo from the GE Global Research Blog
Photo from the GE Global Research Blog

Reading an article Big Tom shared on Google Reader, I came across a post on the GE Global Research Blog regarding a breakthrough in material development that could lead to optical disks with a 500GB data capacity. One can only imagine how many jaffle recipes this could contain.

ASIDE: I was going to say I "stumbled" upon a post rather than "came across", but one Web 2.0 service has somehow ruined that phrase for me!

Reading further in, it appears the holographic technology that would make such large capacities possible is due to the data being stored in three dimensions by altering the reflectivity of certain areas of the disc. In traditional optical media such as CDs and DVDs the data is instead encoded as a series of pits or by causing areas to become transparent in the case of recordable media.

Icon from the Tango Desktop projectThe potential uses for this technology are huge. The BBC reported a series of Hollywood movies could be stored instead of purchasing multiple BluRay discs, but that the market for such discs is uncertain. I guess it's natural to gravitate towards movies and audio when thinking about these technologies, but I'm far more interested in data storage, specifically data archiving.

It seems whenever a new optical recording technology has come along there’s a several year window of opportunity to be able to practically use blank discs to backup data before magnetic hard drives grow too large. With our first CD burner back in 1999 I was able to backup my internal 10GB drive onto 650MB discs without hassle. When we got our first DVD burner we could backup our internal 80GB drive onto 4.5GB disks without hassle. Before the DVD burner, backing up 80GBs worth of data onto CDs would have been a nightmare, just as backing up my current 4TB+ of data would be with DVDs now.

If this new optical technology were to come to pass and become commercially available, it would represent the largest real and percentage term jump in digital optical storage and would once again make backups practical. I could backup my drives without hassle. That is, until hard drive manufacturers develop 10, then 20, then 50, then 100 TB drives and we come crying back to the optical developers to come up with a new disc format to save us!

The classic parallel port Zip drive

Unless Iomega comes up with a 10TB Zip disk. I miss my Parallel Port Zip drive and the stylish translucent USB 250MB drive I used with my first iMac. They used to make the cutest clicking noises... until the clicks started getting louder when they ate disks.

A Zip drive ate my homework once when I went to print it at school. I'm serious!


Nobody will ever need 16GiB, right?

Screenshot of Angus Kidman's ZDnet Australia column earlier today
Screenshot of Angus Kidman's ZDnet Australia column earlier today

Angus Kidman writes for ZDnet Australia in his Snorage [sic] column, named as such because in his words: "if everyone thinks storage is so boring, how come we always want more of it? Go on -- you know size matters."

Well apparently he does think that sometimes size doesn't matter, and has gone on record to suggest that nobody would need more than 16GiB of memory in their phones. From the article:

Pronouncing that a given device doesn't need any more storage is a near-foolproof recipe for looking stupid somewhere down the line. However, I'm sceptical that many people need a 16GB mini-SD card for their phone.

SanDisk next month will start offering a 16GB microSD card, which — unsurprisingly given the format — is aimed at the mobile phone market.

That amount of capacity inevitably leads to the question: how are you supposed to fill up that space? Pictures might be one potential answer, but 16GB will give you a hell of a lot of semi-quality phone shots. Indeed, the whole Pictures folder on my PC doesn't take up that much space.

But if this is you, then I suggest you re-compress your movies into a more screen-friendly alternative and stop overloading your PowerPoint presentations with meaningless graphics. Save the capacity for where you really can use it — on a desktop PC where the OS will reclaim it in the blink of an eye.

At least he made one valid point in his introduction: storage ceiling predictions are almost universally wrong. I don't think I need to bring up the old Bill Gates 640KB of RAM chestnut again to demonstrate!

The fact of the matter is we used to think that the diminutive amount of space on a SIM card would be enough for mobile phone users. After all, you can store a few thousand contact numbers on one, what more could you possibly want to put on a phone? Then people started demanding the ability to store more than just numbers about their contacts, then they started demanding the ability to do calendering and other organiser like functions to replace the PDAs they had to carry in addition to their phones. Now we have GPS, mobile internet pages, conferencing, Twitter and instant messaging, streaming music and static audio files, video, photos and graphics, office productivity applications, grilled cheese sandwich makers, waffle irons, nuclear reactors and Secret Squirrel automobiles in our phones, and who knows what we'll have in another few years?

We can have a philosophical discussion on whether or not such things are useful for a phone to do or whether they're counter-productive and restrictive until the cows come home, but the fact is people are doing more with their phones now than we could have ever imagined even a decade ago let alone when the first portable phones were released.

MicroSD card size comparison
Photo I took this evening, comparing a sim card, MicroSD card and an audio CD. I still can't believe how tiny these cards are!

Mr Kidman argues he can't foresee any use for 16GiB of memory for a phone and therefore doesn't see the point of it at all. Just like how 10 years ago nobody thought you would need more storage space than what SIM cards offered. It might be true now that only a few people have oodles of data (I like the word oodles) on their phones, but that's not to say that therefore nobody does, or that nobody will in the future.

I'm fascinated by storage; it's the reason why I spend so much of my free time researching new storage technologies as well as vanilla file and multimedia compression standards, encryption, efficiency and data centres. I'm fascinated by how far we've come in storage capacity, density and size since computing began, and am excited by where we'll be going next. I thoroughly enjoy reading Angus Kidman's column because I can tell he shares the same passion, but this time he does seem a bit far off.

For what it's worth, I'm typing this post on my 16GiB iPhone 3G which is 95% full, so I know of what I speak! Then again I am weird in that way. I mean wired. I mean, wireless, this is a phone we're talking about. A mobile phone not a terrestrial phone. Terrestrial sounds like terrorist, better make sure I don't get blocked in some countries for that remark. Remark sounds like Renmark, a town in South Australia. Which is convenient, because this post went south with this last paragraph it seems. Seams, that reminds me, I need to have my slacks repaired. Or should I just buy new ones? Ones and zeros... just like file storage. See, I did come back to the point of this post, even if it was at the very end.


My kingdom for a bigger notebook hard drive

An ominous sign of things to come?
An ominous sign of things to come?

It's crunch time: alas after months of neglect and with so many assignments and projects active and being worked on at any one time, my internal 149 gibibyte (aka 160 gigabyte) hard drive has finally been maxed out. Bummer!

Having used Windows, Linux, FreeBSD, NetBSD and Mac OS X on computers with drives that are nearing breaking point, I do appreciate how incredibly stable the BSDs and Mac are under capacity stress. By comparison the general wisdom with Windows (at least when I still exclusively used it before 2003) was that you must reserve at least 10% of your drive at all time to maintain stability, NTFS included. By comparison, this MacBook Pro has been close full for a while now and still only EyeTV and the slow as molasses Microsoft Office 2008 for Mac applications are capable of crashing it.

As I've discovered the hard way though, notebook computers present their own set of storage challenges! For most of my life I've been a desktop computer user; it was only in 2006 when I made the decision as a computer science student studying overseas that a souped up notebook computer would make more sense for taking around to different houses and around campus than a new desktop.

Of course the problem with said notebook computers is that you can't just easily slide in an extra hard drive when your existing one starts to fill up! Sure you can buy external drives, but they still won't match the performance of the internal drive. What happens then is I tend to backup material to the external drives, but projects I'm compiling, editing video for or otherwise working on end up staying on the internal drive.

VMware Fusion 2.0 beta 2 New Virtual Machine Assistant
What can I say, I love toying around and exploring operating systems!

This is also a problem for virtual machines which I spend lots of time using and writing about. To satisfy my own addiction and fanaticism for studying operating systems, as well as for my work which involves compiling and testing applications, I have multiple VMs on this internal drive. Running these virtual machines on an external drive is completely out of the question given the performance would really, really suffer. Having 12 virtual machines which combined take up 72GiB on a 149GiB notebook drive though is also completely out of the question!

With my desktops in the past I tended to dedicate a smaller drive with the fastest RPM for the operating system and two larger, equally sized drives mirroring each other (later using RAID instead of software) for the data. On my current desktop back in Singapore which I SSH and SFTP into from here in Adelaide I have FreeBSD 7.0 AMD64 on its own dedicated, 10,000RPM SATAII drive with 32MiB of cache shared with binaries, and two 7,200 RPM drives for the home directories, port collections, documentation and served data. Ideally I'd love to have another super fast drive just for /swap too!

On this laptop I've got everything under the sun on one drive. Perhaps partitioning the drive and assigning the /Users directory to a secondary partition might help to compartmentalise the information and improve performance. On BSD and Linux it's trivial to assign the /home directory to a separate volume, on Mac OS X I'm not so sure. Seems like I have some homework ahead of me!

This much taken up, on a 149GiB internal notebook drive. Bummer!
This much taken up... on a 149GiB internal notebook drive.

Of course it probably wouldn't hurt cleaning this drive out either. I have a few Ruby scripts which I run each afternoon which cleans up my desktop and puts files in the appropriate places, but it can't determine what is safe to delete and what isn't. I need an electronic secretary I think. Make someone sign a NDA, then go through my drive and get rid of things. No, wait... perhaps that isn't such a good idea.

As my fabulous father always says after ringing me from his office in Singapore which has more paper, books, phone receivers, emails and blood pressure tablets than Parliament House: "All I need is a time machine Ruben... then I'd work just fine"