Rubénerd!

The webs are all a-twitter this evening about Seagate buying out LaCie, the storage company perhaps best known for their premium Mac hard drives, and memory keys in the shape of keys. I can still remember flipping through the catalogues and gawking at those gorgeous bright yellow FireWire 400 drives: they could hold a whopping 8GB! Needless to say, another enduring brand that I grew up is slowly fading away.

Of course, the story is a little more complex than what is generally being reported; Seagate is buying a controlling stake in LaCie, and may buy the company outright with government approval. In light of Google and Motorola Mobility though, I doubt this will be too much of an issue.

Aside from giving me an excuse to get nostalgic again, I only mention this because I only just blogged about LaCie. Looks like I'll need Makise Kurisu's help... again!

While attempting to find the right diagnostic tools for a bunch of drives, in my sleep deprived state I figured if anyone would be able to help, it'd be Makise Kurisu! I should have just gone to bed.

Pixel art by homura on Pixiv.

Getting this error in Red Hat's Palimpsest (Disk Utility)? Use gdisk to define partitions first, then you'll have no problems :).

The problem

Starting with their larger capacity, multi-terabyte/tebibyte drives, Seagate and Western Digital have moved from a 512 byte sector size to 4 kb. The benefits and tradeoffs of this new low level format is beyond the scope of this post; suffice to say it wrecks havoc with certain drive formatting tools!

If you attempt to format one of these newer drives in Palimpsest, you may receive an error similar to this:

The partition is misaligned by 3072 bytes. This may result in very poor performance. Repartitioning is suggested.

Technically the drive is completely usable in this state, but you'll pay a hefty performance penalty.

The solution

At the time of writing this post, both GParted and Palimpsest have trouble with these drives, but gdisk can handle them. Gdisk is a command line based disk editor that works on GUID GPT drives.

I shouldn't need to tell you to back up your data and quadruple check you're performing operations on the right drive. Please be careful!

1. Install from your package manager (no, really?)

2. Launch gdisk as root with the device path of your new drive; you can find that out from Palimpsest's "Device" label. It'll look something like this:

   # gdisk /dev/sd[x]

3. Type n for new partition, and accept the defaults, assuming you want a partition that takes the entire drive.

4. Type "w" to write the changes.

Now you can format the drive with any tool you wish!

Pretty sure this isn't the most elegant way to do it, but setting the /Volumes directory immutable prevents volumes from auto-mounting:

% sudo chflags uchg /Volumes

Then to revert back:

% sudo chflags nouchg /Volumes
% echo The Bird is The Word

Despite being an OS X user since the first 10.0 betas, I know surprisingly little about how Macs handle volumes. Put me in front of a [purely!] FreeBSD, NetBSD or Linux box and I'm set, but Mac has its own way of doing things. Of course ;).

More out of interest sake than being under any illusion of practicality, I decided this evening to try running SpinRite in QEMU on my Mac Pro. The verdict: it works, if you have lots of spare time!

Notes before proceeding

I tested this on a Mac Pro, running SpinRite in QEMU on a non-system drive. I would assume if you booted Mac OS X off an external drive you could try this on your machine's system drive as well, but your mileage may vary.

QEMU is easy enough to build yourself, or its available on Homebrew, MacPorts, Fink and pkgsrc.

Finally, this action is allowing software raw access to your drive, so be extremely careful about getting the labels and identifiers right. Backup your stuff. Do at your own risk!

The procedure

1. Go into Disk Utility, click the drive you want to run SpinRite on, then go to File → Get Info. Under the "disk identifier" heading you should see a string called disk[number]. Make a note of it.

2. Use Disk Utility to unmount the drive. If it says the drive is busy but you're sure you're not doing anything with it, you can force eject it with its shell sibling:

% sudo hdiutil eject -force /Volumes/[label]

2. Temporarily assign yourself ownership of that volume:

% sudo chown [your username] /dev/disk[number]

3. Fire up a QEMU session:

% qemu -hda /dev/disk[X] -cdrom spinrite.iso -boot d

From here on in, its just like SpinRite on a regular machine... although an order of magnitude slower!

Don't forget when you're done to return permissions to root on the drive:

% sudo chown root /dev/disk[number]

Why go to all the trouble?

SpinRite is a preventative hard drive maintenance utility that is run off a bootable FreeDOS image burned either to a CD or run off a floppy disk. Unfortunately, while it boots on Intel Macs, the software requires BIOS level access to drives which EFI obviously fails to provide. As a consequence, the keyboard doesn't work and even if it did, the drives wouldn't be accessible.

One potential workaround is to physically remove internal drives from Macs, install them in a regular PC with a BIOS and perform SpinRite on it. While this works, its terribly clumsy and doesn't lend itself well to performing regular maintenance as Steve Gibson suggests we use it.

This got me thinking whether or not it can be virtualised. Provided the software had raw access to the drive, theoretically one could create a virtual machine, boot off the SpinRite ISO image and have it do its thing. I'd tried it using raw access in VirtualBox before, but it was complicated to configure and ran as slow as molasses.

Turns out, using QEMU to do this on a non-system drive is fairly simple, though just as slow. Oh well, you live and learn!

My old man came back from his business trip to Singapore, and with him he brought some swag from my old haunt.

Compact flash cards

I know that icon is technically a SmartMedia card, but it was just too cool to pass up ;). I still have a MadPlayer and music player that use those cards. They're flexible. They're like the 8.5 and 5.25 inch floppy drives of the memory card world. But I digress!

I've always had an obsession with retro laptops, but their hard drives have been kicking the bucket. Fortunately they're easy enough to replace with adaptors and Compact Flash cards.

On his last trip, Mr Schade Senior was alerted to 60MB/s cards which are much faster than the existing ones I'm using. The fact they only have an 8GiB capacity is just fine given they're replacing crusty old drives that are all smaller anyway!

My Armada M300, Libretto 70CT and iBook G3 are lining up to have these cards installed :D

2TB Hitachi drives

I could regale you with tales of horror with Seagate and Western Digital drives, but (touch wood!) I've had nothing but great experience with Hitachi drives (and IBM units before Hitachi bought their storage business). They're somewhat pricier than other manufacturers, but I trust them. Holding a Hitachi and WD drive in each hand I could also swear the Hitachi was heavier too, but that could just be me.

Unfortunately higher capacity drives are still too expensive per gigabyte even in Singapore, but now I've maxed out my DIY Sim Lim Square tower and my Mac Pro with 2TB hard drives! Omnomnom!

I have rsync operating as a cron job keeping these machines syncronised, and one remaining external hard drive backs up my critical work every few weeks before I take it back offsite again. I would use a cloud backup service, but I don't trust the cloud. It's not that I don't trust them from a technical or ethical standpoint (they do good work), I flatly don't trust lawmakers. Hey, like Google and IPv6! Metered Aussie internet also kills it, as it does many other things.

In Soviet Singapore, Sim Lim Squares You

I just can't believe less than a year ago I was using a MacBook Pro as my production machine and what I dubbed my "Stonehenge" of external hard drives! Honestly, five external drives each with their own FireWire cable and power supply brick, what a mess!

Sigh, I miss Singapore. Friends. The food. The electronics. The public transport. The all week late night shopping. The ultra high speed, unmetered internet. 3G phone service everywhere. Modernity. The safe clean bubbleness. The more intangible feeling that I'm a part of a global community. Aiyo, I even miss Singlish one lah.

As with my beloved FreeBSD, Fedora has a /etc/fstab file that lists partitions to be automatically mounted on boot, but with one important difference: Fedora uses a partition's UUID and not its label.

Why?

I was all ready to pose a question in a newsgroup myself, but fortunately Bill Nottingham from this old thread from the Fedora 9 days put it simply:

UUIDs are unique. (In theory, anyway.) Labels aren't.

Bill

Fair enough, I suppose you could have unintentionally labelled two of your drives the same thing. I never have because I like to use unique labels that match their mountpoints to keep things simple, but I suppose if Sarah Palin or Stephen Conroy ever installed Linux they'd probably try a stunt like that.

Anyway so it seemed if I wanted my brand new formatted drive to be mounted when Fedora booted, I needed to find out what the new drive's partition UUID was instead of just using the label I'd just assigned to it.

How do I find out a partition's UUID?

Good question. A cursory Google search returned this page from ServerFault which lists a dizzying array of options with plenty of justifications. blkid worked for me:

# blkid /dev/sda
> LABEL="moe" UUID="#-#-#-#-#" TYPE="ext4"

If this doesn't work, you may not have /sbin in your $PATH. In that case, just run it from that folder, no worries.

Once you've got the UUID, you can finally add it to /etc/fstab/ along with the file system and mountpoint. One extra step compared to FreeBSD, but not too much of a biggie, to use the technical lingo.

So I want to get a new 2TB drive for my FreeBSD and Fedora tower. Looking at the Cybermind pricelist most of their Western Digital drives referenced EARS, an acronym which I'm assuming has nothing to do with the drives being able to detect audible messages from their operators, or allow them to operate a European aerospace conglomerate. Oh wait that's EADS, never mind.

Now hear this, sorry, really bad pun

It turns out EARS is (amongst other things) a new formatting method for Western Digital drives that uses 4KiB blocks instead of the regular 512 bytes. It seems to me like its a cost cutting measure and would decrease drive density due to greater file size rounding. As far as I can tell no other manufacturer is doing this yet.

Potential performance and density issues aside though, what I'm more interested in is compatibly. Scouring FreeBSD and Fedora web forums it seems a lot of people have been having issues with these drives using any OSs other than Windows Vista or Windows Vista Second Edition/7. Whether it's something FLOSS OSs will adapt to in the future I can't say, but for the time being it makes these drives useless for my needs.

Insert required personal experience here

On the whole I've had good experiences with Western Digital drives; I was burned quite literally by a Seagate a few years ago when it irreversibly destroyed the desperately needed FireWire port on my MacBook Pro and Maxtor are really just a cheapie brand for Seagate now. Then again I've also heard plenty of horror stories from people about WD drives, including ones that WD use refurbished drives in some of their external enclosures.

I suppose hard drives are universally bad and it all comes down to user preference. It's a clever ploy because drive manufacturers can skimp on more stringent QC and lower tolerances, and then tell people they need to buy more drives for backups for when their crappy drives fail. It's one of the few consumer goods that are terrible but can be fixed by buying even more, it's genius!

Is there an economic model that shows a correlation between the demand for a good and a decrease in quality at the same price? It wouldn't be an elastic or Giffin because price isn't the reason people buy more, but it wouldn't be inelastic because that shows a sustained amount of consumption regardless of price. Or perhaps it would be then? Its been a while since I've studied economics ;).

I've finally found a drive that PartedMagic can't handle, but fortunately I still have a friend in dd. Sounds like a trailer for an episode of Dexter's Laboratory. UPDATE: Even better, use an old Knoppix CD!

PartedMagic is a Linux live CD optimised for drive maintainence with tools such as GParted and Partition Image that I've been recommending and using for a few years. I was an avid fan of PartitionMagic before Symantec bought the company that developed it and killed the project, and PartedMagic filled that void really well.

Unfortunately, the [relatively new] WD hard drive that came with my [relatively ancient!] Libretto 70CT has been the first drive since I started using PartedMagic that the bundled applications simply can't handle. GParted reports that it can't read the data and as such I can't resize the partitions, and Partition Image simply panics and quits.

Is someone eating sushi or are things just fishy?

The Windows 95 partition boots on the machine and works flawlessly, ScanDisk doesn't find anything, I've run SpinRite on level 2 and 4 for several long days, I've done every file system integrity check known to humanity... NONE of them reports problems.

What I want to do is keep the Windows 95 partition intact but shrink it by a few gigabytes so I can run FreeBSD with my ncurses environment I've been developing. As it stands now, GParted/libparted can't read the data in the 95 partition even though it boots fine, the drive is fine and the filesystem is fine. I think.

I read on a few forums that people have had success with FAT systems running older live CDs when this problem occurs, and they surmise there may be an error in the newer versions. I'm willing to give it a shot, in the meantime I'm using dd to copy the whole drive to an image file as a backup, but I need to be able to resize this.

I wonder if hdiutil could do it? Hdiutil can resize non HFS+ partitions provided they're not running the system, can't it? I'm sorry this whole post was an excuse to think out loud!

Update!

Turns out those forum folks were absolutely right, I tried an old version of Knoppix from 2005 and it worked flawlessly! The newer versions of libparted must have introduced a bug with old FAT filesystems, or something.

You can use a real, physical drive as a bootable hard disk in Sun Oracle VirtualBox using an undocumented feature, and it even works on Mac hosts!

First make sure the drive is unmounted (aka ejected), it won't cause errors if it's mounted when you attempt this, it simply won't work.

VBoxManage internalcommands createrawvmdk \
-filename [NEW DRIVE NAME].vmdk \
-rawdisk /dev/disk3

Replace the /dev/disk3 with the block identifier that represents the physical drive you want to access, in my case USB and FireWire drives. You can right click any drive in Disk Utility.app to find out what that is on Mac OS X or your fstab file.

This command essentially creates a pointer to the raw physical drive which you can then add to any virtual machine, just like any other virtual drive. As with creating it though, if it's mounted on the host it won't work. Also this command is undocumented and subject to change, so if the above syntax doesn't work they may have changed it since I wrote this up.

Because it enables raw file access, I'm assuming lower level tools like SpinRite can be used in a virtual machine this way, but I have to confirm this.

For more information, I found this VirtualBox forum topic insanely useful with plenty of relevant links.