Posts tagged with "databases"


SQL tried and true, but WXR still horribly broken

The Titanic

I've been blogging for long enough to have suffered a couple of disastrous data losses, but only once have I irreversibly lost stuff. Consequently I backup my blogs daily, but on a hunch early this morning I decided to test my backups on a local installation of WordPress. I choked!

Every evening my server is configured with a couple of cron jobs to do these backups which then get the bzip2 treatment:

  • An SQL query
  • A WordPress RSS/WXR file
  • A grilled cheese sandwich with avocado and gherkins

The SQL backup is fairly vanilla stuff, and always works, as one would expect. WordPress's automatically generated RSS/WXR files are much easier to work with, but from repeated painful experience over many years they're unreliable as heck. Perhaps I should phrase that to say "easier to work with... when they work"!

You're telling us this... why?

I belabor all this because when I tested the WXR files my server has been exporting lately, they don't include categories when you reimport them into vanilla WordPress installs. None. Nada. Zippo. This despite setting the PHP memory ceiling higher as I talked about before and splitting up the WXR files as recommended by various folks.

Despite being introduced years ago, WXR is still horribly broken. As far as I can tell from trudging through the source WordPress doesn't even use a XML parser when importing them. I suppose that's another reason why I don't use sites like WordPress.com, when things like this mess up I can always access the database directly, and why on smaller projects I use SQLite3 databases which you can cheat on and backup by just copying over one file! Ah and I'm nostalgic for sqlplus already ;).

What do you guys do for data backups for your online stuff?


The Sun Oracle Database Machine

Sun Oracle Exadata V2 server

Ever since I postulated back in April as to what a combined Oracle-Sun company would do, in the back of my mind I've also been thinking what it would look like. Now we get a glimpse in the form of the Oracle Exadata V2 server, the second advertisement after their triumphant "Oracle Buys Sun" faux billboard graphic.

I'm surprised they kept the Sun Microsystems logo and the basic design of the hardware, but I'm even more surprised they positioned the Sun logo above the Oracle one. I get the feeling it's not something that should be read into much, but perhaps it's a symbolic tip of the hat and to appease people scared about what Oracle may do with Sun. Don't worry guys, Sun is still alive, and all that. Or maybe it's because "Sun Oracle" is much easier to say than "Oracle Sun". I have no idea what I'm saying here.

Most likely this is a transitionary product, we'll probably see completely new designs in coming quarters that probably won't include a Sun logo. Then again when Commodore Business Machines bought MOS Technology in the 1980s they ended up rebranding the subsidiary but it largely operated independently and still put their logo on all their integrated circuits, so who knows.

Sun Oracle Exadata V2 server

Personally I think it made far more sense for Oracle to be Sun's suitor instead of IBM given that they already had a lot of money and time invested in parallel products. Solaris is an excellent operating system, and an Oracle database on Sun software and hardware is a compelling mix.

I guess I'm still just a little sad to see such an iconic company slowly get eaten by another; as I've repeatedly said here I really admire Sun Microsystems. Perhaps I'll have a greater chance of owning my own Sun workstation after buying one on eBay after a few years. That is, if having original Sun hardware without an Oracle sticker doesn't increase its value too much.


Work struggle between SQLite over MySQL

A Green Hummer. I take that to mean just the colour!

It seems as with many technologies, regardless of your own abilities or competency if you’re using the "wrong" implementation you’re perceived as an amateur, and vice versa.

For example, I've read so many disparaging remarks by rude people on forums saying that people who run Ubuntu and Fedora are amateurs, and while I admit to being a FreeBSD and Debian guy I still replied by asking them to prove the automatic connection they were alleging. The inventor of the Linux kernel uses Fedora, so clearly their blanket assumption was false!

Anyway I'm getting sidetracked, in this case I wanted to talk about databases. I was implementing a simple website for a client that only involved one editor and a limited number of pages, so I opted to use SQLite instead of MySQL (or PostgreSQL) because its far simpler to implement, doesn't require a client/server connection and under the limited loads they'd be experiencing it could be faster.

I was so impressed by how well it performed and how easy it was to use I've started looking at other situations where it may be useful instead of just assuming everything should use MySQL. My blog here for example only has one editor and a limited number of tables with very simple foreign key relationships and a few thousand tuples. WordPress doesn't support SQLite natively, but I've been writing some basic blogging software of my own in Django and Ruby for a long time and when I started using SQLite with them I was really impressed.

But back to the client I was talking about before, no sooner had I proposed the solution and had a working prototype than someone working on another part of the project informed everyone I wasn't taking the task seriously because I was using SQLite which isn't a proper database. I didn't want to start a fight so I heaved a heavy sigh and rewrote the system in MySQL which I still maintain is far too heavy and has features that will never be needed for a database with only three tables and less than a hundred tuples each, but at least now I've been told I've corrected my childish approach.

The person did have a point, SQLite does implement only a subset of the features and SQL commands of MySQL; for example it doesn't use static typing though it can be worked around by using constraints. I also concede the possibility that in the future their needs may necessitate a more feature complete SQL implementation. To me though that's akin to Microsoft claiming Vista was better than Mac OS X Leopard because it was more efficient when using nine trillion cores, or that a Hummer is better than a bicycle because you can haul drums of nuclear waste in it more easily.

I guess I just like elegant systems that have a comfortable ceiling for growth but that at the same time are simpler and therefore easier to maintain and use. Einstein said "Make everything as simple as possible, but not simpler", and then there's the adage of the "right tool for the job.". Perhaps I just need to get used to eating humble pie more often and accepting that I'm working with other people with their own needs and expectations. Still, it'd be nice if people would stop assuming things just because of the tools you use.

Aside: I've hacked together a way to post blog entries from the computer pools at the university, but it's such a long and convoluted procedure I still won't be doing many of them until I get our home internet connection fixed.


Whoops, Oracle does auto not null primary keys

Update: It's been brought to my attention that not defining primary keys as not null does have an impact after all, refer to Dale McGowan's comment below. Suffice to say, I will continue to define them as not null from now on!

Much like a musician who despite getting the right sound realises they've been playing their instrument wrong throughout their career, I often find I'm not using computer software properly or I've been wasting effort. I guess I get myself into a predictable routine that I'm used to.

In this case I found out something about Oracle databases. I've been using them for years, but it took a practical class at university yesterday to realise I've been wasting time with a simple line of code that was completely unnecessary.

If you do any sort of work with databases you'd probably want to smack me on the head for this, but I didn't know that primary keys are automatically assigned as not null; presumably because it doesn't make logical sense and it would break a table to have a primary key that has null values!

Up until Friday this is the schema I would have created and entered into an Oracle database:

create table Jackarse (
  refID char(6) not null,
  name varchar(32),
  grilledCheeseSandwich boolean,
  primary key (refID)
);

And here's what I've since learned I can do instead:

create table Jackarse (
  refID char(6),
  name varchar(32),
  grilledCheeseSandwich boolean,
  primary key (refID)
);

That's right, both these schemas will result in the same table being created in Oracle. I guess it wasn't harming anything having the not null command there, it was just unnecessary.


Should blog posts be treated as time capsules?

Apple Time Capsule
That wasn't quite the "time capsule" I had in mind...

Having spent the better part of the afternoon and evening here working on fixing broken links caused by my recent abrupt domain and webhost move, I encountered a potential problem to do with posterity. All the world's problems pale in comparison to it you must understand.

The problem was, internal links between posts on my blog here that were created before the move pointed to addresses at the old URL. With some SELECT REPLACE requests to my MySQL tables here I was able to change all these links from over five years of posts to point to this new domain (well actually this domain is older than the one I moved from, but that's beside the point).

Once I'd cleaned up these links though, I started getting carried away. I started converting some categories to tags that these older posts had assigned to them, then I started optimising some of the images they contained and reuploading them. Previously I had also added header images and text explaining the context in which old posts were written, and I replaced one entire post from late 2005 with a grilled cheese sandwich I had cooked but dropped on the floor and was subsequently reluctant to eat.

Old blog post from 2004
Screenshot of an old blog post from 2004 modified with a header thingy and whatnot

So here's what I got to thinking: Is it really right to be doing this? I've been reading a lot about the concept of a digital dark age (Wikipedia) where original material gets lost online because it's in a constant state of flux and some material inevitably goes offline never to be seen again.

With this in mind, should blog posts be treated as time capsules? As in, when you create a blog post and publish it, to be honest should they never be changed? In my case I didn't change any of the content per se, I just changed the links and optimised the images, but the text and the images themselves are the same. I know plenty of my earlier posts have spelling mistakes and the images aren't aligned correctly, but I haven't felt right changing them. As far as I care, posts from 2005 were written in 2005 and look the same as they did in 2005.

It's funny how ideas and quandaries that seem really important when they're floating around in your head seem extremely important, but when you explain them by writing them out on a blog post they seem so utterly ridiculous in their irrelevance.

Hey, at least I spelt "quandaries" correctly.


Scatterbrain thoughts on the Sun Oracle deal

Showing my support by running OpenSolaris in VirtualBox on my Mac
Showing my support running OpenSolaris in VirtualBox on my Mac :-)

Well it's official, Sun Microsystems found a suitor not in IBM, Apple or Microsoft, but with Oracle. I must say on the whole I'm cautiously relieved, though there are some things that worry me. What an ambiguous sentence. Well you know what they say, certainty is hobgoblin of the fool... right? Wait, that's not how it goes.

OpenSolarisFirstly, Java and Solaris absolutely make sense for Oracle; I believe I read somewhere that (paraphrasing) Larry Ellison claimed Solaris was the most effective Unix system out there for use with Oracle infrastructure. I haven't played with OpenSolaris (or Linux for that matter) to the extent I've used FreeBSD but I was really impressed by it's completeness and quality. I'm also interested in licencing; as a BSD guy the CDDL that has prevented Linux users from adopting ZFS and Dtrace hasn't bothered me but Oracle's work with Linux may see it change to the GPL... maybe.

What I'm also interested in is Sun's free and open source acquisitions and assets, namely NetBeans, VirtualBox, OpenOffice.org and MySQL. If they keep them and continue to support them I don't see any problem, and one could argue that said projects would benefit from being part of a larger company in the same vein as Linux at IBM. Some brainstorming on each:

  • NetbeansNetBeans would compliment their Java acquisition, but Oracle is a signed member of the Eclipse foundation. Will Oracle attempt to merge it, ditch it or continue to develop it and instead ditch Eclipse?

  • VirtualBoxVirtualBox at Oracle is a fascinating combination. Could Oracle somehow leverage VirtualBox's virtualisation technology to more efficently deploy servers with Oracle's database? They could do it with Solaris too!

  • OpenOffice.orgOpenOffice.org is a real wildcard. One could potentially see Oracle leverage OOo against Microsoft, or perhaps they could pull a Novell and instead fork it into an independent stream and an Oracle branded product that could use groupware Oracle develops. The possibilities are extremely intriguing.

  • MySQLMySQL for me is a major concern. Oracle's databases and MySQL are light years apart in features and from what I've heard scalability, but the gap is much narrower than it was even a few years ago. Is it conceivable Oracle would intentionally cripple MySQL or keep it with only a subset of features so as to not cannibalise their bread and butter? If they do, might we see MySQL forks or even a general move to PostgreSQL?

I've done work for Oracle in the past but I still don't know the inner workings of the company or the details regarding merged product lines (ala PeopleSoft and Siebel). Whatever the outcome, I'll be watching closely.

As I've said here before I have huge respect for Sun and have always wanted a pimped out Sun Workstation to really get stuck into Solaris. In fact if I were offered a Mac Pro or a similarly high end Sun Workstation I'd take the latter just because I've used them at university but ave never had one of my own before!

Jonathan Schwartz's weblog at Sun Microsystems

The biggest question I still have though is: what will happen to one of my top technology idols Jonathan Schwartz? Will he get a position in Oracle? Will he continue to blog from there? Or perhaps a more pertinent question would be: will he be allowed to blog from there?

I wish Sun the best in this transition period.


Rubenerd Show 247 2008.06.27

Click for larger imageThe barely listenable web development episode!

Hashing out and talking through some ideas about the future of Rubenerd.com and how I want to move my five separate sites over to it. Might be listenable for a pleasant background distraction if you mess around with weblog engines, databases and whatnot, otherwise don't feel bad giving this one a miss. Looking at Dave Wares' Photo Gallery as an example of where I want to be.

To make it somewhat more interesting, I've included some retro audio easter eggs :-).

Download MP3 to listen ↓ 01:12:11 33.1MiB

You can also stream this episode and view its Internet Archive page.