the vengeful cow blog

The great migration

Posted by marshall Mon, 09 Aug 2004 14:57:00 GMT

It all started so innocently. I asked my web hosting service about a way to let users change their own passwords. Little did I know that three days later I would have replaced my blog software, my site design, and my server platform...but not before passing painfully through the fires of CSS incompatibility and broken databases.

Note that this is long and technical. :-)

I have two sites hosted on ServerCentral: this one, which Lara and I use for our weblog, and vengefulcow.com, which I and a couple of friends use for projects. These were both running on Cobalt RaQ hardware, which is a dedicated "server appliance" designed for web hosting companies.

One of my friends who's using vengefulcow.com for e-mail wanted to change his password, and to do so he needed to log into the "personal" area of the RaQ server. Unfortunately, the administration software for the RaQ is rather old, and uses JavaScript specific to Netscape 4 and older versions of Internet Explorer. This makes it incompatible with most modern browsers (e.g. Firefox, Netscape 7, Safari, Konqueror, etc.) My friend didn't want to have to go back to using Internet Explorer if he could avoid it, so on Monday morning last week I wrote to ServerCentral and asked if there was an updated administration tool. They said no, but if I wanted to I could move my sites to one of their new servers, which would give me more disk space, more secure e-mail, and administration tools that worked well with modern browsers. Well, that sounded great!

And so on Monday afternoon ServerCentral ran their migration tool to move the data and users of both sites from the old system to the new system. They then updated the DNS entries so that our domain names pointed to the new system. The mail service checked out, the web service was working, all was well...until I tried to log into Movable Type to post something.

Instead of the administration interface, I got a mess of Perl code.

Movable Type (MT) is implemented as a set of CGI scripts, written in Perl. On the old system, I could run these scripts from the directory in which I'd installed MT. On the new system, however, it looked like these needed to go in a designated cgi-bin directory. So I set about moving files and editing the MT configuration to see if I could get it to work. No luck. Now instead of Perl code, I was getting header errors.

Well, fine then. I had been thinking of replacing MT anyway, so this was as good an opportunity as any. I went out looking for Movable Type alternatives, which is currently a hot topic due to the recent licensing changes. I basically wanted something that was better at dealing with comment spam, easier to work with than Movable Type's Perl CGI scripts (preferably PHP), and didn't cost $70 to have more than one weblog author. And like many others, I found that WordPress was just what I was looking for.

So I got WordPress installed (the setup is nice and simple), and then set about trying to import my MT data. As it turns out, WordPress comes with an import tool for reading MT files -- that is, files that have been manually exported from MT. Not the MT database files themselves. All I had were the database files, and I couldn't log into MT to run the export process. Well, OK, I thought, these are Berkeley DB files. Berkeley DB is an open-source product; I should be able to download the files, run some sort of "data dump" utility, and get the data that way. So I downloaded Berkeley DB, compiled it, installed it, ran the dump routine...and it told me the data files were incompatible. It seems Berkeley DB has no support for previous versions of its own files.

At this point I began to realize why I kept seeing people saying over and over again to use MySQL instead of Berkeley DB with Movable Type. At the time, it seemed like overkill; the default storage for MT is Berkeley DB, why go to the trouble of fighting it? Now I know. If I had set up the MySQL support, I could have just copied the data files, or at least it would have been easy to request a data dump from ServerCentral. With Berkeley DB, however, my data is locked away, and it's nearly impossible to get it out.

So if anyone out there is still using Berkeley DB with Movable Type, get your data into MySQL immediately! You're only asking for pain otherwise!

Well, now what? I couldn't export the entries from MT, I couldn't get at the database...but hope was not lost. MT works by generating static HTML files for each post, and those were also copied over from the old site. I could extract the individual posts and comments from those HTML files, and put them in some format that WordPress could import.

That became my mission for Tuesday: write a script that would extract the HTML for the posts and the comments and save the data to a file in the MT export format. Thankfully, Groovy was there to make writing such a script fairly easy. (I could have written pretty much the same thing in Ruby or Python or some other dynamic language, but lately I've become quite familiar with Groovy in particular.) Fortunately, the MT import/export format is fairly well-documented, and along the way I also learned how to keep the URL's working from the old site, so that bookmarks and search results from the old site don't break.

By the time Wednesday rolled around, I had all the old posts in a file that WordPress could import, but WordPress was still using its default page template, which is rather bland. I could have taken my templates from Movable Type and modified them to work with WordPress, but I figured this was a good chance to create a new, brighter design for the site, one that is standards-compliant and makes good use of CSS and XHTML.

So I got a design worked up, implemented it according to the CSS specifications, and it was all going well. It came together quickly, and it looked great in Firefox and Safari. And then I loaded it up in Internet Explorer.

I spent the next several hours in a mixture of frustration and rage as I hacked around one bug after another in Internet Explorer's CSS implementation. I had heard before that it was bad, but I had no idea it was this bad. Internet Explorer has by far the worst CSS implementation out of any browser in common use, it hasn't seen a major update in three years, and it's the most widely used browser. Aaaarrrgh!

Let's say, for example, that you have an ordered list (numbers in front of each line item), and each line item in the ordered list is in paragraph tags (for spacing between each item):

<ol>
  <li><p>Item One</p></li>
  <li><p>Item Two</p></li>
  <li><p>Item Three</p></li>
</ol>

This works fine across all the browsers. Let's say, then, that you want to limit the width of each paragraph to a certain value using CSS, say 400 pixels, using the following CSS:

p {
  width: 400px;
}

Works fine in Firefox, Netscape 7, Safari, and so on. But in Internet Explorer, as soon as you apply a width, the numbers of the list inexplicably jump to the bottom of the paragraph they are numbering! So suppose you apply the width to the <li> tag instead of the <p> tag. Again, works fine in most modern browsers. But in Internet Explorer, the numbers jump to the bottom of the paragraph, the space between each line item disappears, and all the numbers change to 1!!! What on earth possessed Microsoft to create such a horrific mess and leave it unfixed for three years?!

Well, I finally got it working in Internet Explorer too. Hacks around bugs in that browser comprise about 10% of my CSS file. Ridiculous. But at least it all works now, the entries are imported, and the old URL's still work.

And it all started with a password change. :-)