Tales of Merging Three Copies of Movable Type Into One

Published on 2 February 2008 in , ,

A few weeks ago, I mentioned that The F-Word needed a new web home – and after some discussions and debates, the switch was made on Friday to a new home on a shiny Bytemark virtual server on the basis that whilst it cost more, it did have probably the best level of flexibility for the site (now it just has to be used!)

The actual migration of web servers wasn’t that much hassle in many ways. I had the new virtual server up and running in next to no time. The more interesting problems came from trying to sort out the tangle of three Movable Type instances that The F-Word had ended up with.

The background

I guess I should explain how that rather absurd situation began.

In the beginning was the word. And the word was written in static HTML files, built using the Hitoplive HTML pre-processor.

It worked, but as with any site which grows and grows, you can only use static HTML for so long before it gets too big and too insane to handle. During 2003 and 2004 a major headache/project was started to move everything over to Movable Type 2. This sat on my home PC and Catherine just ftp’d the files up when needed – the same way this site was built for many years. Once the licensing issues were sorted out, it was upgraded to MT3.

This was all fine until 2005 when Catherine wanted to add a Blog to the website. Obviously this meant that a copy of MT hosted on my PC wasn’t particularly useful, so I set up a new copy of Movable Type on the webserver for the blog.

So one MT became 2.

Then everything went quiet, bar me slowly moving everything over to PHP (laziness means that actually, a lot of my personal stuff still uses Hitoplive – it’s no longer actively developed, but continues to work nicely).

Then in 2007 Catherine stood down from day to day editing of the site, handing over to Jess McCabe. Now again, the fact that MT was on my home PC was a problem as Jess, obviously, wouldn’t have wanted to pop down on the Northern Line every time she wanted to edit the site.

Because the main site was set up in a more complex way to the Blog (including the use of the Custom Fields plugin), and was co-incidentally running off Berkley DB, it was easier just copy it over the entire install of MT3 from my machine to the webserver. The plan was, at some point, to import the Blog entries into the main Movable Type.

Of course I never got round to it.

Partial Upgrades

Then Movable Type 4 came out.

Because the blog MT was always due to be merged in with main MT, I didn’t upgrade it from MT3. But the main MT was upgraded.

And then we hit the snag.

Although it was supposed to migrate the data across, the new version of the CustomFields plugin lost all the data associated with about 200 articles. Thankfully these were all in the Resources section, but at the time, I didn’t have chance to re-enter all the data.

So Resources got left in the old copy of MT3.

And lo, there were 3 – main, blog and resources. One on MT4, two on MT3.

Frankly, a bit of a mess.

And so, to the present

With the webhosting changes, I decided enough was enough, and with the recent changes in the licensing for Custom Fields, I came up with an action plan:

  1. Migrate hosting of static files
  2. Combine all three Movable Types into one
  3. Remove Custom Fields plugin (the way it was being used meant it was relatively easy to just merge the content into the main Entry Body area)
  4. Move the database to mySQL
  5. Upgrade to Movable Type Open Source v4.1
  6. Move most of the PHP code to use Movable Type tags instead.

Step 1 was a doddle and a lot of step 6 got canned when I remembered that there was about 200 static pages which never got migrated to Movable Type because it wasn’t worth it. Thus it was easier to keep the PHP code instead.

Steps 2-5

By this point all the MT instances were using SQLite rather than BerkleyDB, which was handy as by this point, the strains on the old webserver were showing and Movable Type was often getting very very slow. It proved to be easier to copy the SQLite database files to my home PC and do much of the work there instead.

I also set up a brand new, clean, install of Movable Type Open Source on the new webserver, hooked up to mySQL.

Converting the main Movable Type was a doddle as this was running on MT4. I copied the database to my home PC, and used MT4’s inbuilt “Backup” option – this is a lovely feature that just dumps a XML version of the database out.

I then used the “Restore” feature on the new MTOS to import all the data in there. Frankly, it was a doddle.

Then there was the Resources Movable Type – still in MT3 and using the CustomFields plugin.

I still had a copy of MT3 installed on my home PC so again copied over the database.

As I wanted to merge the CustomFields data into Body Text field, I set up a new template which output data into a custom version of Movable Type’s “export” format.

This did result in some formatting issues, meaning I had to do some hacking of the resultant file before MTOS’s “Import” function would accept it. However it was not major.

Converting the Blog version

This was where the fun was to be. The simple method would have been to merely use MT3’s Export function, but this would have meant new passwords for the 10 bloggers. And because I’m a nice person, I preferred the idea of just having their old passwords work.

The obvious solution was to migrate the Blog Movable Type to MT4 first, then “Backup” the data into the new MTOS.

However most of the Bloggers also had accounts in their name in the old Movable Type (they were non-login accounts where the Editor could enter the article, then assign the correct author name).

I had a feeling that doing a Restore of data this way would either create duplicate accounts, or just use the existing user account details. Or worse, it would have got all the accounts mixed up – with articles assigned to the wrong authors left right and centre – this has happened to me once before and meant several days untangling the resulting mess.

The answer was obvious. Rename the user accounts already in MTOS, and then use “restore” to get the Blog data into MTOS. Just to make sure, I hacked the Restore XML file to change all the author IDs in the XML to be completely unique – with two exceptions.

One Restore later and all the Blog data was happily combined into MTOS. Three MT’s had become 1.

Account Fun

I then merged the duplicate accounts together, by re-assinging articles from main MT login, to the new account created by the Blog “Restore”. Only consisted of about 20 articles so not arduous.

Now there’s the two exceptions to deal with – these two I didn’t want the Blog password to be used, but the original one instead. In both cases, I’d re-assigned the author IDs in the Restore XML file to match an existing user account. Sorted. I thought.

For one account, all was fine.

The second account was another matter – and not because of a Movable Type problem. I’d put the wrong ID in! Typical luck, it was the account with the most articles too.

Over 800 articles assigned to the wrong author…

Now if there’s a quick and easy way to reassign 800 articles from one author to another in Movable Type, I don’t know what it is. Instead I had to use the Batch Editor to change the ownership details one article at a time.

Tab Tab Tab Tab Tab Tab, press letter J twice, repeat again. Whoops, pressed it three times by accident, go back and sort the mess out…

Changing the ownership of 800 entries in Movable Type is that it is dull, dull, dull. You don’t want to do it. Trust me.

One tired hand later…

Actually that painful endeavour aside, I was quite impressed by how straightforward the combination of three instances of Movable Type ended up being. Despite a couple of minor teething problems with user publishing permissions, most of it worked as I’d planned, and it would been extremely smooth had I not managed to put that wrong user ID in.

Of course the sensible thing would have been not to have three different copies of Movable Type for one site in the first place. But then, there’s a lot that can be said for hindsight…