Moving Mail From One IMAP Server To Another

I can sum it up with this: What a bleeping pain in the bleeping bleeps.  But here's how I got mail copied from one postfix/dovecot server to another one.

My LAMP Server Is OLD!

I've run www.moultonlumber.com on a box right in the lumber yard since the site existed several (10 maybe?) years ago.  We ran a straight up html site using a terrible design for a while with a pop3 mail server.  Well, the 2001-ish era Dell Optiplex has been making some funny noises lately, and I've run out of parts carcasses to pick from in case it craps out totally.  Throw this on top of me leaving sometime in the next year (I'm planning on being a stay at home dad again) and not wanting to maintain such a box, and you've got some pretty good reasons to start paying for hosting.

So, having had a good experience with hostmonster before (thecabinetfolks.com — we sold kitchens online for a while but not everyone at the lumber yard wanted to make it work as much as I did), I signed us up with them again.  It's about five bucks a month, fairly peppy, they do backups, and I get shell access.

I set up, in Hostmonster's control panel, new email accounts to correspond with the ones on the old server.  That way there'd be a directory waiting for me to throw stuff into when I got ready.

The Actual Copy

The copying process was fairly ridiculous.  I was able to get my own inbox copied late at night without too much trouble.  But it turns out some meathead has an inbox big enough to make most hoarders blush (his weighed in at a gig more than everyone else's combined), and the process of copying that during the day kept turding out.  I even zipped up that fellow's mail directory, but still it died mid-copy.

How I ended up getting the stuff copied finally was rsync over ssh.  If you don't know, rsync is a command in Unix-ish type systems (most web servers) that make one directory contain the same things as another directory.  You can do it with two directories right on the same server, or over a network.  Here's something like the command I ran:

rsync -avz –progress /home/vmail/moultonlumber/meathead/ server-username@moultonlumber.com:/home/server-username/mail/moultonlumber.com/meathead/

I ran it while being logged in to the old server.  When I set that up, many moons ago (Ubuntu 10.04 was new, 14.04 is coming soon I think — both are LTS versions) /home/vmail/ was where I told ALL mail to go.  In /home/vmail were subdirectories, one for each site I was running on the box.  Doopensource.com actually ran on that box for a little while too.

Anyway, there were subdirectories in each of those as well, one for each email account.  So the directory /home/vmail/moultonlumber/craig/ is where ALL the mail for craig@moultonlumber.com is kept.  Likewise, /home/vmail/moultonlumber/meathead/ is where all meathead@moultonlumber.com is stored.  Meathead is not his real name, any more than Micheal's name was Meathead on All In The Family, but moving along…

I wanted the mail to LAND in the correspoding user directory on the new server.  The second half of the command (starting at server-username) contains the directory that will be syncronized to the old server one.  Your command will be structured the same as mine, but will probably look pretty different.  Don't worry, and give me a shout if you run into trouble.  If I'm available, I'll give you a hand.

The –progress switch was especally helpful for me.  I ran this on a Friday (yesterday, 10-4-2013) and wanted to know how the process was doing, rather than just whether or not it shit the bed.  My day started at 5-ish and I'd been up since 1-ish farting around with my profile, and I was getting rather sick of copy errors with no explanations…

Finally though, I ditched the scp method in favor of rsync and had no more trouble.  All of the emails and the directories that contained them arrived on the new server, just waiting for me to make a client see them properly…

Renaming Directories

At this point, I could have been done.  The problem was that I had an extra subdirectory (looking at it in Thunderbird) of my inbox called INBOX, and that's because each of the directories copied over started with .INBOX.xxx (where xxx is the name I gave a folder in Thunderbird — and yes, there's a preceeding dot) and were referenced in the subscriptions folder with INBOX. (no preceeding dot).  The next step for me was to make all of the mail subdirectories directly in the inbox, rather than in the INBOX directory of my inbox.  You may or may not have this trouble, depending on how your IMAP server was set up on your own old box. 

I fired up a text editor (Geany), opened the subscriptions file, and just replaced any instance of INBOX. with nothing.  So INBOX.vendors became just vendors, INBOX.cabinets.bertch became just cabinets.bertch (my favorite cabinet company by the way– Bertch makes awesome stuff) and so on and so forth.  Then I had to rename the actual directories to match the new subscriptions file.  A quickie in BASH:

for i in .INBOX.*; do mv ${i} ${i/#.INBOX./.}; done

did it for me.  I had some trouble with directories containing spaces, but only with Dubba's (Oops, I meant Meathead's) email.  I tend to use underscores (from years of MySQL maybe) in file and directory names as a rule, and probably wouldn't have had any trouble at all if I'd rsync-ed my own email.

Done?

Seems to be working  fine as far as I can tell.  I'm not going to disassemble the original server yet until I'm sure.