Just assume that most people you encounter are clueless, unless proven otherwise!

seamless data migration part 1

May 19, 2009 11:12 p.m.

Moving web sites, databases or any other type of dynamic data from one server to another is very tough to do seamlessly. Now, I don't claim to know it all, but I've been able to successfully pull it off hundreds of times over the years. With a slight few exceptions of course.

As a sysadmin, I can do this successfully because I have full access to the web site, databases and DNS for everything being migrated. If you don't have access to all these things, it may not be as painless as you would like. I'll explain the steps as I've done them and people can improvise where they see fit.

The following example will contain moving and entire site from one network to another... Web data being on one server, database being on another and then finally DNS changes to follow it all up and make it real. Take out of this what suits your needs for your environment.

First and foremost, the new database and web server need to have all the software installed and configured to replicate the original server as much as possible. Of course, the new server software can be upgraded as long as it won't cause problems with legacy code.

Next, if there is a firewall in place (please say that there is!) it needs to be temporarily configured so the new server will allow root ssh connections and database connections from the old server. If you're using IPTABLES as your firewall, you can add entries similar to this.

-A INPUT -p tcp -s old_server_ip -d new_server_ip --dport 22 -j ACCEPT
-A INPUT -p tcp -s old_server_ip -d new_server_ip --dport 3306 -j ACCEPT

Initial data migration

Now, there are several methods available to accomplish this task depending on the data being moved (i.e. tarball, mysqldump, rsync, etc...). Over the years I've come to use rsync (with appropriate command line switches) for seamless migration. Rsync has the ability to push and pull data from one place to another. This can be accomplished by running an rsync daemon on the receiving server or using rsync over ssh like I tend to do.

For no reason other than consistency, I typically use rsync over ssh to push the data from the old server to the new server instead of pulling it over. Just a personal preference I guess. To each his own, but it works for me. So, further references will be based upon pushing data to the new server instead of pulling it.

Migrating the web site data is fairly easy and consists of a one liner on command line to push the site recursively to the new server while perserving permissions, owner, group, symlinks and timestamps on all directories and files. Become root and go to the parent directory of where your web site lives on the old server. Once you've done that, use the following to push the data to the new server.

rsync -progvult site_directory -e ssh new_server_name_or_ip:/full/path/to/new/site/dir/
(you must include a trailing slash at the end of your path)

The database data is very similar if not the same exact thing. On the database server (MySQL) is what my references related to, become root and go to /var/lib/mysql. This should be where all your databses reside. Now you can use the following command to push the database to the new database server.

rsync -progvult db_directory -e ssh new_server_name_or_ip:/var/lib/mysql/
(once again, trailing slashes are important)

These are the inital steps that I use for migration... At this point your firewall and new server software are configured. Plus the web site and database have had the initial push to the new server.

Next I will tackle the temprary DNS changes, testing of the new server site, final data push and the DNS changes to make it all live on the new server.

apache linux mysql sysadmin

You must be logged in to comment