[BNM] Transferring 60GB of files from one server to another

[David Pashley]

On Dec 02, 2008 at 14:58, Barney Livingston praised the llamas by saying:
> On Tuesday 02 December 2008 14:38:49 James Moss wrote:
> > Im working on a side project and we've just had a fast shiny new linux
> > server installed for us to replace the old one.  However we've got 60GB of
> > files in the wwwroot folder which need to be transferred to the new server
> > over the gigabit network.  Whats the fastest way of doing it? Is SSHing in
> > and doing an FTP transfer fast enough or is there a better way?
> >
> > Weve also got a fairly large DB (its just under 5GB, 3.5GB of which is a
> > single table).  Whats the best way of getting that across too?  I was
> > planning on dumping the current one to an .sql file and executing it on the
> > new server but i presume the inserts will take aaaaages.
> 
> Netcat and tar is probably quickest:
> 
> On the receiving end do:
> 
> netcat -l -p 7000 | tar x
> 
> And on the sending end do:
> 
> tar cf - * | netcat otherhost 7000

Definitely orders of magnitude faster. Avoid scp or rsync for one off
copies, although you may find rsync quicker to recover failed partial
copies, depending on how far you get.
> 
> from http://compsoc.dur.ac.uk/~djw/tarpipe.html
> 
> No security of course.
> 
> I think dump and reimport is the best way to do the db and yes it will 
> probably take ages. If it's Mysql and they're identical versions at both ends 
> you might be able to get away with copying the data directories. You might 
> even get to keep all the broken halves.
> 
With regards to copying the data files, make sure that you don't copy
from i386 to amd64, because that won't work. 

If you want to minimise downtime, set your current server up as a master
server, lock your database, dump your existing database, make note of
the current replication position, unlock your database. Import into your
new database at your leisure, set the slave up using the details you
recorded from the master, let replication catch up. Make both databases
read-only, reconfigure all your clients to talk to the slave, turn off
master, unlock new database. Downtime during recnfiguring master (should
just be restart), dumping database (this'll be the long part) and
reconfiguring clients (depends on the number of clients).

> Barney

-- 
David Pashley
david at davidpashley.com
Nihil curo de ista tua stulta superstitione.