My current MythTV machine has been expanded as far as it will go, so its time to upgrade. I've built a more powerful server and I want to transition everything to it. I need to move all my data at once to the new server because I don't have enough space to run two servers permanently. And I have 1TB of data, so efficient transport will be an important factor.
Initially, I attempted to move the TV recordings with a graphical SFTP client. But after several minutes, I realized my transfers were maxing out at about 2.2MBps. That's less than 20% of the capacity of my 100MBps network. I don't have anything else running, so what's limiting my download speed? I decided to investigate. Here are the results of my tests:
|Transfer Method||Example Command||Speed|
|SFTP with hostname||2.2MBps|
|SCP with hostname||2.2MBps|
|SFTP with IP address||11.2MBps|
|SCP with IP address|| ||11.1MBps|
|SCP with IP & blowfish|| ||11.1MBps|
|SCP with IP & compression|| ||4.4MBps|
I read somewhere that using a less CPU-intensive encryption algorithm like
blowfish would make my SCP transfers faster, but it didn't help. And turning on SSH compression (the
-C) was reported to improve download speeds on slow connections. I tried it on my fast network and it actually made things worse.
So here's what I learned: using a hostname rather than an IP address made a big performance difference. It probably has something to do with my router configuration. I'll ignore that for now…the point is that I got an 11.1MBps transfer rate, which is 91% of the theoretical max of 12.5MBps. The difference is due to protocol overhead.
My MythTV backend is constantly recording new shows. Hence, the set of files I copied to my new server become stale within hours. I need a way to synchronize my new machine with the existing one until I can complete my conversion. Enter
cd /my/video/folder rsync -avz email@example.com:/source/folder/ .
I'm forced to enter my password with this command because I specified my user ID, but at least this got my files transferred.