Find solution
How To Mirror Files?

What's the best means to mirror a webserver directory on a local OS X machine?

  • I maintain a complex website that I update by hand. (Unfortunately, this is unavoidable.) I want my local directory to be an exact mirror of the site, so that when I alter and modify thirty HTML files and I can simply run a script to copy them instead of FTPing them all by hand, in addition to removing deleted files and downloading any server-side changes made by scripts or other users. What's the best means to do this on OS X? I've tried using Transmit's "Mirror" feature, but in my experience its datestamp-calculations are unreliable, and also have the sideffect that anytime I need to reupload or redownload the entire 2 GB site from scratch (if I believe my backup is bad, or if I change servers, for example), I need to synchronize the entire site in the other direction for the timestamps to work out. I've tried to figure out RsyncX, but I'm not sure it's designed to do what I want, and it's a pretty hardcore piece of software. If it WILL do what I want, I'm all for learning how to make it do it, but I haven't found a good tutorial to that effect yet. Any suggestions? I'm sure that there are a million people who need and have systems like this, but for the life of me I can't figure it out myself. Also, any pointers to a good "newbie's guide to Rsync/RsyncX" would be appreciated, since I'd like to start running incremental archived backups as well.

  • Answer:

    rsync is a very good tool for this. http://www.novell.com/coolsolutions/feature/14520.html is helpful for discussing the basics. And of course you will need the man page. 1: Check to make certain you have rsync installed on the remote server. 2: Download your files with something like: rsync --recursive --verbose --archive user@remotehost:www . this will copy all of the files from the directory www on the remote server to your local directory. 3: When you are ready to upload, use: rsync --recursive --verbose --archive ./www user@remotehost: this will copy changed files to the remote server. The --archive flag tells rsync to preserve just about everything (modification times, permissions, etc.) You can use --dry-run to get a preview of what rsync will do.

tweebiscuit at Ask.Metafilter.Com Visit the source

Was this solution helpful to you?

Other answers

Don't use sitecopy. It works well for this in general but will kill server-side changes. Just a warning.

smackfu

furtive

Yes, rsync is the right tool for this. Also, the best way to run rsync for incremental backups is with http://www.rsnapshot.org/.

nicwolff

Oh, you also might want to check the -E, --extended-attributes flag on Tiger as well for Macintosh file transfers. It handles resource forks and metadata.

KirkJobSluder

Rsync is definitely what you want. It was designed for exactly what you're trying to do. The key to using it has already been mentioned: always use --recursive and --archive. The latter is actually a shortcut that turns on a bunch of other switches for preserving date/time/owner/permissions/links. Another good one to always have is -P (shortcut for --partial --progress). This will mean that if you are transferring a large file and the process is interrupted, when it it restarted it will pick up where it left off, rather than having to start at the beginning of that file. To handle file deletions use --delete which will cause any files on the dest that don't exist on the source to be deleted. If you want this to happen in both directions use it on both invocations of the command. Note that you don't have to setup an "rsync server", so ignore any tutorials on that. You can just do rsync over ssh, so as long as you can ssh to the remote, and the remote has the rsync command available, that's all you need. http://samba.anu.edu.au/rsync/documentation.html If you just aren't getting anywhere with rsync you can try http://www.cis.upenn.edu/~bcpierce/unison/faq.html, which is similar.

Rhomboid

Take the time now to implement and learn CVS. For a large, complex web site (2GB!) you will feel a lot better and probably save yourself a lot of grief using a real version control system. This will also give you the huge benefit of letting others make changes to the site, if necessary. There'll be a bit to learn in the beginning (you can pick it up in a week easily) but once you do you're home free.

nixerman

Run an AFP (Apple File Protocol) file server, e.g. on Mac OS X Server. Then keep your web documents on a file share. Connect your workstations to the file share. Set up your Dreamweaver profiles to use the AFP file share as a "Local" directory, which for your purposes is exactly what this is.

Rothko

Thanks guys! I don't think I'll use CVS (most of the 2GB are video files -- there aren't nearly that many HTML pages, and no other editors). And thanks, Rothko, but I don't use Dreamweaver. Now that I know it's the tool for the job, I'll start plugging through with Rsync. Thanks everybody!

tweebiscuit

I actually think Dreamweaver is quite good at this task. Some versions of it tend to crash on OS X but, that little problem aside, its "Synchronize" feature is exactly what you want. You keep a local mirror and you update what needs to be updated. If you have DW already, I'd suggest getting to know this feature, but of course I don't think you buy it just for its synching. Also, if you find Transmit's datestamp functions are unreliable, I'm sure they'd love to hear about it, they're a very responsive company in that way.

AmbroseChapel

Load more

Related Q & A:

Just Added Q & A:

Find solution

For every problem there is a solution! Proved by Solucija.

  • Got an issue and looking for advice?

  • Ask Solucija to search every corner of the Web for help.

  • Get workable solutions and helpful tips in a moment.

Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.