How to filter files when using scp to copy dir recursively?

Viewed 140671

I need to copy all the .class files from server to local with all dir reserved. e.g. server:/usr/some/unknown/number/of/sub/folders/me.class will be /usr/project/backup/some/unknown/number/of/sub/folders/me.class the problem is, there are many other useless files such as .svn-base files that i don't want. how can i filter them so I only scp .class files?

9 Answers

I'd probably recommend using something like rsync for this due to its include and exclude flags, e.g:-

rsync -rav -e ssh --include '*/' --include='*.class' --exclude='*' \
server:/usr/some/unknown/number/of/sub/folders/ \ 
/usr/project/backup/some/unknown/number/of/sub/folders/

Some other useful flags:

  • -r for recursive
  • -a for archive (mostly all files)
  • -v for verbose output
  • -e to specify ssh instead of the default (which should be ssh, actually)

There is no feature in scp to filter files. For "advanced" stuff like this, I recommend using rsync:

rsync -av --exclude '*.svn' user@server:/my/dir .

(this line copy rsync from distant folder to current one)

Recent versions of rsync tunnel over an ssh connection automatically by default.

Since you can scp you should be ok to ssh,
either script the following or login and execute...

# After reaching the server of interest
cd /usr/some/unknown/number/of/sub/folders
tar cfj pack.tar.bz2 $(find . -type f -name *.class)

return back (logout) to local server and scp,

# from the local machine
cd /usr/project/backup/some/unknown/number/of/sub/folders
scp you@server:/usr/some/unknown/number/of/sub/folders/pack.tar.bz2 .
tar xfj pack.tar.bz2

If you find the $(find ...) is too long for your tar change to,

find . -type f -name *.class | xargs tar cfj pack.tar.bz2

Finally, since you are keeping it in /usr/project/backup/,
why bother extraction? Just keep the tar.bz2, with maybe a date+time stamp.

scp -i /home/<user>/.ssh/id_rsa -o "StrictHostKeyChecking=no" -rp /source/directory/path/[!.]* <target_user>@<target_system:/destination/directory/path
  1. Copy your source folder to somedir:

    cp -r srcdir somedir

  2. Remove all unneeded files:

    find somedir -name '.svn' -exec rm -rf {} \+

  3. launch scp from somedir

Related