Moving large number of files

user_1006249 0
Viewed 61026

If I run the command mv folder2/*.* folder, I get "argument list too long" error.

I find some example of ls and rm, dealing with this error, using find folder2 -name "*.*". But I have trouble applying them to mv.

bash
6 Answers

First, thanks to Karl's answer. I have only minor correction to this.

My scenario:

Millions of folders inside /source/directory, containing subfolders and files inside. Goal is to copy it keeping the same directory structure.

To do that I use such command:

find /source/directory -mindepth 1 -maxdepth 1 -name '*' -exec mv {} /target/directory \;

Here:

  • -mindepth 1 : makes sure you don't move root folder
  • -maxdepth 1 : makes sure you search only for first level children. So all it's content is going to be moved too, but you don't need to search for it.

Commands suggested in answers above made result directory structure flat - and it was not what I looked for, so decided to share my approach.

user_9635790

This one-liner command should work for you. Yes, it is quite slow, but works even with millions of files.

for i in /folder1/*; do mv "$i" /folder2; done

It will move all the files from folder /folder1 to /folder2.

user_21404080

find doesn't work with really long lists of files, it will give you the same error "Argument list too long". Using a combination of ls, grep and xargs worked for me:

$ ls|grep RadF|xargs mv -t ../fd/

It did the trick moving about 50,000 files where mv and find alone failed.

user_23090000
Related