Concatenating multiple text files into a single file in Bash

Viewed 384946

What is the quickest and most pragmatic way to combine all *.txt file in a directory into one large text file?

Currently I'm using windows with cygwin so I have access to BASH.

Windows shell command would be nice too but I doubt there is one.

12 Answers

This appends the output to all.txt

cat *.txt >> all.txt

This overwrites all.txt

cat *.txt > all.txt

Just remember, for all the solutions given so far, the shell decides the order in which the files are concatenated. For Bash, IIRC, that's alphabetical order. If the order is important, you should either name the files appropriately (01file.txt, 02file.txt, etc...) or specify each file in the order you want it concatenated.

$ cat file1 file2 file3 file4 file5 file6 > out.txt

You can use Windows shell copy to concatenate files.

C:\> copy *.txt outputfile

From the help:

To append files, specify a single file for destination, but multiple files for source (using wildcards or file1+file2+file3 format).

The Windows shell command type can do this:

type *.txt > outputfile.txt

Type type command also writes file names to stderr, which are not captured by the > redirect operator (but will show up on the console).

the most pragmatic way with the shell is the cat command. other ways include,

awk '1' *.txt > all.txt
perl -ne 'print;' *.txt > all.txt

You can do like this: cat [directory_path]/**/*.[h,m] > test.txt

if you use {} to include the extension of the files you want to find, there is a sequencing problem.

The most upvoted answers will fail if the file list is too long.

A more portable solution would be using fd

fd -e txt -d 1 -X awk 1 > combined.txt

-d 1 limits the search to the current directory. If you omit this option then it will recursively find all .txt files from the current directory.
-X (otherwise known as --exec-batch) executes a command (awk 1 in this case) for all the search results at once.

Note, fd is not a "standard" Unix program, so you will likely need to install it

When you run into a problem where it cats all.txt into all.txt, You can try check all.txt is existing or not, if exists, remove

Like this:

[ -e $"all.txt" ] && rm $"all.txt"

all of that is nasty....

ls | grep *.txt | while read file; do cat $file >> ./output.txt; done;

easy stuff.

Related