How do I include a pipe | in my linux find -exec command?

the solution is easy: execute via sh

... -exec sh -c "zcat {} | agrep -dEOE 'grep' " \;

The job of interpreting the pipe symbol as an instruction to run multiple processes and pipe the output of one process into the input of another process is the responsibility of the shell (/bin/sh or equivalent).

In your example you can either choose to use your top level shell to perform the piping like so:

find -name 'file_*' -follow -type f -exec zcat {} \; | agrep -dEOE 'grep'

In terms of efficiency this results costs one invocation of find, numerous invocations of zcat, and one invocation of agrep.

This would result in only a single agrep process being spawned which would process all the output produced by numerous invocations of zcat.

If you for some reason would like to invoke agrep multiple times, you can do:

find . -name 'file_*' -follow -type f \
    -printf "zcat %p | agrep -dEOE 'grep'\n" | sh

This constructs a list of commands using pipes to execute, then sends these to a new shell to actually be executed. (Omitting the final "| sh" is a nice way to debug or perform dry runs of command lines like this.)

In terms of efficiency this results costs one invocation of find, one invocation of sh, numerous invocations of zcat and numerous invocations of agrep.

The most efficient solution in terms of number of command invocations is the suggestion from Paul Tomblin:

find . -name "file_*" -follow -type f -print0 | xargs -0 zcat | agrep -dEOE 'grep'

... which costs one invocation of find, one invocation of xargs, a few invocations of zcat and one invocation of agrep.

find . -name "file_*" -follow -type f -print0 | xargs -0 zcat | agrep -dEOE 'grep'

I found that running a string shell command (sh -c) works best, for example:

find -name 'file_*' -follow -type f -exec bash -c "zcat \"{}\" | agrep -dEOE 'grep'" \;

If you are looking for a simple alternative, this can be done using a loop:

for i in $(find -name 'file_*' -follow -type f); do
  zcat $i | agrep -dEOE 'grep'
done

or, more general and easy to understand form:

for i in $(YOUR_FIND_COMMAND); do
  YOUR_EXEC_COMMAND_AND_PIPES
done

and replace any {} by $i in YOUR_EXEC_COMMAND_AND_PIPES

Here's what you should do:

find -name 'file_*' -follow -type f -exec sh -c 'zcat "$1" | agrep -dEOE "grep"' sh {} \;

I tried a couple of these answers and they didn't work for me. @flolo's answer doesn't work correctly if your filenames have special characters. According to this answer:

The find command executes the command directly. The command, including the filename argument, will not be processed by the shell or anything else that might modify the filename. It's very safe.

You lose that safety if you put the {} inside the sh command string.

There is a potential problem with @Rolf W. Rasmussen's answer. Yes, it handles special characters (as far as I know), but if the find output is too long, you won't be able to execute xargs -0 ...: there is a command line character limit set by the kernel and sometimes your shell. Coincidentally, every time I want to pipe commands from a find, I run into this limit.

But, they do bring up a valid point regarding the performance limitations. I'm not sure how to overcome that, though personally, I've never run into a situation where my suggestion is too slow.