Given a file with long lines, parallel fails to pass these lines as an argument to any command:
$> cat johny_long_lines.txt | parallel echo {}
parallel: Error: Command line too long (68906 >= 65524) at input 0: 2236439425|\x308286873082856fa003020102020c221ff03...
This gets more confusing when I see that the line is 68900 characters long:
$> cat johny_long_lines.txt | head -n 1 | wc -m
68900
while the max line length allowed by parallel is way more longer that my input:
$> parallel --max-line-length-allowed
131049
Also: if you think that it's a problem of execve, this might interest you:
$> getconf ARG_MAX
2097152
Any idea what I'm doing here wrong?
UPDATE
I figured out that the problem persists for versions 20161222 and 20220522 but not for 20210822 (delivered with Ubuntu 22.04 LTS). Further inspection reveals that this line causes the problem:
# Usable len = maxlen - 3000 for wrapping, div 2 for hexing
int(($Global::minimal_command_line_length - 3000)/2);
Which I can confirm using --show-limits:
$> parallel --show-limits
[...]
Maximal size of command: 131063
Maximal usable size of command: 64031
This annoying feature does not exist in version 20210822 and I my file goes through as expected.
Can this be disabled?