What does Jenkins use to capture stdout and stderr of a shell command?

user_8593689 0
Viewed 38

In Jenkins, you can use the sh step to run Unix shell scripts.

I was experimenting, and I found that the stdout is not a tty, at least on a Docker image.

What does Jenkins use for capturing stdout and stderr of programs running via the sh step? Is the same thing used for running the sh step on a Jenkins node versus on a Docker container?

I ask for my own edification and for some possible practical applications of this knowledge.

To reproduce my experimentation

If you already know an answer, you don't need to read these details for reproducing. I am just adding this here for reproducibility.

I have the following Jenkins/Groovy code:

docker.image('gcc').inside {
    sh '''
        gcc -O2 -Wall -Wextra -Wunused -Wpedantic \
            -Werror write_to_tty.c -o write_to_tty

        ./write_to_tty
    '''
}

The Jenkins log snippet for the sh step code above is

+ gcc -O2 -Wall -Wextra -Wunused -Wpedantic -Werror write_to_tty.c -o write_to_tty
+ ./write_to_tty
stdout is not a tty.

This compiles and runs the following C code:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int main() {
    int stdout_fd = fileno(stdout);

    if (!isatty(stdout_fd)) {
        fprintf(stderr, "stdout is not a tty.\n");
        exit(1);
    }

    char* stdout_tty_name = ttyname(stdout_fd);
    if (stdout_tty_name == NULL) {
        fprintf(stderr, "Failed to get tty name of stdout.\n");
        exit(1);
    }

    FILE* tty = fopen(stdout_tty_name, "w");
    if (tty == NULL) {
        fprintf(stderr, "Failed to open tty %s.\n", stdout_tty_name);
        exit(1);
    }

    fprintf(tty, "Written directly to tty.\n");
    fclose(tty);

    printf("Written to stdout.\n");
    fprintf(stderr, "Written to stderr.\n");

    exit(0);
}
docker tty jenkins unix
2 Answers

I briefly looked at the source here and it seems stdout is written to a file and then read from that, Hence it's not a tty. Also, stderror, if any, will be written to the log.

Here is the Javadoc.

/** * Runs a durable task, such as a shell script, typically on an agent. *

“Durable” in this context means that Jenkins makes an attempt to keep the external process running * even if either the Jenkins controller or an agent JVM is restarted. * Process standard output is directed to a file near the workspace, rather than holding a file handle open. * Whenever a Remoting connection between the two can be reëstablished, * Jenkins again looks for any output sent since the last time it checked. * When the process exits, the status code is also written to a file and ultimately results in the step passing or failing. *

Tasks can also be run on the built-in node, which differs only in that there is no possibility of a network failure. */

user_26270180

I found the source for the sh step, which appears to be implemented using the BourneShellScript class.

When not capturing stdout, the command is generated like this:

cmdString = String.format("{ while [ -d '%s' -a \\! -f '%s' ]; do touch '%s'; sleep 3; done } & jsc=%s; %s=$jsc %s '%s' > '%s' 2>&1; echo $? > '%s.tmp'; mv '%s.tmp' '%s'; wait",
                          controlDir,
                          resultFile,
                          logFile,
                          cookieValue,
                          cookieVariable,
                          interpreter,
                          scriptPath,
                          logFile,
                          resultFile, resultFile, resultFile);

If I correctly matched the format specifiers to the variables, then the %s '%s' > '%s' 2>&1 part roughly corresponds to

${interpreter} ${scriptPath} > ${logFile} 2>&1

So it seems that the stdout and stderr of the script is written to a file.

When capturing output, it is slightly different, and is instead

${interpreter} ${scriptPath} > ${c.getOutputFile(ws)} 2> ${logFile}

In this case, stdout and stderr are still written to files, just not to the same file.

Aside

For anyone interested in how I found the source code:

  1. First I used my bookmark for the sh step docs, which can be found with a search engine
  2. I scrolled to the top and clicked on the "View this plugin on the Plugins site" link
  3. On that page, I clicked the GitHub link
  4. On the GitHub repo, I navigated to the ShellStep.java by drilling down from the src directory
  5. I saw that the class was implemented using the BourneShellScript class, and based on the import for this class, I knew it was part of the durable task plugin
  6. I searched the durable task plugin, and followed the same "View this plugin on the Plugins site" link and then GitHub link to view the GitHub repo for the durable task plugin
  7. Next, I used the "Go to file" button and searched for the class name to jump to its .java file
  8. Finally, I inspected this file to find what I was interested in
user_85936890
Related