How to redirect stdout to a file in Cloud Run?

brunoob 0
Viewed 57

I have a python program that runs a script and this script writes its output to stdout, which I need to use later in my program. Since this script is in binary and I can't change it's source code, so I redirected the output of my script to a file, then I read the file as below :

os.popen("./my_script > test.txt")

with open("test.txt", "r") as f:
    output = " ".join(f.readlines())

print(f"I can process my output : {output} here")

The program works great, until I try to deploy it to Cloud Run. The output is not redirected to a file, but automatically redirected to Cloud Logging, and I can no longer use it in my program.

google-cloud-logging stdout google-cloud-run python
2 Answers

As mentioned in the comments by @John Hanley On Cloud Run, stdout and stderr are handled differently than Docker.

You do not show what your script does. A script that is binary is not a script.

Make sure you know what that program does. Some programs directly write to stderr instead of stdout. You should only be writing to /tmp e.g. os.popen("./my_script > /tmp/test.txt 2>&1").

I recommend that you control stdout and stdout programmatically and not by redirection to a file.

Osvaldo0

Under the hood CloudRun runs your application in container. All prints to stdout and stderr in containers are forwarded to Cloud Logging by a logging agent that CloudRun runs implicitly. But it is not like it is "redirected" or "captured". The agent reads the output from the file where CRI stores it. (Everything is a file in Linux).

It is my understanding that os.popen() opens a process to execute the command. This is generally considered anti-pattern when running application in a container.

Given the os.popen() eventually calls to subprocess.Popen() you can rewrite your code to use subprocess.Popen() directly and substitute your own streams for stdout and stderr

Leo Y0
Related