aws cli - download rds postgres logs in a bash script

user_3867776 0
Viewed 1651

I wrote a simple bash script to download my RDS postgres files. But the kicker is that is all works fine tine terminal, but when I try the same thing in the script I get an error:

An error occurred (DBLogFileNotFoundFault) when calling the DownloadDBLogFilePortion operation: DBLog File: "error/postgresql.log.2017-11-05-23", is not found on the DB instance

The command in question is this:

aws rds download-db-log-file-portion --db-instance-identifier foobar --starting-token 0 --output text --log-file error/postgresql.log.2017-11-05-23 >> test.log

It all works fine, but when I put the exact same line in the bash script I get the error message that there are no db log files - which is nonsense, they are there.

This is the bash script:

download_generate_report() {

for filename in $( aws rds describe-db-log-files --db-instance-identifier $1 | awk {'print $2'} | grep $2 )
do

echo $filename
echo $1

aws rds download-db-log-file-portion --db-instance-identifier $1 --starting-token 0 --output text --log-file $filename  >> /home/ubuntu/pgbadger_script/postgres_logs/postgres_$1.log.$2
done

}

Tnx, Tom

rds aws-cli bash
2 Answers

I re-wrote your script a little and it seems to work for me. It barked about grep. This uses jq.

for filename in $( aws rds describe-db-log-files --db-instance-identifier $1 | jq -r '.DescribeDBLogFiles[] | .LogFileName' )
do

aws rds download-db-log-file-portion --db-instance-identifier $1 --output text --no-paginate --log-file $filename  >> /tmp/postgres_$1.log.$2
done
user_104427060

Thanks you Ian, I have an issue, with aws cli 2.4, because the log files download truncated.

to solve this I changed --no-paginate with --starting-token 0 more info in the RDS Reference

finally in bash:

#/bin/bash
set -x
for filename in $( aws rds describe-db-log-files --db-instance-identifier $1 | jq -r '.DescribeDBLogFiles[] | .LogFileName' ) 
do 
    aws rds download-db-log-file-portion --db-instance-identifier $1 --output text --starting-token 0  --log-file $filename  >> $filename 
done
user_27087590
Related