get cloudfront usage report via aws cli

user_6106791 0
Viewed 577

I have a bunch of Cloudfront distributions scattered across a number of AWS accounts. I'd like to get the Usage Reports for all Cloudfront distros across all AWS accounts.

Now, I have the change-account bit already automated, but I'm not sure how to get the CSV report via the AWS CLI.

I know I can do some ClickOps and download the report via the Cloudfront Console, like here:

enter image description here

but I can't find the command to get the report with the AWS CLI.

I know I can get the Cloudfront metrics via the Cloudwatch API but the documentation doesn't mention the API endpoint I should be querying.

Also, there's aws cloudwatch get-metric-statistics, but I'm not sure how to use that to download the Cloudfront Usage CSV Report.

Question: How can I get the Cloudfront Usage Report for all distributions in an AWS account using the AWS CLI?

amazon-cloudwatch amazon-cloudfront aws-cli amazon-web-services
2 Answers
user_6497670

I can't find a Cloudfront API to fetch the Usage Report. I know such report can be constructed from Cloudwatch logs, but I'm lazy and I'd like to download the report directly from Cloudfront.

There is no such command in AWS CLI or function in Boto3 (AWS SDK for Python) introduced yet but there are a couple of workarounds that you can use which are as follows:

  1. Use Selenium to access AWS Console for CloudFront and click on that Download CSV button. You can write a script for that in Python.

  2. You can use the curl command used by CloudFront on AWS Console to fetch the results in XML format and then you can convert them into CSV using Python or any CLI tool. That curl command can be found after clicking on the Download CSV button and then from the item named cloudfrontreporting which appears under the Network tab under Inspect console on the page in Google Chrome browser (or using any other browser of your choice), right-click on that item and then click on Copy as cURL button.

The curl command is as follows:

curl 'https://console.aws.amazon.com/cloudfront/v3/api/cloudfrontreporting' \
  -H 'authority: console.aws.amazon.com' \
  -H 'sec-ch-ua: " Not;A Brand";v="99", "Google Chrome";v="97", "Chromium";v="97"' \
  -H 'content-type: application/json' \
  -H 'x-csrf-token: ${CSRF_TOKEN}' \
  -H 'accept: */*' \
  -H 'origin: https://console.aws.amazon.com' \
  -H 'sec-fetch-site: same-origin' \
  -H 'sec-fetch-mode: cors' \
  -H 'sec-fetch-dest: empty' \
  -H 'referer: https://console.aws.amazon.com/cloudfront/v3/home?region=eu-central-1' \
  -H 'accept-language: en-US,en;q=0.9' \
  -H 'cookie: ${COOKIE}' \
  --data-raw '{"headers":{"X-Amz-User-Agent":"aws-sdk-js/2.849.0 promise"},"path":"/2014-01-01/reports/series","method":"POST","region":"us-east-1","params":{},"contentString":"<DataPointSeriesRequestFilters xmlns=\"http://cloudfront.amazonaws.com/doc/2014-01-01/\"><Report>Usage</Report><StartTime>2022-01-28T11:23:35Z</StartTime><EndTime>2022-02-04T11:23:35Z</EndTime><TimeBucketSizeMinutes>ONE_DAY</TimeBucketSizeMinutes><ResourceId>All Web Distributions (excludes deleted)</ResourceId><Region>ALL</Region><Series><DataKey><Name>HTTP</Name><Description></Description></DataKey><DataKey><Name>HTTPS</Name><Description></Description></DataKey><DataKey><Name>HTTP-BYTES</Name><Description></Description></DataKey><DataKey><Name>HTTPS-BYTES</Name><Description></Description></DataKey><DataKey><Name>BYTES-OUT</Name><Description></Description></DataKey><DataKey><Name>BYTES-IN</Name><Description></Description></DataKey><DataKey><Name>FLE</Name><Description></Description></DataKey></Series></DataPointSeriesRequestFilters>","operation":"listDataPointSeries"}' \
  --compressed > report.xml

where ${CSRF_TOKEN} and ${COOKIE} needs to be provided by yourself which can be found from the browser or can be prepared programmatically.

  1. Use logs generated by CloudFront as mentioned here in the answer and code in the question: Boto3 CloudFront Object Usage Count
user_117588430
Related