GCP / GKE export cost to bigquery per project

user19949990 0
Viewed 56

We have different GKE clusters in different GCP Projects, and we'd like to get detailed costs per labels. We have a billing account that spans all the GCP projects.

From the doc I understand both cloud billing and GKE cost allocations have to be configured to export to bigquery for this. No issues with configuring cloud billing exports from the billing account to bigquery, but GKE cost allocation can only be exported to a dataset in the same project.

How can I reconcile that, having one billing account for all projects but needing to export it to a dataset in each project ? Do I need to setup multiple sinks with filters, and if so what would those filters look like ?

Or can I just export cloud billing to one dataset in one project, then GKE to each cluster's project in a different dataset, and "merge" all of this later on in the dashboard ?

Thanks

terraform-provider-gcp google-kubernetes-engine google-cloud-platform
2 Answers

You can configure the cloud billing export to a central dataset in a project. Make sure you enable Detailed usage cost export since the latest GKE cost allocation labels are exported here.

Once the detailed usage cost is setup, GKE cost allocation needs to be enabled at a per cluster level. The GKE labels of enabled clusters would start to appear in the detailed usage cost export within 48 hours. This feature was released in June, 2022 and you no longer need to maintain separate cluster datasets across projects.

I would recommend setting up a cloud cost management platform for GKE cost analysis out of the box. You can check out Economize - connects with your GCP billing dataset and you can create custom GKE cost views.

Amrish Kumar0

There are two ways to export GKE cost data to BigQuery. The original functionality was GKE Usage metering and I believe this worked on a per project basis in terms of the export to BigQuery.

There is also the new GKE cost allocation functionality which is natively integrated with Cloud Billing. It does support filtering on labels, so should work for you. Using this, you should be able to view the cost info for multiple projects under the same billing account. You should create a separate project for cost/billing exports (linked to the billing account you want) and the export here should include all billing info across all projects and will actually include GKE info as well.

Gari Singh0
Related