How to save a view using federated queries across two projects?

user_684908 0
Viewed 548

I'm looking to save a view which uses federated queries (from a MySQL Cloud SQL connection) between two projects. I'm receiving two different errors (depending on which project I try to save in).

If I try to save in the project containing the dataset I get error:

Not found: Connection my-connection-name

If I try to save in the project that contains the connection I get error:

Not found: Dataset my-project:my_dataset

My example query that crosses projects looks like:

SELECT
  bq.uuid,
  sql.item_id,
  sql.title
FROM
  `project_1.my_dataset.psa_v2_202005` AS bq
LEFT OUTER JOIN
  EXTERNAL_QUERY( 'project_2.us-east1.my-connection-name',
    '''SELECT item_id, title
  FROM items''') AS sql
ON
  bq.looks_info.query_item.item_id = sql.item_id

The documentation at https://cloud.google.com/bigquery/docs/cloud-sql-federated-queries#known_issues_and_limitations doesn't mention any limitations here.

Is there a way around this so I can save a view using an external connection from one project and dataset from another?

google-bigquery
1 Answers

Your BigQuery table is located in US and your MySQL data source is located in us-east1. BigQuery automatically chooses to run the query in the location of your BigQuery table (i.e. in US), however, your Cloud MySQL is in us-east1 and that's why your query fails. Therefore the BigQuery table and Cloud SQL instance, must be in the same location in order for this query to succeed.

The solution for this kind of cases is moving your BigQuery dataset to the same location as your Cloud SQL instance manually by following the steps explained in detail in this documentation. However, the us-east1 is not currently supported for copying datasets. Thus, I will recommend you to create a new connection in one of the locations mentioned in the documentation.

I hope you find the above pieces of information useful.

user_70900160
Related