The impersonate_service_account for oauth BigQuery method not working

[ivan-toriya]

Describe the bug
The impersonate_service_account for oauth BigQuery method not working.

# profiles.yml

my-profile:
  target: dev
  outputs:
    dev:
      type: bigquery
      method: oauth
      project: abc-123
      dataset: my_dataset
      impersonate_service_account: dbt-runner@yourproject.iam.gserviceaccount.com

I'm logged into account_a via gcloud auth login --update-adc which has Service Account Token Creator permission for dbt-runner@yourproject.iam.gserviceaccount.com. And it's working fine in day-to-day dbt usage. But when

$ data-diff --dbt
21:36:03 DEBUG    Exception when trying to execute SQL code:                                                                                            dbt.py:265
                      SELECT column_name, data_type, 6 as datetime_precision, 38 as numeric_precision, 9 as numeric_scale FROM                                    
                  ``.``.INFORMATION_SCHEMA.COLUMNS WHERE table_name = '' AND                
                  table_schema = ''                                                                                                         
                                                                                                                                                                  
                  Got error: 403 POST https://bigquery.googleapis.com/bigquery/v2/projects/my-project/jobs?prettyPrint=false: Access Denied:                  
                  Project my-project: User does not have bigquery.jobs.create permission in project my-project.                                           
                                                                                                                                                                  
                  Location: None                                                                                                                                  
                  Job ID: 42

New model or no access to prod table.

Describe the environment

WSL Ubuntu 20.04.6 LTS
python version: 3.10.11
dbt version: 1.3.2
data-diff=0.8.4

[ivan-toriya]

And forgive me for creating this as a bug, not a feature request. My logic was, authentication should work as previously expected.

[ivan-toriya]

btw, VSCode extension just runs the model indefinitely without returning the error.

[charles-astrafy]

We are running into the same issue here. We expect data-diff to leverage the "impersonate_service_account" of profiles.yml

[dlawin]

@charles-astrafy, @ivan-toriya, @nhnam1105

This is resolved in v0.9.6, thanks again for the contribution @ivan-toriya