英文:
Service Account unable to access Google Drive despite having the right access
问题
我在使用Composer帐户访问Google Drive文件时遇到了这个问题。相同的Composer可以在另一个环境（相同的GCP项目）中访问文件，该环境使用Composer1，但在使用Composer2的新环境中出现了错误 - 拒绝访问：BigQuery BigQuery：在获取Drive凭据时被拒绝访问。
我尝试提升服务帐户的权限，但没有帮助，因为帐户已经具有所需的权限。如果有人能指导如何排除故障或修复它，将不胜感激。谢谢！
更新 - 发现Composer2环境缺少以下Google API范围，但无法找到在Composer2中添加它的方法。
https://www.googleapis.com/auth/cloud-platform
https://www.googleapis.com/auth/spreadsheets
https://www.googleapis.com/auth/drive
英文:
I'm facing this issue with the Google Drive File access using the Composer account .
The same composer is able to access the file in another environment(same GCP project) using Composer1 but the new environment with composer2 gives this error - Access Denied: BigQuery BigQuery: Permission denied while getting Drive credentials.
I tried elevating permissions of the service account too but didn't help , as the account already had required permissions. Appreciate if someone can guide on how to troubleshoot or fix it. Thanks !
Update - Found the composer2 environment is missing below Google API scopes but unable to find out a way to add it in composer2.
https://www.googleapis.com/auth/cloud-platform
https://www.googleapis.com/auth/spreadsheets
https://www.googleapis.com/auth/drive
答案1
得分: 1
OAuth范围不受Composer2支持，因为它依赖于GKE Autopilot。在Composer 2中，一般访问权限应通过分配给环境服务帐户的权限来管理 链接 1，链接 2。
然而，在某些情况下，当您需要指定OAuth范围，比如在Google Sheets连接中，需要直接在方法调用中执行。根据Drive访问文档，您可以使用以下方式在BigQuery客户端的Python API中设置这个范围：
from google.cloud import bigquery
import google.auth
# 创建具有Drive和BigQuery API范围的凭据。
# 在运行此代码之前，必须为项目启用这两个API。
credentials, project = google.auth.default(
scopes=[
"https://www.googleapis.com/auth/drive",
"https://www.googleapis.com/auth/bigquery",
]
)
# 构建一个BigQuery客户端对象。
client = bigquery.Client(credentials=credentials, project=project)
另一个选择是在您的DAG中直接使用BigQuery API，而不是使用Airflow操作符，并从调用中启用Drive身份验证。
英文:
OAuth scopes are not supported by Composer2 as it relies on GKE Autopilot.
In Composer 2 access in general should be managed via permissions assigned to the environment service account link 1, link 2.
However, in some cases when you need to specify an OAuth scope like with this Google Sheets connection, this is required to be done directly in the method call. According to the Drive access documentation , you can set this with the Python API in the BigQuery client in the following way:
from google.cloud import bigquery
import google.auth
# Create credentials with Drive & BigQuery API scopes.
# Both APIs must be enabled for your project before running this code.
credentials, project = google.auth.default(
scopes=[
"https://www.googleapis.com/auth/drive",
"https://www.googleapis.com/auth/bigquery",
]
)
# Construct a BigQuery client object.
client = bigquery.Client(credentials=credentials, project=project)
An option you could also try would be to use the BigQuery API directly in your DAG instead of the Airflow operator and to enable drive authentication from the call.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论