英文:
How do I update Big Query partition expiration using Python BigQuery Client?
问题
我们目前正在寻求清理旧的Big Query数据。在创建数据集、表和分区时，我们直到最近才更新了到期时间。因此，旧的分区没有设置到期时间，只有新创建的表和分区有到期时间。
根据上述文档，它提到可以使用SQL、bq命令或API来更新分区到期时间。我想能够自动化整个过程，而不必手动运行每个数据集的命令，而且我只能从服务器上运行脚本，因为我们的访问受限。
是否有办法使用Python脚本和Python BQ客户端自动化这个过程？
英文:
We are currently looking to clean up old Big Query data. While creating the dataset, tables and partitions, we did not update the expiration time until recently. So the old partitions do not have expiration time set. Only the newly created tables and partitions have expiration.
https://cloud.google.com/bigquery/docs/managing-partitioned-tables#bq
According to the above doc, it says to update partition expiry using SQL or bq commands or API. I want to be able to automate the whole process without manually running the commands for each of the dataset and I'll be able to run the script only from the server since our access is restricted.
Is there a way to automate the process using python script and Python BQ client?
答案1
得分: 1
以下是打乱原内容后的翻译：
来自 google-cloud-bigquery 的更新表分区过期示例：
from google.cloud import bigquery
table_id = 'project_id.dataset_id.table_name'
partition_expiration_ms = 30 * 24 * 60 * 60 * 1000 # 以毫秒为单位的30天
client = bigquery.Client()
table_ref = client.get_table(table_id)
table_ref.time_partitioning.expiration_ms = partition_expiration_ms
client.update_table(table_ref, ["time_partitioning"])
英文:
Here a example of update a table partition expiration using google-cloud-bigquery:
from google.cloud import bigquery
table_id = 'project_id.dataset_id.table_name'
partition_expiration_ms = 30 * 24 * 60 * 60 * 1000 # 30 days in milliseconds
client = bigquery.Client()
table_ref = client.get_table(table_id)
table_ref.time_partitioning.expiration_ms = partition_expiration_ms
client.update_table(table_ref, ["time_partitioning"])
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论