允许业务用户将数据从CSV上传到Google Big Query表。

huangapple go评论65阅读模式
英文:

Allow a business user to upload data from csv to a Google Big Query table

问题

可以让业务用户在不需要进行过多技术操作的情况下,将CSV文件上传到BigQuery表中吗?

英文:

I am preparing a tableau dashboard that reads data from certain tables. One of the tables has data that is manually added on an ad-hoc basis. The data comes from the business side. Is there a way to allow the business user to upload the csv to the big query table without having them to do anything too technical?

答案1

得分: 2

1 使用云存储传输:

在这种方法中,首先您需要将一个 CSV 文件提交到云存储,然后您可以设置云存储传输。云存储传输服务允许您安排定期从云存储到 BigQuery 的数据加载。有关更多信息,您可以访问此链接。但请注意,该服务存在一些限制

2 在 Google Cloud 上自动化 Python 脚本。

有不同的方法可以在 Google Cloud 上自动化 Python 脚本。在您的情况下,您可以考虑使用云函数和云计划程序来自动运行您的 Python 代码。首先,您需要创建一个云函数来运行您的代码,然后您可以使用云计划程序自动触发您的函数。更安全的方式是通过使用 Pub/Sub 间接触发云函数。有关更多信息,您可以访问此链接

英文:

As a workaround you can consider below methods:

1 Using Cloud Storage transfers:

In this method,first you need to submit a csv file to the cloud storage.Then you can set up a Cloud Storage transfer.This transfer service for cloud storage allows you to schedule recurring data loads from Cloud Storage to BigQuery.For more information you follow this link. But keep in mind that this service has some limitations.

2 Automate a Python script on Google Cloud.

There are different ways to automate a python script on google cloud. In your case,you can consider using a cloud function with cloud schedular to run your python code automatically.First you need to create a cloud function to run your code. Then you can use a cloud schedular to trigger your function automatically.In a more secure way you can use trigger cloud function indirectly by using pubsub. For more information you can follow this link.

答案2

得分: 1

"目前还没有一种用户友好的方式可以直接从CSV导入到现有的BigQuery表格。您可以将您的Python脚本打包成一个小应用程序,允许业务用户选择一个文件,然后运行您的Python脚本将CSV上传到BigQuery。我曾经做过类似的解决方案。"

英文:

There's not currently a user friendly way to import straight to an existing big query table from a csv. You could package your python script into a small application that allows the business user to select a file and then runs your python script to upload the csv to BQ.
I've had to do a similar solution.

huangapple
  • 本文由 发表于 2023年4月11日 11:18:43
  • 转载请务必保留本文链接:https://go.coder-hub.com/75982169.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定