如何使用Go脚本从Google Cloud Storage获取我的图像(base64)?

huangapple go评论89阅读模式
英文:

How to get my image (base64) in Google-Cloud-Storage with go script

问题

我一直在寻找一个示例GAE脚本(使用go编写),以获取我从PageSpeed Insights的结果截图中获得的图像,并将其保存为json_decode对象,使用Kohana/Cache将其保存到*Google Cloud Storage (GCS)*中。

之所以选择这种方法,是因为我发现这个Kohana模型是将文件写入GCS的最方便的方式,尽管我也在寻找其他方式,比如这个使用Blobstore将文件写入GCS,并使用它们进行服务,而Go API Files已经被弃用,文档在这里有说明。

以下是存储的对象的形式,其中包含截图图像数据(base64),它以默认应用程序存储桶中的公共方式保存,并具有对象名称images/thumb/mythumb.jpg

stdClass Object
(
    [screenshot] => stdClass Object
        (
            [data] => _9j_4AAQSkZJRgABAQAAAQABAAD_...= // base64数据
            [height] => 240
            [mime_type] => image/jpeg
            [width] => 320
        )

    [otherdata] => Array
        (
            [..] => ..
            [..] => ..
        )

)

我想通过以下自定义URL获取此图像,并通过go模块进行处理,同时我需要在一定时间内使其过期,因为我已经成功地定期更新了图像内容:

> http://myappId.appspot.com/image/thumb/mythumb.jpg

我在disptach.yaml中设置了将所有图像请求发送到我的go模块,如下所示:

- url: "*/images/*"
  module: go

并在go.yaml中设置了处理程序来处理图像请求,如下所示:

handlers:
- url: /images/thumb/.*
  script: _go_app

- url: /images
  static_dir: images

使用这个指令,我发现所有的*/images/请求(除了/images/thumb/请求)都从静态目录中提供图像,而/images/thumb/mythumb.jpg*则进入模块应用程序。

所以,在我的名为thumb.go的应用程序文件中,我需要使用什么代码(参见????):

package thumb

import(
    //要导入的内容
    ???? 
    ????
)

const (
    googleAccessID            = "<serviceAccountEmail>@developer.gserviceaccount.com"
    serviceAccountPEMFilename = "YOUR_SERVICE_ACCOUNT_KEY.pem"
    bucket                    = "myappId.appspot.com"
)

var (
    expiration = time.Now().Add(time.Second * 60) //在60秒后过期
)

func init() {
	http.HandleFunc("/images/thumb/", handleThumb)
}

func handleThumb(w http.ResponseWriter, r *http.Request) {
	ctx := cloud.NewContext(appengine.AppID(c), hc)
    ???? //从URL中获取'mythumb.jpg'的字符串的代码是什么
    ???? //从GCS中获取存储的图像数据的代码是什么
    ???? //编码base64数据的代码是什么
	w.Header().Set("Content-Type", "image/jpeg;")    
    fmt.Fprintf(w, "%v", mythumb.jpg)
}

我从一些示例中获取了很多代码,比如这个这个这个,但迄今为止都没有找到一个可行的代码。我还尝试了这个中的一个示例,它几乎接近我的情况,但也没有成功。

总的来说,主要是因为我不知道在我标记为????的行中应该放置什么代码,以及相关的库或路径应该导入什么。我还检查了GCS权限,看看是否有什么遗漏,如这里这里所描述的那样。

非常感谢您的帮助和建议。

英文:

I have been looking for an example GAE script in go to get my image that I got from the resulted screenshot of PageSpeed Insights and saved it as json_decode object using Kohana/Cache to Google Cloud Storage (GCS).

The reason of this method is simply because I found this Kohana model is the most convenient way writing files to GCS, although I am seeking also other way like this to write files to GCS using Blobstore to serve them while the Go API Files has been deprecate as documented here.

Here is the form of stored object containing the screenshot image data (base64) which is saved as public in default application bucket with object name images/thumb/mythumb.jpg:

stdClass Object
(
    [screenshot] => stdClass Object
        (
            [data] => _9j_4AAQSkZJRgABAQAAAQABAAD_...= // base64 data
            [height] => 240
            [mime_type] => image/jpeg
            [width] => 320
        )

    [otherdata] => Array
        (
            [..] => ..
            [..] => ..
        )

)

I want to get this image that set as public using my customized url as below that to be proceed through go module and also I need it to be expired in a certain time because I have managed to update the image content itself regularly:

> http://myappId.appspot.com/image/thumb/mythumb.jpg

I have set in disptach.yaml to send all image request to my go module as below:

- url: "*/images/*"
  module: go

and set the handler in go.yaml to proceed the image request as below:

handlers:
- url: /images/thumb/.*
  script: _go_app

- url: /images
  static_dir: images

Using this directive I have got that all /images/ request (other than /images/thumb/ request) serve images from the static directory and that /images/thumb/mythumb.jpg goes to the module application.

So left what code I have to use (see ????) in my application file named thumb.go as below:

package thumb

import(
    //what to import
    ????
    ????
)

const (
    googleAccessID            = "<serviceAccountEmail>@developer.gserviceaccount.com"
    serviceAccountPEMFilename = "YOUR_SERVICE_ACCOUNT_KEY.pem"
    bucket                    = "myappId.appspot.com"
)

var (
    expiration = time.Now().Add(time.Second * 60) //expire in 60 seconds
)

func init() {
	http.HandleFunc("/images/thumb/", handleThumb)
}

func handleThumb(w http.ResponseWriter, r *http.Request) {
	ctx := cloud.NewContext(appengine.AppID(c), hc)
    ???? //what code to get the string of 'mythumb.jpg' from url
    ???? //what code to get the image stored data from GCS
    ???? //what code to encoce base64 data
	w.Header().Set("Content-Type", "image/jpeg;")    
    fmt.Fprintf(w, "%v", mythumb.jpg)
}

I have taken many codes from some examples like this, this or this but could not get one works so far. I have also tried a sample from this which is almost close to my case but also found no luck.

So in generally t was mainly due to lack on what are the correct code to be put on the line that I marked by ???? as well the relevant library or path to be imported. I have also checked the GCS permission if something have been missing as described here and here.

I shall thank you much for your help and advise.

答案1

得分: 4

根据我在描述中阅读到的内容,似乎只有实际的Go代码中的????行是相关的部分。如果不是这样,请告诉我。

第一个问题:“如何从URL中获取字符串'mythumb.jpg'的代码?”

根据代码,你想从URL(例如http://localhost/images/thumb/mythumb.jpg)中提取mythumb.jpg。在编写Web应用程序教程中有一个可用的工作示例:

package main

import (
    "fmt"
    "net/http"
)

func handler(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Hi there, I love %s!", r.URL.Path[1:])
}

func main() {
    http.HandleFunc("/", handler)
    http.ListenAndServe(":8080", nil)
}

这样,当你访问http://localhost:8080/monkeys时,会打印出Hi there, I love monkeys!

第二个问题:“如何从GCS获取存储的图像数据的代码?”

你可能要使用的API方法是storage.objects.get

你确实链接到了一个关于Google Cloud Storage的JSON API Go示例,它是一个很好的参考,但与你要解决的问题无关。该示例是为客户端应用程序编写的(因此有redirectURL = "urn:ietf:wg:oauth:2.0:oob"行)。此外,该示例使用了已弃用/过时的oauth2和storage包。

对于想要代表自己访问自己的存储桶的应用程序来说,最干净(且非过时)的方法之一是使用golang/oauth2Google APIs Client Library for Go包。

使用golang/oauth2包进行JSON Web Token身份验证的示例在该存储库中可用:

func ExampleJWTConfig() {
    conf := &jwt.Config{
        Email: "xxx@developer.com",
        // Your RSA private key or PEM file contents.
        // If you have a p12 file instead, you
        // can use `openssl` to export the private key into a pem file.
        //
        //    $ openssl pkcs12 -in key.p12 -out key.pem -nodes
        //
        // It only supports PEM containers with no passphrase.
        PrivateKey: []byte("-----BEGIN RSA PRIVATE KEY-----..."),
        Subject:    "user@example.com",
        TokenURL:   "https://provider.com/o/oauth2/token",
    }
    // Initiate an http.Client, the following GET request will be
    // authorized and authenticated on the behalf of user@example.com.
    client := conf.Client(oauth2.NoContext)
    client.Get("...")
}

接下来,不要直接使用oauth2的client,而是将该客户端与前面提到的Google APIs Client Library for Go一起使用:

service, err := storage.New(client)
if err != nil {
    fatalf(service, "Failed to create service %v", err)
}

注意与过时的JSON API Go示例的相似之处?

在你的处理程序中,你将使用func ObjectsService.Get去获取相关对象。假设你知道objectbucket的名称。

你可以使用类似下面的代码来检索下载链接,就像前面的示例一样:

if res, err := service.Objects.Get(bucketName, objectName).Do(); err == nil {
    fmt.Printf("The media download link for %v/%v is %v.\n\n", bucketName, res.Name, res.MediaLink)
} else {
    fatalf(service, "Failed to get %s/%s: %s.", bucketName, objectName, err)
}

然后,获取文件或对其进行其他操作。完整示例:

import (
    "golang.org/x/oauth2"
    "golang.org/x/oauth2/jwt"
    "google.golang.org/api/storage/v1"
    "fmt"
)

...

const (
    bucketName = "YOUR_BUCKET_NAME"
    objectName = "mythumb.jpg"
)

func main() {
    conf := &jwt.Config{
        Email:      "xxx@developer.com",
        PrivateKey: []byte("-----BEGIN RSA PRIVATE KEY-----..."),
        Subject:    "user@example.com",
        TokenURL:   "https://provider.com/o/oauth2/token",
    }

    client := conf.Client(oauth2.NoContext)

    service, err := storage.New(client)
    if err != nil {
        fatalf(service, "Failed to create service %v", err)
    }

    if res, err := service.Objects.Get(bucketName, objectName).Do(); err == nil {
        fmt.Printf("The media download link for %v/%v is %v.\n\n", bucketName, res.Name, res.MediaLink)
    } else {
        fatalf(service, "Failed to get %s/%s: %s.", bucketName, objectName, err)
    }

    // Go fetch the file, etc.
}

第三个问题:“如何对Base64数据进行编码的代码?”

使用encoding/base64包非常简单。它们甚至在示例中提供了一个示例:

package main

import (
    "encoding/base64"
    "fmt"
)

func main() {
    data := []byte("any + old & data")
    str := base64.StdEncoding.EncodeToString(data)
    fmt.Println(str)
}

希望对你有所帮助。

英文:

From what I've read in your description, it seems that the only relevant parts are the ???? lines in the actual Go code. Let me know if that's not the case.

First ????: "what code to get the string of 'mythumb.jpg' from url"?

From reading the code, you're looking to extract mythumb.jpg from a url like http://localhost/images/thumb/mythumb.jpg. A working example is available at the Writing Web Applications tutorial:

package main

import (
    "fmt"
    "net/http"
)

func handler(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Hi there, I love %s!", r.URL.Path[1:])
}

func main() {
    http.HandleFunc("/", handler)
    http.ListenAndServe(":8080", nil)
}

Such that

http://localhost:8080/monkeys

Prints

Hi there, I love monkeys!

Second ????: "what code to get the image stored data from GCS"?

The API method you're probably looking to use is storage.objects.get.

You did link to one of the JSON API Go Examples for Google Cloud Storage, which is a good general reference, but is not related to the problem you're trying to solve. That particular example is put together for Client-side applications (hence the redirectURL = "urn:ietf:wg:oauth:2.0:oob" line). Additionally, this sample uses deprecated/out-of-date oauth2 and storage packages.

One of the cleanest (and non-deprecated) ways to do this for an application which wants to access its own buckets on behalf of itself would be to use the golang/oauth2 and Google APIs Client Library for Go packages.

An example of how to authenticate with JSON Web Token auth with the golang/oauth2 package is available in the repo:

func ExampleJWTConfig() {
	conf := &jwt.Config{
		Email: "xxx@developer.com",
		// The contents of your RSA private key or your PEM file
		// that contains a private key.
		// If you have a p12 file instead, you
		// can use `openssl` to export the private key into a pem file.
		//
		//    $ openssl pkcs12 -in key.p12 -out key.pem -nodes
		//
		// It only supports PEM containers with no passphrase.
		PrivateKey: []byte("-----BEGIN RSA PRIVATE KEY-----..."),
		Subject:    "user@example.com",
		TokenURL:   "https://provider.com/o/oauth2/token",
	}
	// Initiate an http.Client, the following GET request will be
	// authorized and authenticated on the behalf of user@example.com.
	client := conf.Client(oauth2.NoContext)
	client.Get("...")
}

Next, instead of using the oauth2 client directly, use that client with the Google APIs Client Library for Go mentioned earlier:

service, err := storage.New(client)
if err != nil {
	fatalf(service, "Failed to create service %v", err)
}

Notice the similarity to the out-of-date JSON API Go Examples?

In your handler, you'll want to go out and get the related object using func ObjectsService.Get. Assuming that you know the name of the object and bucket, that is.

Straight from the previous example, you can use code similar to what's below to retrieve the download link:

if res, err := service.Objects.Get(bucketName, objectName).Do(); err == nil {
    fmt.Printf("The media download link for %v/%v is %v.\n\n", bucketName, res.Name, res.MediaLink)
} else {
    fatalf(service, "Failed to get %s/%s: %s.", bucketName, objectName, err)
}

Then, fetch the file, or do whatever you want with it. Full example:

import (
	"golang.org/x/oauth2"
    "golang.org/x/oauth2/jwt"
	"google.golang.org/api/storage/v1"
    "fmt"
)

...

const (
    bucketName = "YOUR_BUCKET_NAME"
    objectName = "mythumb.jpg"
)

func main() {
	conf := &jwt.Config{
		Email: "xxx@developer.com",
		PrivateKey: []byte("-----BEGIN RSA PRIVATE KEY-----..."),
		Subject:    "user@example.com",
		TokenURL:   "https://provider.com/o/oauth2/token",
	 }

	client := conf.Client(oauth2.NoContext)

	service, err := storage.New(client)
	if err != nil {
		fatalf(service, "Failed to create service %v", err)
	}

	if res, err := service.Objects.Get(bucketName, objectName).Do(); err == nil {
	    fmt.Printf("The media download link for %v/%v is %v.\n\n", bucketName, res.Name, res.MediaLink)
	} else {
	    fatalf(service, "Failed to get %s/%s: %s.", bucketName, objectName, err)
	}

    // Go fetch the file, etc.
}

Third ????: "what code to encoce base64 data"?

Pretty simple with the encoding/base64 package. SO simple, that they've included an example:

package main

import (
	"encoding/base64"
	"fmt"
)

func main() {
	data := []byte("any + old & data")
	str := base64.StdEncoding.EncodeToString(data)
	fmt.Println(str)
}

Hope that helps.

huangapple
  • 本文由 发表于 2015年3月19日 23:33:15
  • 转载请务必保留本文链接:https://go.coder-hub.com/29148777.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定