Golang minio client can put and remove but not stat or get objects

huangapple go评论79阅读模式
英文:

Golang minio client can put and remove but not stat or get objects

问题

我的Go应用程序可以在同一Docker网络上上传和删除Minio服务器上的对象,但是statObject和getObject返回的信息为空。文件确实存在于Minio控制台中。

以下是我的代码。

func downloadFromMinio(rs *appResource, name string, userId float64) (multipart.File, error) {

    var file multipart.File

    user, err := getUser(rs, int64(userId))

    if err != nil {
        log.Println(err)
        return file, err
    }

    clientIdString := strconv.FormatInt(*user.CompanyId, 10)

    bucketName := "company-" + clientIdString

    log.Println(bucketName)

    log.Println(name)

    found, err := rs.mc.BucketExists(context.Background(), bucketName)
    if err != nil {
        log.Println("Error checking if minio bucket exists")
        log.Println(err)
        return file, err
    }
    if found {
        log.Println("Bucket found")
    }

    objInfo, err := rs.mc.StatObject(context.Background(), bucketName, name, minio.StatObjectOptions{})
    if err != nil {
        log.Println("Error checking minio object metadata")
        log.Println(err)
        return file, err
    } else {
        log.Println(objInfo)
    }

    downloadInfo, err := rs.mc.GetObject(context.Background(), bucketName, name, minio.GetObjectOptions{})

    if err != nil {
        log.Println("Error downloading minio object")
        log.Println(err)
        return file, err
    }

    log.Println("Successfully downloaded minio bytes: ", downloadInfo)

    return file, err
}

以下是控制台输出:

2022/09/10 15:57:39 company-2
2022/09/10 15:57:39 cce2gmsabmis73dfo400
2022/09/10 15:57:39 Bucket found
2022/09/10 15:57:39 {dcd806b9b10e200a826066d5d5a1ab53 cce2gmsabmis73dfo400 2022-09-10 06:02:04 +0000 UTC 497741 application/octet-stream 0001-01-01 00:00:00 +0000 UTC map[Content-Type:[application/octet-stream]] map[] map[] 0 {{ }  } []  false false   0001-01-01 00:00:00 +0000 UTC  <nil> <nil>}
2022/09/10 15:57:39 Successfully downloaded minio bytes:  &{0xc0002f2f60 0xc000077620 0xc000077680 0xc0000776e0 0 {  {0 0 <nil>} 0  {0 0 <nil>} map[] map[] map[] 0 {{ }  } []  false false   {0 0 <nil>}  <nil> <nil>} false false false <nil> false false}

以下是重新上传文件的输出。我期望在等效的下载信息中看到非空数据,并且我期望调用函数中的if file == nil为false,而不是true。如果没有这个测试,我会得到一个超出范围的错误,表示文件为空。

Successfully uploaded bytes:  {company-2 ccevamsabmis738hp8e0 dcd806b9b10e200a826066d5d5a1ab53 497741 0001-01-01 00:00:00 +0000 UTC   0001-01-01 00:00:00 +0000 UTC }
英文:

My go application can upload and remove objects from a Minio server on the same docker network but statObject and getObject return empty information. The file does exist in the minio console.

Here is my code.

func downloadFromMinio(rs *appResource, name string, userId float64) (multipart.File, error) {
var file multipart.File
user, err := getUser(rs, int64(userId))
if err != nil {
log.Println(err)
return file, err
}
clientIdString := strconv.FormatInt(*user.CompanyId, 10)
bucketName := &quot;company-&quot; + clientIdString
log.Println(bucketName)
log.Println(name)
found, err := rs.mc.BucketExists(context.Background(), bucketName)
if err != nil {
log.Println(&quot;Error checking if minio bucket exists&quot;)
log.Println(err)
return file, err
}
if found {
log.Println(&quot;Bucket found&quot;)
}
objInfo, err := rs.mc.StatObject(context.Background(), bucketName, name, minio.StatObjectOptions{})
if err != nil {
log.Println(&quot;Error checking minio object metadata&quot;)
log.Println(err)
return file, err
} else {
log.Println(objInfo)
}
downloadInfo, err := rs.mc.GetObject(context.Background(), bucketName, name, minio.GetObjectOptions{})
if err != nil {
log.Println(&quot;Error downloading minio object&quot;)
log.Println(err)
return file, err
}
log.Println(&quot;Successfully downloaded minio bytes: &quot;, downloadInfo)
return file, err
}

Here is the console output

2022/09/10 15:57:39 company-2
2022/09/10 15:57:39 cce2gmsabmis73dfo400
2022/09/10 15:57:39 Bucket found
2022/09/10 15:57:39 {dcd806b9b10e200a826066d5d5a1ab53 cce2gmsabmis73dfo400 2022-09-10 06:02:04 +0000 UTC 497741 application/octet-stream 0001-01-01 00:00:00 +0000 UTC map[Content-Type:[application/octet-stream]] map[] map[] 0 {{ }  } []  false false   0001-01-01 00:00:00 +0000 UTC  &lt;nil&gt; &lt;nil&gt;}
2022/09/10 15:57:39 Successfully downloaded minio bytes:  &amp;{0xc0002f2f60 0xc000077620 0xc000077680 0xc0000776e0 0 {  {0 0 &lt;nil&gt;} 0  {0 0 &lt;nil&gt;} map[] map[] map[] 0 {{ }  } []  false false   {0 0 &lt;nil&gt;}  &lt;nil&gt; &lt;nil&gt;} false false false &lt;nil&gt; false false}

Here is the output from uploading the file again. I would expect to see non blank data in the equivalent download info and I would expect if file == nil in the calling function to be false instead of true as it is. Without this test I get a slice out of range error, indicating a nil file..

Successfully uploaded bytes:  {company-2 ccevamsabmis738hp8e0 dcd806b9b10e200a826066d5d5a1ab53 497741 0001-01-01 00:00:00 +0000 UTC   0001-01-01 00:00:00 +0000 UTC }

答案1

得分: 2

好的,以下是翻译好的内容:

嗯,MinIO客户端在内部是异步的,这意味着只有在尝试从GetObject()返回的对象中读取时,才会实际查询MinIO(或其他S3存储)。

api-get-object.go中摘录:

// Read将最多len(b)个字节读入b。它返回读取的字节数(0 <= n <= len(b))和遇到的任何错误。在文件结束时返回io.EOF。
func (o *Object) Read(b []byte) (n int, err error) {
    // ...

    // 发送并接收第一个请求的响应。
	response, err := o.doGetRequest(readReq)
	if err != nil && err != io.EOF {
		// 保存错误以供将来调用。
		o.prevErr = err
		return response.Size, err
	}

    // ...
}

你应该这样做:

import (
    "io"
    // 其他导入...
)

downloadInfo, err := rs.mc.GetObject(context.Background(), bucketName, name, minio.GetObjectOptions{})
if err != nil {
    // 处理这个错误
}

// 实际查询MinIO发生在这里。
downloadInfoBytes, err := io.ReadAll(downloadInfo)
if err != nil {
    // 处理这个错误。
    //
    // 可能,这个错误将是"minio.ErrorResponse"类型的。检查它的字段以了解实际问题是什么。
}
英文:

Well, the MinIO client is asynchronous under the hood, which means that an actual query to MinIO (or other S3 storage) happens only on trying to read from the object returned by GetObject().

Abstract from api-get-object.go:

// Read reads up to len(b) bytes into b. It returns the number of
// bytes read (0 &lt;= n &lt;= len(b)) and any error encountered. Returns
// io.EOF upon end of file.
func (o *Object) Read(b []byte) (n int, err error) {
    // ...

    // Send and receive from the first request.
	response, err := o.doGetRequest(readReq)
	if err != nil &amp;&amp; err != io.EOF {
		// Save the error for future calls.
		o.prevErr = err
		return response.Size, err
	}

    // ...
}

You should do something like this:

import (
    &quot;io&quot;
    // other imports...
)

downloadInfo, err := rs.mc.GetObject(context.Background(), bucketName, name, minio.GetObjectOptions{})
if err != nil {
    // handle this error somehow
}

// The actual query to MinIO happens here.
downloadInfoBytes, err := io.ReadAll(downloadInfo)
if err != nil {
    // Handle this error.
    //
    // Possibly, this error will be of the &quot;minio.ErrorResponse&quot; type. Check its fields to understand what is an actual problem. 
}

huangapple
  • 本文由 发表于 2022年9月10日 15:18:04
  • 转载请务必保留本文链接:https://go.coder-hub.com/73669999.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定