Extracting Data From Kafka REST Proxy in Go

huangapple go评论78阅读模式
英文:

Extracting Data From Kafka REST Proxy in Go

问题

我正在使用Kafka的REST代理实例来生产和消费消息。我正在使用API获取新消息,但是我无法将这些消息转换为Go语言中的结构模型。例如:

// 获取记录
req, err := http.NewRequest(http.MethodGet, fmt.Sprintf(FETCH_CONSUMER, URL, GROUP, CONSUMER), nil)
if err != nil {
panic(err)
}
req.Header.Add("Accept", CONTENT_TYPE)
respRecords, err := client.Do(req)
if err != nil {
panic(err)
}
defer respRecords.Body.Close()
fmt.Printf("Response %s\n", respRecords.Status)
fmt.Println(respRecords.Body)
recordsBodyResp := bufio.NewScanner(respRecords.Body)

for recordsBodyResp.Scan() {
fmt.Printf("<--Body %s\n", recordsBodyResp.Text())
}

返回的值的格式如下:

[{"topic":"backward","key":null,"value":{"AdoptionID":"abcd123","IPAddress":"8.8.8.8","Port":"80","Status":"requested"},"partition":0,"offset":7}]

由于它是一个对象数组,我想提取键"value"的值部分到一个结构体中。
这就是我卡住的地方。

英文:

I am using the REST proxy instance of Kafka for producing and consuming messages.Using the API to get new messages but I am not able to convert those messages to a struct model in Go. For example:

// Get records
req, err := http.NewRequest(http.MethodGet, fmt.Sprintf(FETCH_CONSUMER, URL, GROUP, CONSUMER), nil)
if err != nil {
	panic(err)
}
req.Header.Add(&quot;Accept&quot;, CONTENT_TYPE)
respRecords, err := client.Do(req)
if err != nil {
	panic(err)
}
defer respRecords.Body.Close()
fmt.Printf(&quot;Response %s\n&quot;, respRecords.Status)
fmt.Println(respRecords.Body)
recordsBodyResp := bufio.NewScanner(respRecords.Body)

for recordsBodyResp.Scan() {
	fmt.Printf(&quot;&lt;--Body %s\n&quot;, recordsBodyResp.Text())
	
}

The value returned is in the following format:

[{&quot;topic&quot;:&quot;backward&quot;,&quot;key&quot;:null,&quot;value&quot;:{&quot;AdoptionID&quot;:&quot;abcd123&quot;,&quot;IPAddress&quot;:&quot;8.8.8.8&quot;,&quot;Port&quot;:&quot;80&quot;,&quot;Status&quot;:&quot;requested&quot;},&quot;partition&quot;:0,&quot;offset&quot;:7}]

Since it's an array of objects, I want to extract the value portion of the key "value" into a struct.
That is where I am stuck.

答案1

得分: 2

你可以创建一个类似这样的结构体:

type AutoGenerated []struct {
    Topic     string      `json:"topic"`
    Key       interface{} `json:"key"`
    Value     Value       `json:"value"`
    Partition int         `json:"partition"`
    Offset    int         `json:"offset"`
}
type Value struct {
    AdoptionID string `json:"AdoptionID"`
    IPAddress  string `json:"IPAddress"`
    Port       string `json:"Port"`
    Status     string `json:"Status"`
}

然后在该结构体中进行解析。

参考这个示例代码

package main

import (
    "fmt"
    "encoding/json"
)

func main() {
    type Value struct {
        AdoptionID string `json:"AdoptionID"`
        IPAddress  string `json:"IPAddress"`
        Port       string `json:"Port"`
        Status     string `json:"Status"`
    }

    type AutoGenerated []struct {
        Topic     string      `json:"topic"`
        Key       interface{} `json:"key"`
        Value     Value       `json:"value"`
        Partition int         `json:"partition"`
        Offset    int         `json:"offset"`
    }

    byt := []byte(`[{"topic":"backward","key":null,"value":{"AdoptionID":"abcd123","IPAddress":"8.8.8.8","Port":"80","Status":"requested"},"partition":0,"offset":7}]`)
    var dat AutoGenerated

    if err := json.Unmarshal(byt, &dat); err != nil {
        panic(err)
    }
    fmt.Printf("%#v", dat)
}
英文:

You can create a struct like:

type AutoGenerated []struct {
	Topic     string      `json:&quot;topic&quot;`
	Key       interface{} `json:&quot;key&quot;`
	Value     Value       `json:&quot;value&quot;`
	Partition int         `json:&quot;partition&quot;`
	Offset    int         `json:&quot;offset&quot;`
}
type Value struct {
	AdoptionID string `json:&quot;AdoptionID&quot;`
	IPAddress  string `json:&quot;IPAddress&quot;`
	Port       string `json:&quot;Port&quot;`
	Status     string `json:&quot;Status&quot;`
}

And Unmarshal in that struct.

See this sample code:

package main

import (
	&quot;fmt&quot;
	    &quot;encoding/json&quot;

)

func main() {



type Value struct {
    AdoptionID string `json:&quot;AdoptionID&quot;`
    IPAddress  string `json:&quot;IPAddress&quot;`
    Port       string `json:&quot;Port&quot;`
    Status     string `json:&quot;Status&quot;`
}

type AutoGenerated []struct {
    Topic     string      `json:&quot;topic&quot;`
    Key       interface{} `json:&quot;key&quot;`
    Value     Value       `json:&quot;value&quot;`
    Partition int         `json:&quot;partition&quot;`
    Offset    int         `json:&quot;offset&quot;`
}


    byt := []byte(`[{&quot;topic&quot;:&quot;backward&quot;,&quot;key&quot;:null,&quot;value&quot;:{&quot;AdoptionID&quot;:&quot;abcd123&quot;,&quot;IPAddress&quot;:&quot;8.8.8.8&quot;,&quot;Port&quot;:&quot;80&quot;,&quot;Status&quot;:&quot;requested&quot;},&quot;partition&quot;:0,&quot;offset&quot;:7}]`)
   var dat AutoGenerated

   if err := json.Unmarshal(byt, &amp;dat); err != nil {
        panic(err)
    }
    fmt.Printf(&quot;%#v&quot;, dat)

}

huangapple
  • 本文由 发表于 2021年9月28日 21:05:15
  • 转载请务必保留本文链接:https://go.coder-hub.com/69362308.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定