从Sphinx数据库转换为相对复杂的Golang结构的最佳实践

huangapple go评论101阅读模式
英文:

Best Practice to go from sphinx db to somewhat complex golang struct

问题

我正在尝试将一个通过MySQL查询的Sphinx查询转换为Golang结构体,但遇到了困难。这似乎是一个常见的问题,但到目前为止,我仍然无法将其转换为映射或自行解析输出。具体来说,我在Sphinx中有一个类似于{Source: {ID:string, subId:string, Campaigns:[]{CampaignID:string, Status:string}}}的模式。

我尝试使用简单的rows.scan,但这并不能帮助我解析重复的字段,我只能得到一个未解析的字符串。在Sphinx中,键没有用引号括起来,所以JSON.unmarshal似乎完全没有帮助。而且使用sqlx,我尝试构建以下结构体

type CampaignStatus struct {
    CampaignId string
    Status string
}
type Source struct {
    Id               string
    SubId    string
    StatusByCampaign []CampaignStatus
}
type Status struct {
    Source
}

并将一个Status结构体传递给Row.ScanStruct(),我要么得到一个"Missing destination Name Source"错误,要么如果我在Status中命名源成员,我会得到"sql: Scan error on column index 0: unsupported Scan, storing driver.Value type []uint8 into type *v1.Source"。完全透明地说,Sphinx模式中还有一些与StatusByCampaign并列的其他列,它们在其后面,我在我的用例中不关心它们。

这似乎是一个以前肯定遇到过的问题,但我似乎找不到解决方案,除了编写自己的解析器,而我不愿意这样做。

英文:

I am trying to convert a sphinx query via MySQL into a golang struct and am having a hard time. It seems like this should be a common sort of problem, but so far I'm stuck converting it to a map or parsing output myself. Specifically, I have a schema in sphinx that looks like {Source: {ID:string, subId:string, Campaigns:[]{CampaignID:string, Status:string}}}

I've tried using the simple rows.scan but that doesn't help me parse the repeating field, I just get it as an unparsed string. In sphinx, the key's aren't in quotation marks, so JSON.unmarshal doesn't seem to help at all. And using sqlx, I've tried to build up the following struct

type CampaignStatus struct {
    CampaignId string
    Status string
}
type Source struct {
    Id               string
    SubId    string
    StatusByCampaign []CampaignStatus
}
type Status struct {
    Source
}

and passing in a Status struct to Row.ScanStruct() and I get back either a "Missing destination Name Source" error or if I name the source member in Status, I get "sql: Scan error on column index 0: unsupported Scan, storing driver.Value type []uint8 into type *v1.Source". Full disclosure, the sphinx schema has a few other columns as peers with StatusByCampaign, they come after it and I don't care about them in my use case.

This seems like something that has to have been come across before, but I can't seem to find a solution outside of writing my own parser, which I am loath to do.

答案1

得分: 1

我发现我正在使用的遗留系统使用了一个叫做ServiceStack的ASP.Net库来序列化/反序列化我的Sphinx数据库,它使用了自己的自定义JSV格式。ServiceStack的作者称其为JSON和CSV的优化混合体,但据我所知,这种格式在其他地方并没有被采用。

所以看起来我要么要用Go重写那个库,要么更有可能改变索引算法来使用JSON。

英文:

I found out the legacy system I'm working with was using an ASP.Net library called ServiceStack to serialize/deserialize my sphinx db, and it was using it's own custom JSV format, which the author of ServiceStack say as an optimized mix of JSON and CSV, but hasn't been adopted anywhere else as far as I can tell.

So it looks like I'm going to either rewrite that library in Go, or more likely change the indexing algorithm to use JSON.

答案2

得分: 0

我认为问题可能出在你定义结构体的方式上。根据你提供的 JSON 数据 {Source: {ID:string, subId:string, Campaigns:[]{CampaignID:string, Status:string}}},你的结构体应该类似下面的样子:

type object struct {
    Source struct {
        ID        string     `db:"id"`
        SubId     string     `db:"sub_id"`
        Campaigns []compaign `db:"compaigns"`
    } `json:"source"`
}

type compaign struct {
    CampaignID string `db:"compaign_id"`
    Status     string `db:"status"`
}

根据下面的评论,尝试使用 obj := make(map[string]interface{}),而不是定义一个结构体。

另外,根据 https://github.com/jmoiron/sqlx 的说明,结构体应该使用 db 标签而不是 json,所以我进行了更改。你不需要对数据进行编组。

例如:db.Select(&people, "SELECT * FROM person ORDER BY first_name ASC"),其中 &people 是一个指向结构体的引用,比如 var people PeopleStruct

我对你使用的驱动程序不太熟悉,但它应该是类似的,你需要将一个结构体或接口的引用传递给查询函数,它会将数据填充到该引用中。通常情况下,在查询之后不需要对结构体进行解组。

英文:

I think it might be the way you are defining your structs. Given the json you provided {Source: {ID:string, subId:string, Campaigns:[]{CampaignID:string, Status:string}}} your struct should look like something close to whats below.

type object struct {
	Source struct {
		ID        string     `db:"id"`
		SubId     string     `db:"sub_id"`
		Campaigns []compaign `db:"compaigns"`
	} `json:"source"`
}

type compaign struct {
	CampaignID string `db:"compaign_id"`
	Status     string `db:"status"`
}

Given the comments below. Try obj := make(map[string]interaface{}) instead of the defining a struct.

Also taking a look at https://github.com/jmoiron/sqlx the struct should be using the db flag instead of json, so I changed it. You shouldn't have to marshal the data

example db.Select(&people, "SELECT * FROM person ORDER BY first_name ASC") Where &people is a reference to a struct. E.g var people PeopleStruct

Now I'm not famailar with your driver, but it should be the same, in that you pass a reference to a struct or interface with your query and it looks the data into that reference. You don't normally have to unmarshall in the struct after the query.

huangapple
  • 本文由 发表于 2017年4月8日 02:29:18
  • 转载请务必保留本文链接:https://go.coder-hub.com/43284894.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定