从Python项目中加载数据存储实体到Go语言会导致嵌套的结构体切片错误。

huangapple
go评论122阅读模式
英文:

Loading datastore entities from Python project in Go leads to nested structs slices of slices error

问题

我正在为我的Google AppEngine项目编写一个模块，使用Go语言出于性能原因，但需要能够从数据存储中读取一些实体。我编写了Go代码，以便能够读取我在Python中构建的实体，但是我遇到了以下错误：

datastore: flattening nested structs leads to a slice of slices: field "Messages"

Python中的模型定义：

  1. class ModelB(ndb.Model):
  2.     msg_id = ndb.StringProperty(indexed=False)
  3.     cat_ids = ndb.StringProperty(repeated=True, indexed=False)
  4.     list_ids = ndb.StringProperty(repeated=True, indexed=False)
  5.     default_list_id_index = ndb.IntegerProperty(indexed=False)
  7. class ModelA(ndb.Model):
  8.     date_join = ndb.DateTimeProperty(auto_now_add=True)
  9.     name = ndb.StringProperty()
  10.     owner_salutation = ndb.StringProperty(indexed=False)
  11.     owner_email_address = ndb.StringProperty()
  12.     logo_url = ndb.StringProperty(indexed=False)
  13.     ...
  14.     messages = ndb.LocalStructuredProperty(ModelB, name='bm', repeated=True)

Go中的代码：

  1. type ModelB struct {
  2.     MessageID          string   `datastore:"msg_id,noindex"`
  3.     CategoryIDs        []string `datastore:"cat_ids,noindex"`
  4.     ListIDs            []string `datastore:"list_ids,noindex"`
  5.     DefaultListIDIndex int      `datastore:"default_list_id_index,noindex"`
  6. }
  8. type ModelA struct {
  9.     DateJoin          time.Time `datastore:"date_join,"`
  10.     Name              string    `datastore:"name,"`
  11.     OwnerSalutation   string    `datastore:"owner_salutation,noindex"`
  12.     OwnerEmailAddress string    `datastore:"owner_email_address,"`
  13.     LogoURL           string    `datastore:"logo_url,noindex"`
  14.     Messages          []ModelB  `datastore:"bm,"`
  15. }

这里有什么问题吗？这只是Go与Python模型定义之间的功能不兼容吗？

尝试解码ModelB

将ModelA重新定义如下：

  1. import pb "appengine_internal/datastore"
  2. import proto "code.google.com/p/goprotobuf/proto"
  4. type ModelA struct {
  5.     DateJoin          time.Time `datastore:"date_join,"`
  6.     Name              string    `datastore:"name,"`
  7.     OwnerSalutation   string    `datastore:"owner_salutation,noindex"`
  8.     OwnerEmailAddress string    `datastore:"owner_email_address,"`
  9.     LogoURL           string    `datastore:"logo_url,noindex"`
  10.     Messages          []ModelB  `datastore:"-"`
  11. }
  13. // Load is implemented for the PropertyLoaderSaver interface.
  15. func (seller *ModelA) Load(c <-chan datastore.Property) error {
  16.     f := make(chan datastore.Property, 100)
  17.     for p := range c {
  18.         if p.Name == "bm" {
  19.             var val pb.EntityProto
  20.             err := proto.Unmarshal([]byte(p.Value.(string)), &val)
  21.             if err != nil {
  22.                 return err
  23.             }
  24.             //TODO: Store result as a new ModelB
  25.         } else {
  26.             f <- p
  27.         }
  28.     }
  29.     close(f)
  30.     return datastore.LoadStruct(seller, f)
  31. }

但是我收到以下错误：
proto: required field "{Unknown}" not set

英文:

I am writing a module in my Google AppEngine project in Go for performance reasons but need to be able to read from some of the entities I have in datastore. I wrote out the Go code to be able to read the entities I built out in Python but I am getting the following error:

datastore: flattening nested structs leads to a slice of slices: field &quot;Messages&quot;

Model Definitions in Python:

  1. class ModelB(ndb.Model):
  2.     msg_id = ndb.StringProperty(indexed=False)
  3.     cat_ids = ndb.StringProperty(repeated=True, indexed=False)
  4.     list_ids = ndb.StringProperty(repeated=True, indexed=False)
  5.     default_list_id_index = ndb.IntegerProperty(indexed=False)
  7. class ModelA(ndb.Model):
  8.     date_join = ndb.DateTimeProperty(auto_now_add=True)
  9.     name = ndb.StringProperty()
  10.     owner_salutation = ndb.StringProperty(indexed=False)
  11.     owner_email_address = ndb.StringProperty()
  12.     logo_url = ndb.StringProperty(indexed=False)
  13.     ...
  14.     messages = ndb.LocalStructuredProperty(ModelB, name=&#39;bm&#39;, repeated=True)

And in Go:

  1. type ModelB struct {
  2. 	MessageID          string   `datastore:&quot;msg_id,noindex&quot;`
  3. 	CategoryIDs        []string `datastore:&quot;cat_ids,noindex&quot;`
  4. 	ListIDs            []string `datastore:&quot;list_ids,noindex&quot;`
  5. 	DefaultListIDIndex int      `datastore:&quot;default_list_id_index,noindex&quot;`
  6. }
  8. type ModelA struct {
  9. 	DateJoin          time.Time `datastore:&quot;date_join,&quot;`
  10. 	Name              string    `datastore:&quot;name,&quot;`
  11. 	OwnerSalutation   string    `datastore:&quot;owner_salutation,noindex&quot;`
  12. 	OwnerEmailAddress string    `datastore:&quot;owner_email_address,&quot;`
  13. 	LogoURL           string    `datastore:&quot;logo_url,noindex&quot;`
  14. 	Messages          []ModelB  `datastore:&quot;bm,&quot;`
  15. }

Is there something I'm doing wrong here? Is just a feature incompatibility between Go vs Python model definitions?

Attempt to Decode ModelB

Re-define ModelA as follows:

  1. import pb &quot;appengine_internal/datastore&quot;
  2. import proto &quot;code.google.com/p/goprotobuf/proto&quot;
  4. type ModelA struct {
  5. 	DateJoin          time.Time `datastore:&quot;date_join,&quot;`
  6. 	Name              string    `datastore:&quot;name,&quot;`
  7. 	OwnerSalutation   string    `datastore:&quot;owner_salutation,noindex&quot;`
  8. 	OwnerEmailAddress string    `datastore:&quot;owner_email_address,&quot;`
  9. 	LogoURL           string    `datastore:&quot;logo_url,noindex&quot;`
  10. 	Messages          []ModelB  `datastore:&quot;-&quot;`
  11. }
  13. // Load is implemented for the PropertyLoaderSaver interface.
  15. func (seller *ModelA) Load(c &lt;-chan datastore.Property) error {
  16.   f := make(chan datastore.Property, 100)
  17.   for p := range c {
  18.     if p.Name == &quot;bm&quot; {
  19.       var val pb.EntityProto
  20.       err := proto.Unmarshal([]byte(p.Value.(string)), &amp;val)
  21.       if err != nil {
  22.         return err
  23.       }
  24.       //TODO: Store result as a new ModelB
  25.     } else {
  26.       f &lt;- p
  27.     }
  28.   }
  29.   close(f)
  30.   return datastore.LoadStruct(seller, f)
  31. }

But I receive the following error:
proto: required field &quot;{Unknown}&quot; not set

答案1

得分: 5

Go的数据存储包不支持像那样的两层切片。你可以有[]ModelB，只要ModelB不包含任何切片。或者，你可以在ModelA中使用ModelB，而ModelB可以包含切片。但是你不能同时拥有[]ModelB和ModelB有切片。查看代码以获取错误条件。你的选择：

  1. 不要在Go中这样做
  2. 编写自己的数据存储反序列化器来处理这种情况-这可能很困难
  3. 更改你的Python数据结构以满足Go的要求，并重新编写你的数据
英文:

The Go datastore package doesn't support two layers of slices like that. You can have []ModelB, as long as ModelB doesn't contain any slices. Or, you can use ModelB in ModelA, and ModelB can have slices in it. But you can't have both []ModelB and ModelB has slices. See the code for the error condition. Your options:

  1. don't do it in Go
  2. write your own datastore deserializer to handle this case - this is probably hard
  3. change your python data structures to satisfy the Go requirements and rewrite your data

答案2

得分: 3

我猜如果你足够深入地挖掘，你会找到答案：

首先，在Python中定义LocalStructuredProperty属性时，你需要设置keep_keys=True。

  1. class ModelB(ndb.Model):
  2.     msg_id = ndb.StringProperty(indexed=False)
  3.     cat_ids = ndb.StringProperty(repeated=True, indexed=False)
  4.     list_ids = ndb.StringProperty(repeated=True, indexed=False)
  5.     default_list_id_index = ndb.IntegerProperty(indexed=False)
  7. class ModelA(ndb.Model):
  8.     date_join = ndb.DateTimeProperty(auto_now_add=True)
  9.     name = ndb.StringProperty()
  10.     owner_salutation = ndb.StringProperty(indexed=False)
  11.     owner_email_address = ndb.StringProperty()
  12.     logo_url = ndb.StringProperty(indexed=False)
  13.     ...
  14.     messages = ndb.LocalStructuredProperty(ModelB, name='bm', repeated=True, keep_keys=True)

在我的代码中，通过简单地重新定义并映射实体，对每个实体执行put()来修复表示。

然后在我的Go代码中：

  1. type ModelB struct {
  2.     MessageID          string   `datastore:"msg_id,noindex"`
  3.     CategoryIDs        []string `datastore:"cat_ids,noindex"`
  4.     ListIDs            []string `datastore:"list_ids,noindex"`
  5.     DefaultListIDIndex int      `datastore:"default_list_id_index,noindex"`
  6. }
  8. type ModelA struct {
  9.     DateJoin          time.Time `datastore:"date_join,"`
  10.     Name              string    `datastore:"name,"`
  11.     OwnerSalutation   string    `datastore:"owner_salutation,noindex"`
  12.     OwnerEmailAddress string    `datastore:"owner_email_address,"`
  13.     LogoURL           string    `datastore:"logo_url,noindex"`
  14.     Messages          []ModelB  `datastore:"-"`
  15. }
  17. // Load is implemented for the PropertyLoaderSaver interface.
  18. func (s *ModelA) Load(c <-chan datastore.Property) (err error) {
  19.     f := make(chan datastore.Property, 32)
  20.     errc := make(chan error, 1)
  21.     defer func() {
  22.         if err == nil {
  23.             err = <-errc
  24.         }
  25.     }()
  26.     go func() {
  27.         defer close(f)
  28.         for p := range c {
  29.             if p.Name == "bm" {
  30.                 var b ModelB
  31.                 err := loadLocalStructuredProperty(&b, []byte(p.Value.(string)))
  32.                 if err != nil {
  33.                     errc <- err
  34.                     return
  35.                 }
  36.                 s.Messages = append(s.Messages, b)
  37.             } else {
  38.                 f <- p
  39.             }
  40.         }
  41.         errc <- nil
  42.     }()
  43.     return datastore.LoadStruct(s, f)
  44. }

由于一个关键函数没有被导出，并且为了简化需要复制的代码量，我从appengine/datastore包中复制了一些内容，我放弃了对Reference类型的支持。我在问题跟踪器上提交了一个问题，看看是否可以导出loadEntity函数：https://code.google.com/p/googleappengine/issues/detail?id=10426

  1. import (
  2.     "errors"
  3.     "time"
  5.     "appengine"
  6.     "appengine/datastore"
  8.     pb "appengine_internal/datastore"
  9.     proto "code.google.com/p/goprotobuf/proto"
  10. )
  12. func loadLocalStructuredProperty(dst interface{}, raw_proto []byte) error {
  13.     var val pb.EntityProto
  14.     err := proto.Unmarshal(raw_proto, &val)
  15.     if err != nil {
  16.         return err
  17.     }
  18.     return loadEntity(dst, &val)
  19. }
  21. //Copied from appengine/datastore since its not exported
  23. // loadEntity loads an EntityProto into PropertyLoadSaver or struct pointer.
  24. func loadEntity(dst interface{}, src *pb.EntityProto) (err error) {
  25.     c := make(chan datastore.Property, 32)
  26.     errc := make(chan error, 1)
  27.     defer func() {
  28.         if err == nil {
  29.             err = <-errc
  30.         }
  31.     }()
  32.     go protoToProperties(c, errc, src)
  33.     if e, ok := dst.(datastore.PropertyLoadSaver); ok {
  34.         return e.Load(c)
  35.     }
  36.     return datastore.LoadStruct(dst, c)
  37. }
  39. func protoToProperties(dst chan<- datastore.Property, errc chan<- error, src *pb.EntityProto) {
  40.     defer close(dst)
  41.     props, rawProps := src.Property, src.RawProperty
  42.     for {
  43.         var (
  44.             x       *pb.Property
  45.             noIndex bool
  46.         )
  47.         if len(props) > 0 {
  48.             x, props = props[0], props[1:]
  49.         } else if len(rawProps) > 0 {
  50.             x, rawProps = rawProps[0], rawProps[1:]
  51.             noIndex = true
  52.         } else {
  53.             break
  54.         }
  56.         var value interface{}
  57.         if x.Meaning != nil && *x.Meaning == pb.Property_INDEX_VALUE {
  58.             value = indexValue{x.Value}
  59.         } else {
  60.             var err error
  61.             value, err = propValue(x.Value, x.GetMeaning())
  62.             if err != nil {
  63.                 errc <- err
  64.                 return
  65.             }
  66.         }
  67.         dst <- datastore.Property{
  68.             Name:     x.GetName(),
  69.             Value:    value,
  70.             NoIndex:  noIndex,
  71.             Multiple: x.GetMultiple(),
  72.         }
  73.     }
  74.     errc <- nil
  75. }
  77. func fromUnixMicro(t int64) time.Time {
  78.     return time.Unix(t/1e6, (t%1e6)*1e3)
  79. }
  81. // propValue returns a Go value that combines the raw PropertyValue with a
  82. // meaning. For example, an Int64Value with GD_WHEN becomes a time.Time.
  83. func propValue(v *pb.PropertyValue, m pb.Property_Meaning) (interface{}, error) {
  84.     switch {
  85.     case v.Int64Value != nil:
  86.         if m == pb.Property_GD_WHEN {
  87.             return fromUnixMicro(*v.Int64Value), nil
  88.         } else {
  89.             return *v.Int64Value, nil
  90.         }
  91.     case v.BooleanValue != nil:
  92.         return *v.BooleanValue, nil
  93.     case v.StringValue != nil:
  94.         if m == pb.Property_BLOB {
  95.             return []byte(*v.StringValue), nil
  96.         } else if m == pb.Property_BLOBKEY {
  97.             return appengine.BlobKey(*v.StringValue), nil
  98.         } else {
  99.             return *v.StringValue, nil
  100.         }
  101.     case v.DoubleValue != nil:
  102.         return *v.DoubleValue, nil
  103.     case v.Referencevalue != nil:
  104.         return nil, errors.New("Not Implemented!")
  105.     }
  106.     return nil, nil
  107. }
  109. // indexValue is a Property value that is created when entities are loaded from
  110. // an index, such as from a projection query.
  111. //
  112. // Such Property values do not contain all of the metadata required to be
  113. // faithfully represented as a Go value, and are instead represented as an
  114. // opaque indexValue. Load the properties into a concrete struct type (e.g. by
  115. // passing a struct pointer to Iterator.Next) to reconstruct actual Go values
  116. // of type int, string, time.Time, etc.
  117. type indexValue struct {
  118.     value *pb.PropertyValue
  119. }

希望这能帮到你！

英文:

I guess if you dig enough you'll find the answer:

First off, when defining the LocalStructuredProperty properties in Python, you need to set keep_keys=True

  1. class ModelB(ndb.Model):
  2. msg_id = ndb.StringProperty(indexed=False)
  3. cat_ids = ndb.StringProperty(repeated=True, indexed=False)
  4. list_ids = ndb.StringProperty(repeated=True, indexed=False)
  5. default_list_id_index = ndb.IntegerProperty(indexed=False)
  6. class ModelA(ndb.Model):
  7. date_join = ndb.DateTimeProperty(auto_now_add=True)
  8. name = ndb.StringProperty()
  9. owner_salutation = ndb.StringProperty(indexed=False)
  10. owner_email_address = ndb.StringProperty()
  11. logo_url = ndb.StringProperty(indexed=False)
  12. ...
  13. messages = ndb.LocalStructuredProperty(ModelB, name=&#39;bm&#39;, repeated=True, keep_keys=True)

A simple redefinition in my code and mapping over my entities doing a put() on each fixed up the representation.

Then in my Go Code:

  1. type ModelB struct {
  2. MessageID          string   `datastore:&quot;msg_id,noindex&quot;`
  3. CategoryIDs        []string `datastore:&quot;cat_ids,noindex&quot;`
  4. ListIDs            []string `datastore:&quot;list_ids,noindex&quot;`
  5. DefaultListIDIndex int      `datastore:&quot;default_list_id_index,noindex&quot;`
  6. }
  7. type ModelA struct {
  8. DateJoin          time.Time `datastore:&quot;date_join,&quot;`
  9. Name              string    `datastore:&quot;name,&quot;`
  10. OwnerSalutation   string    `datastore:&quot;owner_salutation,noindex&quot;`
  11. OwnerEmailAddress string    `datastore:&quot;owner_email_address,&quot;`
  12. LogoURL           string    `datastore:&quot;logo_url,noindex&quot;`
  13. Messages          []ModelB  `datastore:&quot;-&quot;`
  14. }
  15. // Load is implemented for the PropertyLoaderSaver interface.
  16. func (s *ModelA) Load(c &lt;-chan datastore.Property) (err error) {
  17. f := make(chan datastore.Property, 32)
  18. errc := make(chan error, 1)
  19. defer func() {
  20. if err == nil {
  21. err = &lt;-errc
  22. }
  23. }()
  24. go func() {
  25. defer close(f)
  26. for p := range c {
  27. if p.Name == &quot;bm&quot; {
  28. var b ModelB
  29. err := loadLocalStructuredProperty(&amp;b, []byte(p.Value.(string)))
  30. if err != nil {
  31. errc &lt;- err
  32. return
  33. }
  34. s.Messages = append(s.Messages, b)
  35. } else {
  36. f &lt;- p
  37. }
  38. }
  39. errc &lt;- nil
  40. }()
  41. return datastore.LoadStruct(s, f)
  42. }

I had to copy a bunch from the appengine/datastore package as a key function wasn't exported and to simplify the amount of code I needed to copy, I dropped support for Reference types. I opened a ticket on the issue tracker to see if we can get the loadEntity function exported: https://code.google.com/p/googleappengine/issues/detail?id=10426

  1. import (    
  2. &quot;errors&quot;    
  3. &quot;time&quot;    
  4. &quot;appengine&quot;    
  5. &quot;appengine/datastore&quot;        
  6. pb &quot;appengine_internal/datastore&quot;    
  7. proto &quot;code.google.com/p/goprotobuf/proto&quot;    
  8. )    
  9. func loadLocalStructuredProperty(dst interface{}, raw_proto []byte) error {    
  10. var val pb.EntityProto    
  11. err := proto.Unmarshal(raw_proto, &amp;val)    
  12. if err != nil {    
  13. return err    
  14. }    
  15. return loadEntity(dst, &amp;val)    
  16. }
  17. //Copied from appengine/datastore since its not exported
  18. // loadEntity loads an EntityProto into PropertyLoadSaver or struct pointer.
  19. func loadEntity(dst interface{}, src *pb.EntityProto) (err error) {
  20. c := make(chan datastore.Property, 32)
  21. errc := make(chan error, 1)
  22. defer func() {
  23. if err == nil {
  24. err = &lt;-errc
  25. }
  26. }()
  27. go protoToProperties(c, errc, src)
  28. if e, ok := dst.(datastore.PropertyLoadSaver); ok {
  29. return e.Load(c)
  30. }
  31. return datastore.LoadStruct(dst, c)
  32. }
  33. func protoToProperties(dst chan&lt;- datastore.Property, errc chan&lt;- error, src *pb.EntityProto) {
  34. defer close(dst)
  35. props, rawProps := src.Property, src.RawProperty
  36. for {
  37. var (
  38. x       *pb.Property
  39. noIndex bool
  40. )
  41. if len(props) &gt; 0 {
  42. x, props = props[0], props[1:]
  43. } else if len(rawProps) &gt; 0 {
  44. x, rawProps = rawProps[0], rawProps[1:]
  45. noIndex = true
  46. } else {
  47. break
  48. }
  49. var value interface{}
  50. if x.Meaning != nil &amp;&amp; *x.Meaning == pb.Property_INDEX_VALUE {
  51. value = indexValue{x.Value}
  52. } else {
  53. var err error
  54. value, err = propValue(x.Value, x.GetMeaning())
  55. if err != nil {
  56. errc &lt;- err
  57. return
  58. }
  59. }
  60. dst &lt;- datastore.Property{
  61. Name:     x.GetName(),
  62. Value:    value,
  63. NoIndex:  noIndex,
  64. Multiple: x.GetMultiple(),
  65. }
  66. }
  67. errc &lt;- nil
  68. }
  69. func fromUnixMicro(t int64) time.Time {
  70. return time.Unix(t/1e6, (t%1e6)*1e3)
  71. }
  72. // propValue returns a Go value that combines the raw PropertyValue with a
  73. // meaning. For example, an Int64Value with GD_WHEN becomes a time.Time.
  74. func propValue(v *pb.PropertyValue, m pb.Property_Meaning) (interface{}, error) {
  75. switch {
  76. case v.Int64Value != nil:
  77. if m == pb.Property_GD_WHEN {
  78. return fromUnixMicro(*v.Int64Value), nil
  79. } else {
  80. return *v.Int64Value, nil
  81. }
  82. case v.BooleanValue != nil:
  83. return *v.BooleanValue, nil
  84. case v.StringValue != nil:
  85. if m == pb.Property_BLOB {
  86. return []byte(*v.StringValue), nil
  87. } else if m == pb.Property_BLOBKEY {
  88. return appengine.BlobKey(*v.StringValue), nil
  89. } else {
  90. return *v.StringValue, nil
  91. }
  92. case v.DoubleValue != nil:
  93. return *v.DoubleValue, nil
  94. case v.Referencevalue != nil:
  95. return nil, errors.New(&quot;Not Implemented!&quot;)
  96. }
  97. return nil, nil
  98. }
  99. // indexValue is a Property value that is created when entities are loaded from
  100. // an index, such as from a projection query.
  101. //
  102. // Such Property values do not contain all of the metadata required to be
  103. // faithfully represented as a Go value, and are instead represented as an
  104. // opaque indexValue. Load the properties into a concrete struct type (e.g. by
  105. // passing a struct pointer to Iterator.Next) to reconstruct actual Go values
  106. // of type int, string, time.Time, etc.
  107. type indexValue struct {
  108. value *pb.PropertyValue
  109. }

答案3

得分: 1

someone1的解决方案很好，但是我有很多百万个实体，并且不想重新放置它们（以添加keep_keys=True到LocalStructuredProperty）。

因此，我创建了一个简化版的EntityProto，它去除了对键和路径等的依赖...只需将pb.EntityProto替换为LocalEntityProto，现有的使用Python编写的实体应该可以正常加载（我正在使用PropertyLoadSaver来处理嵌套结构）。

免责声明：我只是用它来从Go中读取-我还没有尝试将相同的实体写回以查看它们是否仍然可以在Python中加载。

  1. import pb "google.golang.org/appengine/internal/datastore"
  2. import proto "github.com/golang/protobuf/proto"
  4. type LocalEntityProto struct {
  5.     Kind             *pb.EntityProto_Kind `protobuf:"varint,4,opt,name=kind,enum=appengine.EntityProto_Kind" json:"kind,omitempty"`
  6.     KindUri          *string              `protobuf:"bytes,5,opt,name=kind_uri" json:"kind_uri,omitempty"`
  7.     Property         []*pb.Property       `protobuf:"bytes,14,rep,name=property" json:"property,omitempty"`
  8.     RawProperty      []*pb.Property       `protobuf:"bytes,15,rep,name=raw_property" json:"raw_property,omitempty"`
  9.     Rank             *int32               `protobuf:"varint,18,opt,name=rank" json:"rank,omitempty"`
  10.     XXX_unrecognized []byte               `json:"-"`
  11. }
  13. func (m *LocalEntityProto) Reset()         { *m = LocalEntityProto{} }
  14. func (m *LocalEntityProto) String() string { return proto.CompactTextString(m) }
  15. func (*LocalEntityProto) ProtoMessage()    {}
  17. func (m *LocalEntityProto) GetKind() pb.EntityProto_Kind {
  18.     if m != nil && m.Kind != nil {
  19.         return *m.Kind
  20.     }
  21.     return pb.EntityProto_GD_CONTACT
  22. }
  24. func (m *LocalEntityProto) GetKindUri() string {
  25.     if m != nil && m.KindUri != nil {
  26.         return *m.KindUri
  27.     }
  28.     return ""
  29. }
  31. func (m *LocalEntityProto) GetProperty() []*pb.Property {
  32.     if m != nil {
  33.         return m.Property
  34.     }
  35.     return nil
  36. }
  38. func (m *LocalEntityProto) GetRawProperty() []*pb.Property {
  39.     if m != nil {
  40.         return m.RawProperty
  41.     }
  42.     return nil
  43. }
  45. func (m *LocalEntityProto) GetRank() int32 {
  46.     if m != nil && m.Rank != nil {
  47.         return *m.Rank
  48.     }
  49.     return 0
  50. }
英文:

The solution by someone1 works great but I have many millions of entities and didn't want to have to re-put them all (to add the keep_keys=True to the LocalStructuredProperty).

So, I created a cut-down version of EntityProto which removes the dependency on the key & path etc... Simply replace pb.EntityProto with LocalEntityProto and the existing python-written entities should load OK (I'm using a PropertyLoadSaver for the nested struct).

Disclaimer: I'm only using this to read from Go - I haven't tried writing the same entities back to see if they still load in Python.

  1. import pb &quot;google.golang.org/appengine/internal/datastore&quot;
  2. import proto &quot;github.com/golang/protobuf/proto&quot;
  3. type LocalEntityProto struct {
  4. Kind             *pb.EntityProto_Kind `protobuf:&quot;varint,4,opt,name=kind,enum=appengine.EntityProto_Kind&quot; json:&quot;kind,omitempty&quot;`
  5. KindUri          *string              `protobuf:&quot;bytes,5,opt,name=kind_uri&quot; json:&quot;kind_uri,omitempty&quot;`
  6. Property         []*pb.Property       `protobuf:&quot;bytes,14,rep,name=property&quot; json:&quot;property,omitempty&quot;`
  7. RawProperty      []*pb.Property       `protobuf:&quot;bytes,15,rep,name=raw_property&quot; json:&quot;raw_property,omitempty&quot;`
  8. Rank             *int32               `protobuf:&quot;varint,18,opt,name=rank&quot; json:&quot;rank,omitempty&quot;`
  9. XXX_unrecognized []byte               `json:&quot;-&quot;`
  10. }
  11. func (m *LocalEntityProto) Reset()         { *m = LocalEntityProto{} }
  12. func (m *LocalEntityProto) String() string { return proto.CompactTextString(m) }
  13. func (*LocalEntityProto) ProtoMessage()    {}
  14. func (m *LocalEntityProto) GetKind() pb.EntityProto_Kind {
  15. if m != nil &amp;&amp; m.Kind != nil {
  16. return *m.Kind
  17. }
  18. return pb.EntityProto_GD_CONTACT
  19. }
  20. func (m *LocalEntityProto) GetKindUri() string {
  21. if m != nil &amp;&amp; m.KindUri != nil {
  22. return *m.KindUri
  23. }
  24. return &quot;&quot;
  25. }
  26. func (m *LocalEntityProto) GetProperty() []*pb.Property {
  27. if m != nil {
  28. return m.Property
  29. }
  30. return nil
  31. }
  32. func (m *LocalEntityProto) GetRawProperty() []*pb.Property {
  33. if m != nil {
  34. return m.RawProperty
  35. }
  36. return nil
  37. }
  38. func (m *LocalEntityProto) GetRank() int32 {
  39. if m != nil &amp;&amp; m.Rank != nil {
  40. return *m.Rank
  41. }
  42. return 0
  43. }

huangapple
  • 本文由 发表于 2013年12月21日 03:38:10
  • 转载请务必保留本文链接：https://go.coder-hub.com/20710802.html
  • app-engine-ndb
  • go
  • google-app-engine
  • python
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定