Google App Engine Datastore sharding in Go

I have made a shard in GAE and only sometimes, not always got this message after running the sharding code.

(datastore_v3: BAD_REQUEST): Key path element must not be incomplete: [ResumeShard: ]

The code:

//Sharding
//Getting shard ID
rand.Seed(time.Now().UnixNano())
shardId := rand.Int63n(5)
resumeShardKey := datastore.NewKey(*pC, "ResumeShard", "", accountKey.IntID()+shardId, nil)

var resumeShard param.ResumeShard

if err = datastore.Get(*pC, resumeShardKey, &resumeShard); err != nil {

	if err == datastore.ErrNoSuchEntity {
		resumeShard = param.ResumeShard{}
		resumeShard.Counter = 1
	} else {
		log.Println(err.Error())
	}

} else {
	resumeShard.Counter += 1
}

*pC is a pointer to the context from appengine.

accountKey is a datastore Key unique to each user.

This error shows up randomly, like 3 times in 10 requests. I am wondering because I have to use datastore.NewKey(..) to create keys for shards, how can I get the complete key that I will receive after calling Put(..) since this is sharding and the example on GAE doc uses datastore.NewKey(..) as well.

Please suggest.

You get an error stating that the key you provide is incomplete. This is because you specify an empty key name ("") (stringID), and accountKey.IntID() plus a random number (from the range 0..4 - both inclusive) as the numeric key ID (intID). Note that if your accountKey.IntID() is not initialized, it will be 0 and the random number may also be 0 in which case a 0 is used as the intID.

And quoting from the doc of datastore.Key:

> Either one or both of stringID and intID must be zero. If both are zero, the key returned is incomplete.

The chance of getting 0 for the intID is 1 out of 5 which is 20% (this is close to your reported 3 out of 10).

A value of 0 for intID is invalid, it has to be a non-zero value as 0 is a special value just like the empty string "" for the stringID, both which are used to indicate that the other property is used to identify the entity; or - if both are the zero value - to indicate that the key is an incomplete key (and so system assigned random intID is wanted).

Please note that this solution is bad as you use keys with relative numeric ID (intID), relative compared to the numeric ID of the account key (accountKey.IntID()). System-assigned keys will be randomly distributed, but there is a chance that 2 account keys may be close to each other (e.g. one is the other +1). In such case your shard keys would overlap!

If you use datastore.Put() to save an entity, the system assigned random key is the return value of Put():

func Put(c context.Context, key *Key, src interface{}) (*Key, error)