Google App Engine Golang datastore.Query.GetAll not working locally

I am new to Google App Engine and meet some problems with datastore.

I wrote a test GaeDatastore_test.go to test datastore.Query.GetAll method see below

package persist

import (
	"fmt"
	"testing"

	"appengine/aetest"
	"appengine/datastore"
)

type Mock struct {
	Name string
}

func TestAll(t *testing.T) {
	ctx, _ := aetest.NewContext(nil)
	defer ctx.Close()
	d := &Mock{
		"hello",
	}
	fmt.Println(datastore.Put(ctx, datastore.NewIncompleteKey(ctx, "test", nil), d))
	fmt.Println(datastore.Put(ctx, datastore.NewIncompleteKey(ctx, "test", nil), d))
	fmt.Println(datastore.Put(ctx, datastore.NewIncompleteKey(ctx, "test", nil), d))
	q := datastore.NewQuery("test")
	var ms []Mock
	q.GetAll(ctx, &ms)
	fmt.Printf("%#v", ms)
}

When I run the test file using

> goapp test

it does not return the 3 entities I have stored.
I could see key returned after calling datastore.Put and I could use datastore.Get to retrieve them using the key. But the 'ms' is always nil.

Then I tried to initialise ms using

> make([]Mock,10)

But it does not make much difference and the data still not coming through.

I checked query.go source file . Internally it uses loadEntity which is the same method used by Get / GetMulti and use append to push elements to the slice. It is quite weird that the ms is nil.

The client is connecting to a Python server simulating GAE env. Could some one help?

I believe it to be because you are doing a cross group query and the write has not been applied to the data store yet. The dev server simulates production-like write visibility. When you are not in a transaction or entity group, queries will not show results right away.

From the docs,

> The write operation returns immediately after the Commit phase and the Apply phase then takes place asynchronously.

and,

> ... Apply is rolled forward to completion when one of the following
> occurs:
>
> Periodic Datastore sweeps check for uncompleted Commit jobs and apply
> them. Certain operations (get, put, delete, and ancestor queries) that
> use the affected entity group cause any changes that have been
> committed but not yet applied to be completed before proceeding with
> the new operation.

and,

> Because Datastore gets and ancestor queries apply any outstanding
> modifications before executing, these operations always see a
> consistent view of all previous successful transactions. This means
> that a get operation (looking up an updated entity by its key) is
> guaranteed to see the latest version of that entity.

For the source of these quotes and much more detail on writes and data visibility rules see: https://developers.google.com/appengine/docs/go/datastore/#Go_Datastore_writes_and_data_visibility

Here is a modified version of your code that performs a Get on the returned key after each put. A Get() forces that key to be applied. This ensure that your GetAll query will return all your test entities.

package persist

import (
        "fmt"
        "testing"

        "appengine/aetest"
        "appengine/datastore"
)

type Mock struct {
        Name string
}

func TestAll(t *testing.T) {
        ctx, _ := aetest.NewContext(nil)
        defer ctx.Close()
        d := &Mock{
                "hello",
        }

        for i := 0; i < 3; i++ {
                k, err := datastore.Put(ctx, datastore.NewIncompleteKey(ctx, "test", nil), d)
                fmt.Println(k, err)
                datastore.Get(ctx, k, nil)
        }
        q := datastore.NewQuery("test")
        var ms []Mock
        q.GetAll(ctx, &ms)
        fmt.Printf("%#v", ms)
}