Not getting results with datastore.GetAll in golang

I have two functions: one which writes entities to datastore and the other is suppose to retrieve them. When I use datastore.GetAll() function in my retrieval it returns no results. I do have a test that verifies that the writing seems to be working correctly. Any ideas as to why retrieval isn't working?

Here is the application code:

package tracker

import (
   "fmt"
    
   "appengine"
   "appengine/datastore"
)

type User struct {
    Email string
}

func createUser(ctx appengine.Context, email string) (*datastore.Key, error) {
    u := &User{
        Email: email,
        }
    incompleteKey := datastore.NewIncompleteKey(ctx, "User", nil)
    key, err := datastore.Put(ctx, incompleteKey, u)
    if err != nil {
        return key, err
    }

    return key, nil
}

func getUser(ctx appengine.Context, email string) (u *User, e error) {
    users := []User{}
    q := datastore.NewQuery("User").Filter("Email", email)
    keys, err := q.GetAll(ctx, &users)
    if err != nil {
        return nil, err
    }
    fmt.Printf("KEYS: %v", keys)
    return &users[0], nil
}

Here is the test code:

package tracker

import (
    "fmt"
    "testing"
    
    "appengine/datastore"
    "appengine/aetest"
)

// This test is passing.
func TestCreateUser(t *testing.T) {
    ctx, err := aetest.NewContext(nil)
    if err != nil {
        t.Fatal(err)
    }
    defer ctx.Close()

    email := "testing@testing.go"
    newKey, err := createUser(ctx, email)
    if err != nil {
        t.Errorf("Failed to create a new user: %v", err)
    }

    u := User{}
    datastore.Get(ctx, newKey, &u)

    if u.Email != email {
        t.Errorf("Expected email to be %s, found %v.", email, u.Email)
    }
}

func TestGetUser(t *testing.T) {
    ctx, err := aetest.NewContext(nil)
    if err != nil {
        t.Fatal(err)
    }
    defer ctx.Close()

    email := "testing@testing.go"
    newKey, err := createUser(ctx, email)
    fmt.Printf("key, %v; ", newKey)
    u, err := getUser(ctx, newKey)
    fmt.Printf("user, %v; error: %s", u, err)

    if u.Email != email {
        t.Error("Expected email to be %s, found %v.", email, u.Email)
    }
}

datastore.GetAll() doesn't return result to you because with that query eventual consistency applies. The SDK simulates the eventual consistency and doesn't return you the newly saved entities immediately.

But in your TestCreateUser() method when you use datastore.Get(), that will return you the entity even if it was newly ("just now") saved, because it's a lookup by a key, and they are strongly consistent.

What happens behind the scenes is that when you call datastore.Put(), the entity data (values of properties) is saved and its key is indexed, then datastore.Put() returns, and indices of other properties and composite indices are updated asynchronously "in the background". So if you try to execute a query that uses an index to find/list entities (you were querying by the Email property), that query will not see (will not include) new entities until they are properly indexed. When you do a datastore.Get(), that loads the entity by its key and not by other (non-key property or composite) indices, so getting an entity by key will see the new entity "immediately" (after datastore.Put() has returned).

If you want to test this in local environment, you may provide StronglyConsistentDatastore option when creating the instance that will be used to create a new context, like this:

inst, err := aetest.NewInstance(&aetest.Options{StronglyConsistentDatastore: true})
if err != nil {
	t.Fatalf("Failed to create instance: %v", err)
}
defer inst.Close()

req, err := inst.NewRequest("GET", "/", nil)
if err != nil {
	t.Fatalf("Failed to create req: %v", err)
}
ctx := appengine.NewContext(req)

Also note that if you use a sleep (e.g. time.Sleep(time.Millisecond * 500)), datastore.GetAll() will also return the new entity, but the above option is the proper way to test this.

There are numerous similar questions (+answers), read them for more details:

https://stackoverflow.com/questions/28590519/google-app-engine-datastore-testing-queries-fails

https://stackoverflow.com/questions/29228712/how-to-filter-a-gae-query

https://stackoverflow.com/questions/30503343/google-app-engine-datastore-query-with-cursor-wont-iterate-all-items