"new style" google pubsub golang functions not working right

I'm trying to use the Go pubsub library against the local emulated pubsub server. I'm finding that the "old style" (deprecated) functions (e.g. CreateSub and PullWait) work find, but the "new style" API (e.g. Iterators and SubscriptionHandles) does not work as expected.

I've written two different unit tests that both test the same sequence of actions, one using the "new style" API and one using the "old style" API.

The sequence is:

• create a subscription
• fail to pull any messages (since none available)
• publish a message
• pull that message, but don't ACK it
• lastly pull it again which should take 10s since the message ACK timeout has to expire first

https://gist.github.com/ianrose14/db6ecd9ccb6c84c8b36bf49d93b11bfb

The test using the old-style API works just as I would expect:

=== RUN   TestPubSubRereadLegacyForDemo
--- PASS: TestPubSubRereadLegacyForDemo (10.32s)
	pubsubintg_test.go:217: PullWait returned in 21.64236ms (expected 0)
	pubsubintg_test.go:228: PullWait returned in 10.048119558s (expected 10s)
PASS

Whereas the test using the new-style API works unreliably. Sometimes things work as expected:

=== RUN   TestPubSubRereadForDemo
--- PASS: TestPubSubRereadForDemo (11.38s)
	pubsubintg_test.go:149: iter.Next() returned in 17.686701ms (expected 0)
	pubsubintg_test.go:171: iter.Next() returned in 10.059492646s (expected 10s)
PASS

But sometimes I find that iter.Stop() doesn't return promptly as it should (and note how the second iter.Next too way longer than it should):

=== RUN   TestPubSubRereadForDemo
--- FAIL: TestPubSubRereadForDemo (23.87s)
	pubsubintg_test.go:149: iter.Next() returned in 7.3284ms (expected 0)
	pubsubintg_test.go:171: iter.Next() returned in 20.074994835s (expected 10s)
	pubsubintg_test.go:183: iter.Stop() took too long (2.475055901s)
FAIL

And other times I find that the first Pull after publishing the message takes too long (it should be near instant):

=== RUN   TestPubSubRereadForDemo
--- FAIL: TestPubSubRereadForDemo (6.32s)
	pubsubintg_test.go:147: failed to pull message from iterator: context deadline exceeded
FAIL

Any ideas? Are there any working examples using the new-style API? Unfortunately, the Go starter project here uses the old, deprecated API.

(Note: It looks like the line numbers in your example output don't match the code that you've linked.)

> But sometimes I find that iter.Stop() doesn't return promptly as it should

A few changes have landed recently which fix the excessive delay when calling iter.Stop. It should now return promptly if all messages have been acked. Try syncing and testing it out again.

> (and note how the second iter.Next too way longer than it should):

In you code that uses the new API, you first do a pull from an empty subscription, using a context with a 1s deadline. Let's call this "Pull request A". Although the underlying http request is cancelled, it seems that the connection is not being closed in any way that the server respects. So, as far as the server is concerned, "A" is still pending. Immediately after publishing, you make a new pull request, let's call that "B". After a message is returned via pull request B, you leave the message unacked, and make another pull request, "C".

Now, when you publish the message, the server will deliver it to either "A" or "B". If it delivers it to "A" first, you will see the first pull exceeding the 5s context deadline. If it is published to "B" first, you will see the first pull returning quickly, as expected. After the message is published to "B", and left unacked, the server will redeliver it to "A", or "C". If it picks "A" first, then you will end up with the second pull taking longer than expected. If it picks "C", then you will see both first and second pulls taking as long as you expect.

If you don't do that initial pull from the empty subscription, you should see your test behave as you expect.

Note: You don't see any of this when you use the old API because you're not doing the extra "pull from empty subscription" request with the old API (presumably because it doesn't properly support a cancellable context).

Aside: if you want to leave a message unacked, you should call Message.Done(false).