What could be causing 'AMQPLinkError' when completing messages in Azure Service Bus?

Azure Service Bus exception when completing messages:
ServiceBusError: The required field 'first' is not set in 'amqp:disposition:list'. Error condition: amqp:invalid-field.

So essentially what im trying to do is read a batch of messages from the servicebus, iterate over the messages to collect the payloads, combine them, and write it all to storage and go back to iterate over the batch of messages again to complete them one by one.

The code works fine in terms of getting the payloads into storage, however once the messages are being completed off, i get this error that i cannot find much information about and it's not very clear on what the missing piece is meant to be or why it's not there... But the underlying exceptions is "AMQPLinkError"

Code block

The code is pretty much similar to what's in the documentation with some small changes.

I've so far tried both sync and async version, tried playing around with batch sizes and timing params, but still get the same issue.

The weirdest thing is that some runs complete fine, however at other times this exception is raised without clear reasoning as to why.

All this code runs in databricks.

Hi @Andrius Vitkauskas,

I work on that team and we are fixing that bug right now. To go around there are two ways to do so:

1. use the uamqp transport by passing in uamqp_transport=True during connection. You will have to pip install uamqp
2. go back to the previous version of service bus, which essentially moves you back to uamqp.

Sorry about the issues. We hope to get this resolved asap.