英文:
Mongo Driver for Azure Cosmos - Reads and Rate Limiting
问题
出现以下配置问题,
驱动程序版本:3.12.1,用于Java的mongodb-driver
服务器版本:Azure Cosmos DB的Mongo API版本为3.2(我知道这是古老的版本)
我们运行了一些相当高的读/写负载,可能会因为Cosmos API for Mongo的速率限制而受到影响。在这种情况下,我预期会发生异常。我们执行了相当基本的查询,代码片段看起来类似于
public DatabaseQueryResult find(String collectionName, Map<String, Object> queryData) {
Document toFind = new Document(queryData);
MongoCollection<Document> collection = this.mongoDatabase.getCollection(collectionName);
FindIterable<Document> findResults = collection.find(toFind);
if (findResults != null) {
Document dataFound = findResults.first();
return new DatabaseQueryResult(dataFound.toJson(this.settings));
}
// 其他内容...
}
当在Azure上受到速率限制时,您将会收到如下响应
{
"$err":"Message: {\"Errors\":[\"Request rate is large. More Request Units may be needed, so no changes were made. Please retry this request later. Learn more: http://aka.ms/cosmosdb-error-429\"]}\r\n s",
"code":16500,
"_t":"OKMongoResponse",
"errmsg":"Message: {\"Errors\":[\"Request rate is large. More Request Units may be needed, so no changes were made. Please retry this request later. Learn more: http://aka.ms/cosmosdb-error-429\"]}\r\n",
"ok":0
}
我预期在这里会抛出一个异常 - 但是似乎在较新的驱动程序中并非如此。正在发生的是,
- collection.find返回一个带有上述JSON错误结果的FindIterable,作为第一个文档
- 我们最终返回一个带有JSON错误的DatabaseQueryResult作为查询有效载荷
我不希望这种情况发生 - 我更希望mongo驱动程序在查询操作返回OKMongoResponse且"ok"为0时抛出MongoCommandException/MongoQueryException异常。对于使用CommandProtocol对象的写入操作,这似乎没问题,并且响应会按预期进行验证 - 似乎只有读取操作发生了变化。
比较这两个驱动程序版本,这似乎是读取行为的变化 - 可能是由于在版本3.11中引入的可重试读取。现在响应验证似乎在此部分。
问题: 有没有一种方法可以配置我的Mongo客户端,使得驱动程序在读取操作时验证服务器响应,并且如果接收到ok为0的OKMongoResponse,则抛出异常?
当然,我可以自己验证结果,但如果可能的话,我更希望由驱动程序来执行这项任务。
英文:
Having an issue with the following configuration,
Driver version : 3.12.1, mongodb-driver for Java
Server Version: 3.2 of Mongo API for Azure Cosmos DB (Ancient, I know)
We run some fairly high read/write loads and may hit rate limiting from the Cosmos API for Mongo. In this case, I expect an exception to occur. We're doing pretty vanilla queries, code snippet looks similar to
public DatabaseQueryResult find(String collectionName, Map<String, Object> queryData) {
Document toFind = new Document(queryData);
MongoCollection<Document> collection = this.mongoDatabase.getCollection(collectionName);
FindIterable<Document> findResults = collection.find(toFind);
if (findResults != null) {
Document dataFound = findResults.first();
return new DatabaseQueryResult(dataFound.toJson(this.settings))
}
// other stuff...
}
When rate limited by Azure, you'll receive a response like so
{
"$err":"Message: {\"Errors\":[\"Request rate is large. More Request Units may be needed, so no changes were made. Please retry this request later. Learn more: http://aka.ms/cosmosdb-error-429\"]}\r\n s",
"code":16500,
"_t":"OKMongoResponse",
"errmsg":"Message: {\"Errors\":[\"Request rate is large. More Request Units may be needed, so no changes were made. Please retry this request later. Learn more: http://aka.ms/cosmosdb-error-429\"]}\r\n",
"ok":0
}
I expect an exception to be thrown here - but that doesn't seem to be the case with the later driver. What's happening is,
- collection.find is returning a FindIterable with the JSON error result as above as the first document
- We're eventually returning a DatabaseQueryResult with JSON error as the query payload
I don't want this to happen - I'd much prefer the mongo driver to throw a MongoCommandException/MongoQueryException if a query operation returns an OKMongoResponse where "ok" 0. This seems fine on writes,
which will use a CommandProtocol object and the response is validated as I'd expect - it's just reads that seems to have changed.
Comparing the 2 driver versions, this seems to be a change in read behaviour - perhaps due to retryable reads that were introduced in version 3.11? Response validation now seems to be around <a href="https://github.com/mongodb/mongo-java-driver/blob/b1e2eadf27593a4e90d1f3d697ae4f5ea87b251e/driver-core/src/main/com/mongodb/internal/connection/QueryProtocol.java#L518
">this section.</a>
Q: Is there a way to configure my Mongo client so that the driver will validate server responses on read operations and throw an exception if it receives a OKMongoResponse, and ok == 0?
I can of course validate the results myself, but I'd prefer not to and let the driver do this if possible
答案1
得分: 1
我不确定为什么Mongo更改了这个驱动程序。在Cosmos的一侧有一些可能会有帮助的东西。您可以提出支持票据,并要求他们启用服务器端重试。这将更改Cosmos的行为,使请求排队而不是在请求过多时抛出429错误。
这更多地反映了Mongo在虚拟机或Atlas(也在虚拟机上运行)上运行时的行为,而不是像Cosmos DB这样的多租户服务。
英文:
I'm not sure why Mongo changed this driver. There is something on the Cosmos side which may help. You can raise a support ticket and ask them to turn on server-side retries. This will change the behavior of Cosmos such that requests will queue up rather than throw 429's when there are too many.
This more reflects how Mongo behaves when running on a VM or in Atlas (which also runs on VM's) rather than a multi-tenant service like Cosmos DB.
答案2
得分: 0
使用3.2-3.4服务器时,驱动程序使用在此处描述的find命令(链接:https://github.com/mongodb/specifications/blob/master/source/find_getmore_killcursors_commands.rst),而不是OP_QUERY。
该驱动程序肯定不是在为cosmosdb编写的,所以并不会“返回OKMongoResponse”。
如果您认为存在驱动程序问题,请使用收到的确切线路协议响应和驱动程序给出的确切结果更新问题。
可重试写操作需要会话(cosmosdb进行了广告宣传,但不支持,参见https://stackoverflow.com/questions/63900885/importing-bson-to-cosmosdb-mongodb-api-using-mongorestore/63902985#63902985),通常使用带有3.6+服务器的OP_MSG协议。我不知道如果3.2服务器宣传支持会话,驱动程序会采取什么措施,这不是MongoDB可能出现的组合。
请注意,MongoDB不支持cosmosdb(因此,MongoDB驱动程序也不会正式支持)。
英文:
With 3.2-3.4 servers the drivers use find command described here, not OP_QUERY.
The driver surely is not "returning OKMongoResponse" since it isn't written for cosmosdb.
If you think there is a driver issue, update the question with exact wire protocol response received and the exact result you receive from the driver.
Retryable writes require sessions (which cosmosdb advertises but does not support, see https://stackoverflow.com/questions/63900885/importing-bson-to-cosmosdb-mongodb-api-using-mongorestore/63902985#63902985) and normally use the OP_MSG protocol which come with 3.6+ servers. I don't know what drivers would do if a 3.2 server advertises session support, this isn't a combination that is possible with MongoDB.
Note that MongoDB does not support cosmosdb (and consequently MongoDB drivers don't, officially, either).
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论