CosmosClient：ReadItemAsync成功，GetItemLinqQueryable失败。

huangapple
go评论94阅读模式
英文:

CosmosClient : ReadItemAsync succeeds, GetItemLinqQueryable fails

问题

Using .Net 6, Azure.Cosmos 3.33

============= Some extra context, only to be thorough ==============

问题实际上是关于在 CosmosDb 3 中查询项的几种方式，但为了避免误解，这里有关于底层基础结构的完整免责声明：

  1. public interface IWithKey<out TK>
  2. {
  3.    public TK Id { get; }
  4. }
  6. public interface IWithPartitionKey<out TK>
  7. {
  8.    public TK PartitionKey { get; }
  9. }
  11. public interface ICosmosDbEntity<out TK, PK> : IWithKey<TK>, IWithPartitionKey<PK> where TK : struct
  12. {
  13. }
  15. public abstract class CosmosDbEntity<TK, PK> : ICosmosDbEntity<TK, PK> where TK : struct
  16. {
  17.    [JsonPropertyName("id")] public TK Id { get; protected set; }
  19.    [JsonIgnore] public virtual PK PartitionKey { get; } = default!;
  21.    protected CosmosDbEntity(TK id)
  22.    {
  23.       Id = id;
  24.    }
  25. }

我的实际数据类：

  1. public class MyType : CosmosDbEntity<Guid, PartitionKey>
  2. {
  3.    [JsonIgnore]
  4.    public override PartitionKey PartitionKey => SomeGuid.AsPartitionKey();
  6.    public Guid SomeGuid { get; }
  8.    public MyType(Guid id, Guid someGuid) : base(id)
  9.    {
  10.       SomeGuid = someGuid;
  11.    }
  12. }

自定义序列化器类，旨在使用 system.Text.Json 而不是 Newtonsoft 的 Json.Net：

  1. public class CosmosNetSerializer : CosmosSerializer
  2. {
  3.    private readonly JsonSerializerOptions? _serializerOptions;
  5.    public CosmosNetSerializer() => _serializerOptions = null;
  7.    public CosmosNetSerializer(JsonSerializerOptions serializerOptions) =>
  8.       this._serializerOptions = serializerOptions;
  10.    public override T FromStream<T>(Stream stream)
  11.    {
  12.       using (stream)
  13.       {
  14.          if (typeof(Stream).IsAssignableFrom(typeof(T)))
  15.          {
  16.             return (T)(object)stream;
  17.          }
  19.          return JsonSerializer.DeserializeAsync<T>(stream, _serializerOptions).GetAwaiter().GetResult();
  20.       }
  21.    }
  23.    public override Stream ToStream<T>(T input)
  24.    {
  25.       var outputStream = new MemoryStream();
  27.       JsonSerializer.SerializeAsync<T>(outputStream, input, _serializerOptions).GetAwaiter().GetResult();
  29.       outputStream.Position = 0;
  30.       return outputStream;
  31.    }
  32. }

以及 Cosmos 客户端的实例化方式：

  1. var options = new CosmosClientOptions
  2. {
  3.    ConnectionMode = //...,
  5.    // JsonSerializerDefaults.Web 通常使字段比较小写
  6.    Serializer = new CosmosNetSerializer(new(JsonSerializerDefaults.Web))
  7. };
  9. // Cosmos 版本 3.33
  10. return Microsoft.Azure.Cosmos.CosmosClient
  11.    .CreateAndInitializeAsync(connectionStrings.CosmosDb,
  12.       credentials, listOfContainers, options)
  13.    .GetAwaiter()
  14.    .GetResult();

============= end of context ==============

现在，考虑一下在我的 Azure Cosmos db 中查询项目的几种方式：

  1. Guid id = ...;
  2. string partitionKey = ...;
  4. **1. ReadItemAsync (with partition key) => OK**
  6. var response = container.ReadItemAsync<MyType>(id.ToString(),
  7.    new PartitionKey(partitionKey)).Result;
  9. var item = response?.Resource;
  10. Assert.NotNull(item);
  12. **2. GetItemLinqQueryable (without partition key) => NOT OK**
  14. var item = container.GetItemLinqQueryable<MyType>(true)
  15.    .Where(m => m.Id == id)
  16.    .AsEnumerable()
  17.    .FirstOrDefault();
  19. Assert.NotNull(item);
  21. **3. GetItemLinqQueryable (without 'Where') + DeleteItemAsync (with partition key) => OK**
  23. var items = container.GetItemLinqQueryable<MyType>(true)
  24.   .ToList();
  26. foreach (var item in items)
  27. {
  28.    container.DeleteItemAsync<MyType>(item.Id.ToString(), new PartitionKey(partitionKey)).Wait();
  29. }
  31. **4. With iterator (without partition key) => OK**
  33. var items = container.GetItemLinqQueryable<MyType>(true)
  34.    .Where(m => m.Id == input.Id) // <-- the clause is still here!
  35.    .ToFeedIterator();
  37. while (items.HasMoreResults)
  38. {
  39.    var item = items.ReadNextAsync().Result;
  40.    Assert.NotNull(item);
  41. }
  43. **5. : GetItemLinqQueryable (with partition key) => NOT OK**
  45. var options = new QueryRequestOptions
  46. {
  47.    PartitionKey = new PartitionKey(partitionKey)
  48. };
  50. var item = container.GetItemLinqQueryable<MyType>(
  51.    true, 
  52.    null, 
  53.    options // <-- there IS a partition key!
  54. )
  55.    .Where(m => m.Id == input.Id);
  56.    .FirstOrDefault();
  58. Assert.NotNull(item);
  60. **6. GetItemQueryIterator (without partition key) => OK**
  62. var query = container.GetItemQueryIterator<MyType>(
  63.    $"select * from t where t.id='{itemId.ToString()}'");
  65. while (query.HasMoreResults)
  66. {
  67.    var items = await query.ReadNextAsync();
  68.    var item = items.FirstOrDefault();
  69. }

问题：

#1, #3, #4, #6 可以正常工作，但 #2 和 #5 失败。在 #2 和 #5 中，item 为 null。
为什么方法 #2 或 #5 找不到项目？

故障排除

起初，我以为这可能是由于我的自定义 CosmosSerializer（也许 id 没有正确比较 -- 尽管我的序列化器没有触及它，它只使用另一个特殊字段）导致的，但 #3 似乎证明了这不是问题，因为它也可以使用 id。

显然，在查询之前，我总是检查项目是否存在。我设置了断点并查看 CosmosDb 容器，甚至检查 Guid 是否正确。

在场景 #5 中尝试使用 PartitionKey.None ... 没有帮助

我尝试在 Id 的声明上方添加 [JsonPropertyName("id")]，以确保这不是大小写问题。但是 场景 #4 反驳了大小写问题！（.Where(...) 在查询中添加了 WHERE Id=...，并且 仍然有效）

英文:

Using .Net 6, Azure.Cosmos 3.33

============= Some extra context, only to be thorough ==============

the question is really about the several ways of querying items in CosmosDb 3, but to avoid misunderstandings here is a full disclaimer of the underlying infrastructure :

  1.    public interface IWithKey&lt;out TK&gt;
  2.    {
  3.       public TK Id { get; }
  4.    }
  6.    public interface IWithPartitionKey&lt;out TK&gt;
  7.    {
  8.       public TK PartitionKey { get; }
  9.    }
  11.    public interface ICosmosDbEntity&lt;out TK, PK&gt; : IWithKey&lt;TK&gt;, IWithPartitionKey&lt;PK&gt; where TK : struct
  12.    {
  13.    }
  15.    public abstract class CosmosDbEntity&lt;TK, PK&gt; : ICosmosDbEntity&lt;TK, PK&gt; where TK : struct
  16.    {
  17.       [JsonPropertyName(&quot;id&quot;)] public TK Id { get; protected set; }
  19.       [JsonIgnore] public virtual PK PartitionKey { get; } = default!;
  21.       protected CosmosDbEntity(TK id)
  22.       {
  23.          Id = id;
  24.       }
  25.    }

My actual data class :

  1. public class MyType : CosmosDbEntity&lt;Guid, PartitionKey&gt;
  2. {
  3.    [JsonIgnore]
  4.    //[Newtonsoft.Json.JsonIgnore]
  5.    public override PartitionKey PartitionKey =&gt; SomeGuid.AsPartitionKey();
  7.    public Guid SomeGuid { get; }
  10.    public MyType(Guid id, Guid someGuid) : base(id)
  11.    {
  12.       SomeGuid = someGuid;
  13.    }
  14. }

The custom serializer class, designed to use system.Text.Json instead of Newtonsoft's Json.Net :

  1.    public class CosmosNetSerializer : CosmosSerializer
  2.    {
  3.       private readonly JsonSerializerOptions? _serializerOptions;
  5.       public CosmosNetSerializer() =&gt; _serializerOptions = null;
  7.       public CosmosNetSerializer(JsonSerializerOptions serializerOptions) =&gt;
  8.          this._serializerOptions = serializerOptions;
  10.       public override T FromStream&lt;T&gt;(Stream stream)
  11.       {
  12.          using (stream)
  13.          {
  14.             if (typeof(Stream).IsAssignableFrom(typeof(T)))
  15.             {
  16.                return (T)(object)stream;
  17.             }
  19.             return JsonSerializer.DeserializeAsync&lt;T&gt;(stream, _serializerOptions).GetAwaiter().GetResult();
  20.          }
  21.       }
  23.       public override Stream ToStream&lt;T&gt;(T input)
  24.       {
  25.          var outputStream = new MemoryStream();
  27.          JsonSerializer.SerializeAsync&lt;T&gt;(outputStream, input, _serializerOptions).GetAwaiter().GetResult();
  29.          outputStream.Position = 0;
  30.          return outputStream;
  31.       }
  32.    }

And how the Cosmos client gets instantiated :

  1.  var options = new CosmosClientOptions
  2.  {
  3.     ConnectionMode = //...,
  5.     // JsonSerializerDefaults.Web normally makes fields comparison camel-case
  6.     Serializer = new CosmosNetSerializer(new(JsonSerializerDefaults.Web))
  7.  };
  9.  // Cosmos version 3.33
  10.  return Microsoft.Azure.Cosmos.CosmosClient
  11.     .CreateAndInitializeAsync(connectionStrings.CosmosDb,
  12.        credentials, listOfContainers, options)
  13.     .GetAwaiter()
  14.     .GetResult();

============= end of context ==============

Now, consider those several ways of querying items in my Azure Cosmos db :

  1.      Guid id = ...;
  2.      string partitionKey = ...;

1. ReadItemAsync (with partition key) => OK

  1.      var response = container.ReadItemAsync&lt;MyType&gt;(id.ToString(),
  2.         new PartitionKey(partitionKey)).Result;
  4.      var item = response?.Resource;
  5.      Assert.NotNull(item);

2. GetItemLinqQueryable (without partition key) => NOT OK

  1.      var item = container.GetItemLinqQueryable&lt;MyType&gt;(true)
  2.         .Where(m =&gt; m.Id == id)
  3.         .AsEnumerable()
  4.         .FirstOrDefault();
  6.      Assert.NotNull(item);

3. GetItemLinqQueryable (without 'Where') + DeleteItemAsync (with partition key) => OK

  1.         var items = container.GetItemLinqQueryable&lt;MyType&gt;(true)
  2.           .ToList();
  4.         foreach (var item in items)
  5.         {
  6.            container.DeleteItemAsync&lt;MyType&gt;(item.Id.ToString(), new PartitionKey(partitionKey)).Wait();
  7.         }

4. With iterator (without partition key) => OK

  1.      var items = container.GetItemLinqQueryable&lt;MyType&gt;(true)
  2.         .Where(m =&gt; m.Id == input.Id) // &lt;-- the clause is still here!
  3.         .ToFeedIterator();
  5.      while (items.HasMoreResults)
  6.      {
  7.         var item = items.ReadNextAsync().Result;
  8.         Assert.NotNull(item);
  9.      }

5. : GetItemLinqQueryable (with partition key) => NOT OK

  1.      var options = new QueryRequestOptions
  2.      {
  3.         PartitionKey = new PartitionKey(partitionKey)
  4.      };
  6.      var item = container.GetItemLinqQueryable&lt;MyType&gt;(
  7.         true, 
  8.         null, 
  9.         options // &lt;-- there IS a partition key!
  10.      )
  11.         .Where(m =&gt; m.Id == input.Id);
  12.         .FirstOrDefault();
  14.      Assert.NotNull(item);

6. GetItemQueryIterator (without partition key) => OK

  1.      var query = container.GetItemQueryIterator&lt;MyType&gt;(
  2.         $&quot;select * from t where t.id=&#39;{itemId.ToString()}&#39;&quot;);
  4.      while (query.HasMoreResults)
  5.      {
  6.          var items = await query.ReadNextAsync();
  7.          var item = items.FirstOrDefault();
  8.      }

Problem :

#1, #3, #4, #6 work, but #2 and #5 fail. In #2 and #5, item is null.
Why can method #2 or #5 not find the item?

Troubleshooting

At first I thought it might be cause by my custom CosmosSerializer (maybe the id was not compared properly -- despite the fact that my serializer does not touch it, it only works with another special field) but #3 seems to prove but that's not it, as it works with te id too.

Obviously I always checked that the item was present before querying it. I set a breakpoint and go see the CosmosDb container, and even check that the Guid is correct.

I tried with PartitionKey.None in Scenario #5 ... didn't help

I tried adding [JsonPropertyName(&quot;id&quot;)] above the declaration of Id, to be sure that it wasn't a casing issue. But Scenario #4 disproved that casing is the issue anyways! (the .Where(...) adds a WHERE Id=... with a capital 'i' in the query and it still works)

答案1

得分: 1

The solution/answer has been given by the devs of the Cosmos SDK, directly on their forums.

Here is what they wrote:

  • In regards to the 2nd SO example:

Currently, SDK doesn't support custom serializers in GetItemLinqQueryable.

If you invoke container.GetItemLinqQueryable&lt;MyType&gt;(true).Where(m =&gt; m.Id == id).Expression then you can see translated to SQL query.

It translates to: SELECT VALUE root FROM root WHERE (root[&quot;Id&quot;] = &lt;some id&gt;).

As you can see, it uses the original property name (Id with a capital 'i'), not the custom name from JsonPropertyName attribute (id in lowercase). It's a known issue, and the SDK team is working on this.

See related LINQ queries doesn't use custom CosmosSerializer #2685 for more information.

  • In regards to the 5th SO example:

This part of code: .Where(m =&gt; m.Id == input.Id).FirstOrDefault(); raises Microsoft.Azure.Cosmos.Linq.DocumentQueryException: 'Method 'FirstOrDefault' is not supported.'

Currently, SDK does not directly support FirstOrDefault() method on the GetItemLinqQueryable query.

See LINQ to SQL translation - Azure Cosmos DB for NoSQL | Microsoft Learn for more information.

英文:

The solution/answer has been given by the devs of the Cosmos SDK, directly on their forums.

Here is what they wrote :

  • In regards to the 2nd SO example:

Currently, SDK doesn't support custom serializers in GetItemLinqQueryable .

If you invoke container.GetItemLinqQueryable&lt;MyType&gt;(true).Where(m =&gt; m.Id == id).Expression then you can see translated to SQL query.

It translates to : SELECT VALUE root FROM root WHERE (root[&quot;Id&quot;] = &lt;some id&gt;).

As you can see, it uses original the property name (Id with a capital 'i'), not the custom name from JsonPropertyName attribute (id in lowercase). It's a known issue and the SDK team working on this.

See related LINQ queries doesn't use custom CosmosSerializer #2685 for more information.

  • In regards to the 5th SO example:

This part of code:
.Where(m =&gt; m.Id == input.Id).FirstOrDefault(); raises Microsoft.Azure.Cosmos.Linq.DocumentQueryException : &#39;Method &#39;FirstOrDefault&#39; is not supported.

Currently, SDK does not directly support FirstOrDefault() method on the GetItemLinqQueryable query.

See LINQ to SQL translation - Azure Cosmos DB for NoSQL | Microsoft Learn for more information.

huangapple
  • 本文由 发表于 2023年5月11日 17:21:44
  • 转载请务必保留本文链接：https://go.coder-hub.com/76226063.html
  • c#
  • cosmosclient
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定