create/query in cosmos throws cancelled exception

I'm running v4 functions using .net 6 isolated architecture. implementing my cosmosClient in the injection using following.
var cosmosClient = CosmosClient.CreateAndInitializeAsync(cosmosSettings?.Endpoint, cosmosSettings?.Key, containers).Result;
services.AddSingleton(cosmosClient);

in my repositoryclass constructor I have "

  public CosmosRepo(CosmosClient client, ILogger<CosmosRepo> logger,
        IOptions<MyConfig> config)
    {
        _logger = logger;
        _config = config.Value;
        _container = client.GetContainer(_config.CosmosDb.DatabaseId, _collectionName);
    }

there is only one call to cosmos in this class

  private async Task<ItemDbo> Save(ItemDbo itemDbo)
    {
        if (itemDbo is null)
        {
            throw new ArgumentNullException(nameof(itemDbo));
        }

        try
        {
            var result = await _container.UpsertItemAsync(itemDbo);
            return result.Resource;
        }
        catch (CosmosException e)
        {
            _logger.LogCritical($"itemDbo upsert is failing for entity {itemDbo.Id} " + e.Message);
            throw;
        }
    }

For some reason I am seeing thousands of create/query operations happening where the result is cancelled. My data is coming in but completely dirtying my insights and understanding of the issue.
The upsert is created from a list of tasks.. about 25 tasks per iteration of function.
its just

  List<Task> tasklist; 
  loop 25X
  tasklist.add(Save(itemDbo)).ContinueWith(i =>
            {
                if (!i.IsCompletedSuccessfully)
                {
                    _logger.LogCritical($"Issue upserting the itemDbo data {itemDbo.Id}"); 
                }
            }));

  await Task.WhenAll(updateAndInsertTasks);`

The exception/error seems 100% only on this path, which doesn't make sense
...dbs/cosmosdb/container/colls/itemDbo/docs

In my logs I see the number of operations is enormous of zero ru cost.

My app insights dependencies failures is crazy.

This is especially frustrating as my traces are empty for any of my try/catches.. my exceptions are null as well. I am running about 2.5 million requests per hour.

Please see https://github.com/Azure/azure-cosmos-dotnet-v3/issues/3032#issuecomment-1227434296 for more details.

The details in your App Insight screenshot are not from an UpsertItem call, they are from a Query. Some other part of your code is executing queries (GetQueryItemIterator and ReadNextAsync).

Canceled in Application Insights refers to timed out HTTP requests. This means your application is experiencing connectivity issues on the HTTP layer, the SDK is retrying and eventually succeeding but from the application perspective, it is important that you investigated why the timeouts are happening.

Please refer to https://learn.microsoft.com/azure/cosmos-db/nosql/troubleshoot-dotnet-sdk-request-timeout for a list of potential reasons. If this is running on App Service, it also has some Diagnostics tools that might help narrow down the cause.

One thing is for sure, please avoid using .Result, .GetAwaiter().GetResult() or any place where you are blocking async calls.