EF Core:如何防止包含操作增加返回的行数?

huangapple go评论57阅读模式
英文:

EF Core: How do I prevent includes from increasing the number of rows returned?

问题

我有一个ASP.NET Blazor应用程序,使用Telerik网格来显示交易数据。由于网格需要显示大量相关信息(交易来源,包括付款终端的详细信息,租用自哪里等等),查询中有相当多的.Include()子句。

例如,如果网格的页面大小设置为30,则以下查询将从数据库中检索30行...

Data = ctx.Transactions
  .OrderByDescending(a => a.Date)
  .AsQueryable();

然而,如果我添加一些相关数据(未全部显示)...

Data = ctx.Transactions
  .Include(t => t.Card)
  .Include(t => t.Device)
  .ThenInclude(d => d.DistributorRentals)
  .Include(t => t.Device)
  .ThenInclude(d => d.CauseRentals)
  .ThenInclude(cr => cr.CharityAccount)
  .ThenInclude(ca => ca.Charity)
  .OrderByDescending(a => a.Date)
  .AsQueryable();

...那么从数据库中检索的行数(在SQL Server Profiler中可见)会显著增加。

我需要查询返回一个IQueryable<T>,因为网格需要这样做以处理分页和过滤。简化的网格标记如下(为了清晰起见,大多数与相关数据有关的列都已删除)...

<TelerikGrid Data="Data"
             ScrollMode="@GridScrollMode.Virtual"
             Height="800px"
             RowHeight="40"
             PageSize="30">
  <GridColumns>
    <GridColumn Field="Date" Title="Date" Width="210px" />
    <GridColumn Field="Amount" Width="180px" />
  </GridColumns>
</TelerikGrid>

如果我不需要使用IQueryable<T>,我将在枚举之前将数据强制转换为匿名类型,这将减少行数。然而,这将要求我编写大量的代码来处理网格为我执行的过滤和排序。将数据强制转换为具体类型(这需要使用IQueryable<T>)不能在数据库中完成(因为它不知道我的C#类型),因此我必须在转换之前枚举查询,这样就删除了使用IQueryable<T>的所有优势。

有人知道如何解决这个问题吗?我需要包含的数据,但不想从数据库中检索那么多行。

英文:

I have an ASP.NET Blazor app that uses a Telerik grid to display transaction data. As the grid needs to show a lot of related information (source of the transaction, including details about the payment terminal it was made on, who that was rented from and so on), there are quite a lot of .Include() clauses on the query.

For example, if the grid's page size is set to 30, then the following query will pull 30 rows from the database...

    Data = ctx.Transactions
      .OrderByDescending(a =&gt; a.Date)
      .AsQueryable();

However, if I add some related data (not all shown)...

    Data = ctx.Transactions
      .Include(t =&gt; t.Card)
      .Include(t =&gt; t.Device)
      .ThenInclude(d =&gt; d.DistributorRentals)
      .Include(t =&gt; t.Device)
      .ThenInclude(d =&gt; d.CauseRentals)
      .ThenInclude(cr =&gt; cr.CharityAccount)
      .ThenInclude(ca =&gt; ca.Charity)
      .OrderByDescending(a =&gt; a.Date)
      .AsQueryable();

...then the number of rows pulled from the database (as seen in SQL Server Profiler) jumps up significantly.

I need the query to return an IQueryable&lt;T&gt; as the grid requires this so it can handle the paging and filtering. Simplified grid markup is as follows (most columns for related data removed for clarity)...

&lt;TelerikGrid Data=&quot;Data&quot;
             ScrollMode=&quot;@GridScrollMode.Virtual&quot;
             Height=&quot;800px&quot;
             RowHeight=&quot;40&quot;
             PageSize=&quot;30&quot;&gt;
  &lt;GridColumns&gt;
    &lt;GridColumn Field=&quot;Date&quot; Title=&quot;Date&quot; Width=&quot;210px&quot; /&gt;
    &lt;GridColumn Field=&quot;Amount&quot; Width=&quot;180px&quot; /&gt;
  &lt;/GridColumns&gt;
&lt;/TelerikGrid&gt;

I I didn't need to use an IQueryable&lt;T&gt;, I would cast the data to an anonymous type before enumerating, which would cut down the number of rows. However, that would require me to write an enormous amount of code to handle the filtering and sorting that the grid does for me. Casting the data into a concrete type (which would be required to use IQueryable&lt;T&gt;) can't be done in the database (as it won't know about my C# type), so I'd have to enumerate the query before casting, which then removes all the advantages of using an IQueryable&lt;T&gt;.

Anyone any idea how I can solve this? I need the included data, but don't want so many rows puled back from the database.

答案1

得分: 2

这是笛卡尔积,每当你在查询中连接表格时就会发生。这不仅增加了结果的深度(行数),还增加了结果的广度(列数)。这导致了大量的数据块。

在EF Core中可用的一个选项是使用拆分查询选项,但是在使用分页时可能会遇到一些显著的限制。这个选项将查询拆分成每个实体/表格执行一个查询,但这可能会导致在确保排序结果和关联正确链接的问题上出现问题。它解决了深度问题,但仍然会导致大量数据返回,因为您要获取每个包含的实体的全部数据。

在处理搜索结果等情况时,减轻这个问题的最佳方法是使用视图模型进行投影。您实际上需要哪些实体的字段?可能会出现一些情况,您可能需要计数,或检查行是否存在,或只返回一个或前几个相关的行。所有这些都可以缩减为一个视图模型或视图模型图的投影,它比用于填充它的表格的总和要小得多。使用Select或Automapper的ProjectTo来仅填充用于结果的所需详细信息的视图模型。

然后,当您想要打开/处理特定的交易时,可以通过ID从数据库中获取包含所有详细信息的Tx,将单个Tx图提取出来。

英文:

That is the Cartesian Product which will happen whenever you Join tables in a query. Not only does this increase the depth of the results, (# of rows) it also increases the breadth of the results. (# of columns) This leads to huge blocks of data.

One option available in EF Core is to use the Split Query option, however there are some significant limitations to this option that you will possibly run into when using pagination. This options splits the queries up so that it will execute a query per entity/table, but this can lead to problems around ensuring that sorted results and associations are linked up correctly. It solves the depth issue, but still results in a lot of data coming back as you are fetching every Included entity in its entirety.

The best way to mitigate this issue when it comes to things like search results is to use projection with a view model. What fields from what entities do you actually need? There may be situations where you might want a count, or check for the existence of a row, or just return one or the top few associated rows. All of this can be trimmed down into a projection of a view model or graph of view models which is much smaller than the sum of the tables used to populate it. Use Select or Automapper's ProjectTo to populate a view model with just the details needed for the results.

Then when you want to open/work on a particular Transaction you can fetch that Tx with all of the details included, pulling the single Tx graph from the database by ID.

huangapple
  • 本文由 发表于 2023年6月22日 03:16:11
  • 转载请务必保留本文链接:https://go.coder-hub.com/76526467.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定