如何在不借助客户端评估的情况下对一个包含分隔符的列进行LINQ-SQL查询。

huangapple go评论68阅读模式
英文:

How to do a LINQ-SQL query on a delimited column, without resorting to client-side evaluation

问题

我试图查询一个视图,该视图具有一个名为"Operator"的列。示例数据如下:

  • Ed
  • Edward
  • Ed;Adam
  • [null]
  • Bob; Edward
  • Bob; Ed; Tom

给定用户输入的 List<string>,假设是 "Ed"、"Peter"、"Mark",我想要返回完全包含其中一个用户输入的行,例如行 #1、#3 和 #6。

一个等效的SQL查询,可以返回我想要的结果如下:

SELECT * 
FROM myview 
WHERE "Operator" LIKE 'Ed' OR "Operator" LIKE 'Ed;%' OR "Operator" LIKE '%;Ed' OR "Operator" LIKE '%;Ed;%' 
--重复为"Peter"和"Mark"等

我尝试了第一种方法:

var query = IQueryable<Myview> farms;
var List<string> operators = new List<string> { "Ed","Peter", "Mark" };

var filteredFarms = query.Where(t => t.Operator != null && operators.Any(op => t.Operator.Split(';').Any(o => o == op)));

这不出所料地导致了 “The LINQ expression ... could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'.” 错误。

我还尝试了第二种方法:

var filteredFarms = query.Where(t => t.Operator != null && operators.Any(op => EF.Functions.Like(t.Operator, "%"+op+";%")));

这是一个手动构建"like"查询的测试,但结果出现了相同的错误。

是否有办法在LINQ中执行此操作,而无需客户端评估?您可以假设";"仅用作分隔符。

英文:

I am trying to query a view, which has an "Operator" column. Sample data:

  • Ed
  • Edward
  • Ed;Adam
  • [null]
  • Bob; Edward
  • Bob; Ed; Tom

Given a user input of a List&lt;string&gt;, let's say "Ed","Peter", "Mark", I want to return rows that fully contain one of the user inputs, in my example rows #1, #3, and #6.

An equivalent SQL query that returns what I want would be:

SELECT * 
FROM myview 
WHERE &quot;Operator&quot; LIKE &#39;Ed&#39; OR &quot;Operator&quot; LIKE &#39;Ed;%&#39; OR &quot;Operator&quot; LIKE &#39;%;Ed&#39; OR &quot;Operator&quot; LIKE &#39;%;Ed;%&#39; 
--repeated for &quot;Peter&quot; and &quot;Mark&quot; etc

I've tried #1:

var query = IQueryable&lt;Myview&gt; farms;
var List&lt;string&gt; operators = new List&lt;string&gt; { &quot;Ed&quot;,&quot;Peter&quot;, &quot;Mark&quot; };

var filteredFarms = query.Where(t =&gt; t.Operator != null &amp;&amp; operators.Any(op =&gt; t.Operator.Split(&#39;;&#39;).Any(o =&gt; o == op)));

Which unsurprisingly gives &quot;The LINQ expression ... could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to &#39;AsEnumerable&#39;, &#39;AsAsyncEnumerable&#39;, &#39;ToList&#39;, or &#39;ToListAsync&#39;.&quot;

I also tried #2:

var filteredFarms = query.Where(t =&gt; t.Operator != null &amp;&amp; operators.Any(op =&gt; EF.Functions.Like(t.Operator, &quot;&quot;+op+&quot;;%&quot;)));

As a test to see if I could build the "like" query manually, but that failed with the same error.

Is there any way I can do this using LINQ without client-side evaluation? You may assume that ";" is only used as a separator.

答案1

得分: 1

在关系数据库中,并不存在被限定的列。这些只是字符串值。它们无法被查询或索引。尝试在这种列的中间找到一个字符串将迫使数据库扫描整个表。

在 PostgreSQL 中不需要这样的值。PostgreSQL 拥有数组类型,可以使用数组操作符进行搜索

CREATE TABLE Farms (
    ...
    Operators  text ARRAY,
    ...
);
CREATE INDEX idx_farm on "Farms" USING GIN ("Operators");
...

SELECT *
FROM Farms
WHERE Operators @> ARRAY['Joe']

NpgSQL 支持数组映射和查询。这意味着您可以轻松编写 LINQ 查询,检查数组中是否包含一个元素:

var query = dbContext.Farms.Where(farm => farm.Operators.Contains("Joe"));

数组字段可以使用GIN 索引进行索引化,提供快速查询。但只有少数几个数组操作符能够利用索引,这就是我使用 @> 而不是 ANY 的原因。

在其他数据库中,您可以使用它们的 JSON 或 XML 支持来存储、查询和有时索引值,而不是使用字符串操作。

英文:

There's no such thing as delimited columns in relational databases. These are just string values. They can't be queried or indexed. Trying to find a string in the middle of such a column will force the database to scan the entire table.

There's no need for such values in PostgreSQL. PostgreSQL has array types that can be searched using array operators:

CREATE TABLE Farms (
    ...
    Operators  text ARRAY,
    ...
);
CREATE INDEX idx_farm on &quot;Farms&quot; USING GIN (&quot;Operators&quot;);
...

SELECT *
FROM Farms
WHERE Operators @&gt; ARRAY[&#39;Joe&#39;]

NpgSQL supports array mapping and querying. This means you can easily write a LINQ query that checks for an element in the array:

var query=dbContext.Farms.Where(farm=&gt;farm.Operators.Contains(&quot;Joe&quot;));

Array fields can be indexed with a GIN index, providing fast querying. Only a few of the array operators take advantage of indexing though, which is why I used @&gt; instead of ANY.

In other databases you could use their JSON or XML support to store, query and sometimes index values instead of using string manipulations

答案2

得分: -1

尝试:

&lt;!-- 开始片段:js 隐藏:false 控制台:true 巴别: false --&gt;

                List&lt;string&gt; query = new List&lt;string&gt;()
                {
                    &quot;Ed&quot;,
                    &quot;Edward&quot;,
                    &quot;Ed; Adam&quot;,
                    null,
                    &quot;Bob; Edward&quot;,
                    &quot;Bob; Ed; Tom&quot;
                };
                List&lt;string&gt; operators = new List&lt;string&gt; { &quot;Ed&quot;, &quot;Peter&quot;, &quot;Mark&quot; };

                var results = query.Where(x =&gt; x != null &amp;&amp; x.Split(new char[] { &#39;;&#39; }).Any(y =&gt; operators.Any(z =&gt; y.Trim().Contains(z))));

&lt;!-- 结束片段 --&gt;
英文:

Try :

<!-- begin snippet: js hide: false console: true babel: false -->

            List&lt;string&gt; query = new List&lt;string&gt;()
            {
                &quot;Ed&quot;,
                &quot;Edward&quot;,
                &quot;Ed; Adam&quot;,
                null,
                &quot;Bob; Edward&quot;,
                &quot;Bob; Ed; Tom&quot;
            };
            List&lt;string&gt; operators = new List&lt;string&gt; { &quot;Ed&quot;, &quot;Peter&quot;, &quot;Mark&quot; };

            var results = query.Where(x =&gt; x != null &amp;&amp; x.Split(new char[] { &#39;;&#39; }).Any(y =&gt; operators.Any(z =&gt; y.Trim().Contains(z))));

<!-- end snippet -->

答案3

得分: -1

我将代码部分从您提供的内容中翻译出来,如下所示:

I ended up moving part of the logic to an SQL function:

CREATE OR REPLACE FUNCTION public."FarmContainsOperator"(param_op TEXT, id bigint)
 RETURNS bigint
 LANGUAGE plpgsql
AS $function$
DECLARE
	count integer; 
BEGIN
count :=  (SELECT COUNT(*)
	FROM public.myview WHERE 
		("Operator" LIKE param_op OR 
		"Operator" LIKE param_op||'%;' OR
		"Operator" LIKE '%;'||param_op OR
		"Operator" LIKE '%;'||param_op||';%') AND "farmID"=id
		);
return count;
END;
$function$

Then I declared that function in my model:

public static int FarmContainsOperator(string opName, long id) => throw new NotSupportedException();

And in the context:

modelBuilder
                .HasDbFunction(typeof(Myview).GetRuntimeMethod(nameof(Myview.FarmContainsOperator), new[] { typeof(string), typeof(long) }))
                .HasName("FarmContainsOperator");

So now I can do this:

var query = IQueryable<Myview> farms;
var List<string> operators = new List<string> { "Ed","Peter", "Mark" };

var predicate = PredicateBuilder.New<Myview>(false);

foreach (string op in operators)
{
	predicate = predicate.Or(x => Myview.FarmContainsOperator(op, x.farmID) > 0);
	
}
query = query.Where(predicate);

And it works. Probably it would be more efficient to use a TVF instead of scalar function, but it's sufficient for now.

英文:

I ended up moving part of the logic to an SQL function:

CREATE OR REPLACE FUNCTION public.&quot;FarmContainsOperator&quot;(param_op TEXT, id bigint)
 RETURNS bigint
 LANGUAGE plpgsql
AS $function$
DECLARE
	count integer; 
BEGIN
count :=  (SELECT COUNT(*)
	FROM public.myview WHERE 
		(&quot;Operator&quot; LIKE param_op OR 
		&quot;Operator&quot; LIKE param_op||&#39;;%&#39; OR
		&quot;Operator&quot; LIKE &#39;%;&#39;||param_op OR
		&quot;Operator&quot; LIKE &#39;%;&#39;||param_op||&#39;;%&#39;) AND &quot;farmID&quot;=id
		);
return count;
END;
$function$

Then I declared that function in my model:

public static int FarmContainsOperator(string opName,long id) =&gt; throw new NotSupportedException();

And in the context:

modelBuilder
                .HasDbFunction(typeof(Myview).GetRuntimeMethod(nameof(Myview.FarmContainsOperator), new[] { typeof(string), typeof(long) }))
                .HasName(&quot;FarmContainsOperator&quot;);

So now I can do this:

var query = IQueryable&lt;Myview&gt; farms;
var List&lt;string&gt; operators = new List&lt;string&gt; { &quot;Ed&quot;,&quot;Peter&quot;, &quot;Mark&quot; };

var predicate = PredicateBuilder.New&lt;Myview&gt;(false);

foreach (string op in operators)
{
	predicate = predicate.Or(x =&gt; Myview.FarmContainsOperator(op, x.farmID) &gt; 0);
	
}
query = query.Where(predicate);

And it works. Probably it would be more efficient to use a TVF instead of scalar function, but it's sufficient for now.

huangapple
  • 本文由 发表于 2023年7月3日 22:39:17
  • 转载请务必保留本文链接:https://go.coder-hub.com/76605793.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定