antlr4:运算符优先级更改

huangapple go评论72阅读模式
英文:

antlr4: operator precedence changes

问题

关于antlr4和其标记优先级的问题,我有以下语法:

语法规则...

这用于进行命题逻辑。我希望它首先评估andor,然后再进行蕴含。如果我使用所提议的语法,它按预期工作。请注意,value_assignment规则被注释掉了。我有一些测试用例来测试功能:

测试类...

我认为我还需要展示(较小版本的)访问者以清楚地说明行为。实现如预期的那样直观:

访问者类...

我在代码中使用println语句以查看何时调用不同的规则。如果我测试所示的语法,其中

(variable=value_assignment)?

被注释掉,输出如预期:

测试评估:<true & false => true>
visitExpressionBinaryImplies 被调用...
visitExpressionBinaryAndOr 被调用...
visitExpressionBoolean 被调用...
visitExpressionBoolean 被调用...
visitExpressionBoolean 被调用...

测试评估:<false & true => true>
visitExpressionBinaryImplies 被调用...
visitExpressionBinaryAndOr 被调用...
visitExpressionBoolean 被调用...
visitExpressionBoolean 被调用...
visitExpressionBoolean 被调用...

测试通过:testEvaluation("true & false => true", [0, 1, 2], [0, 1, 2])
测试通过:testEvaluation("false & true => true", [0, 1, 2], [0, 1, 2])

但是,当我包含这些语句时,优先级发生变化:

测试评估:<true & false => true>
visitExpressionBinaryAndOr 被调用...
visitExpressionBoolean 被调用...
visitExpressionBinaryImplies 被调用...
visitExpressionBoolean 被调用...
visitExpressionBoolean 被调用...

测试评估:<false & true => true>
visitExpressionBinaryAndOr 被调用...
visitExpressionBoolean 被调用...
visitExpressionBinaryImplies 被调用...
visitExpressionBoolean 被调用...
visitExpressionBoolean 被调用...

测试未通过:testEvaluation("true & false => true", [0, 1, 2], [])

如你所见,蕴含在连接之后被调用,这不是我想要的。此外,第一个测试案例只是偶然通过测试,因为预期的运算符优先级不会得到满足。有人可以解释一下为什么使用value_assignment规则(我只是删除了注释符号)会改变运算符优先级吗?

非常感谢你的帮助!

英文:

I have a question concerning antlr4 and the precedence of its tokens. I have the following grammar:

grammar TestGrammar;

@header {
package some.package;
}

fragment A : (&#39;A&#39;|&#39;a&#39;) ;
fragment E : (&#39;E&#39;|&#39;e&#39;) ;
fragment F : (&#39;F&#39;|&#39;f&#39;) ;
fragment L : (&#39;L&#39;|&#39;l&#39;) ;
fragment R : (&#39;R&#39;|&#39;r&#39;) ;
fragment S : (&#39;S&#39;|&#39;s&#39;) ;
fragment T : (&#39;T&#39;|&#39;t&#39;) ;
fragment U : (&#39;U&#39;|&#39;u&#39;) ;
BOOL : (T R U E | F A L S E) ;

AND : &#39;&amp;&#39; ;
OR : &#39;|&#39; ;
IMPLIES : &#39;=&gt;&#39; ;

AS : &#39;als&#39; ;

ID : [a-zA-Z_][a-zA-Z0-9_]+ ;

value_assignment : AS name=ID ;

formula  :
  BOOL /*(variable=value_assignment)?*/  #ExpressionBoolean
  | identifier=ID /*(variable=value_assignment)?*/  #ExpressionIdentifier
  | leftFormula=formula operator=(AND | OR) rightFormula=formula /*(variable=value_assignment)?*/  #ExpressionBinaryAndOr
  | leftFormula=formula operator=IMPLIES rightFormula=formula /*(variable=value_assignment)?*/  #ExpressionBinaryImplies
;

It is used to do some propositional logic. I want it to first evaluate and or or, and the implication afterwards. If I am using the proposed grammar, it works as expected. Please notice, that the value_assignment-rules are commented out. I have some test-cases to play around with the functionality:

public class TestGrammarTest {

	private static ParserRuleContext parse(final String input) {
		final TestGrammarLexer lexer = new TestGrammarLexer(CharStreams.fromString(input));
		final CommonTokenStream tokens = new CommonTokenStream(lexer);
		return new TestGrammarParser(tokens).formula();
	}

	private static Set&lt;Object&gt; states() {
		final Set&lt;Object&gt; states = new HashSet&lt;Object&gt;();

		states.add(0);
		states.add(1);
		states.add(2);

		return states;
	}

	@DataProvider (name = &quot;testEvaluationData&quot;)
	public Object[][] testEvaluationData() {
		return new Object [][] {
			{&quot;true &amp; false =&gt; true&quot;, states(), states()},
			{&quot;false &amp; true =&gt; true&quot;, states(), states()},
		};
	}

	@Test (dataProvider = &quot;testEvaluationData&quot;)
	public void testEvaluation(final String input, final Set&lt;Object&gt; states, final Set&lt;Object&gt; expectedResult) {
		System.out.println(&quot;test evaluation of &lt;&quot; + input + &quot;&gt;&quot;);
		Assert.assertEquals(
				new TestGrammarVisitor(states).visit(parse(input)),
				expectedResult
			);
	}

}

I think that I also need to show (a smaller version of) my visitor to make the behaviour clear. The implementation is straight-forward as you would expect it:

public class TestGrammarVisitor extends TestGrammarBaseVisitor&lt;Set&lt;Object&gt;&gt; {

	final Set&lt;Object&gt; states;

	public TestGrammarVisitor(final Set&lt;Object&gt; theStates) {
		states = Collections.unmodifiableSet(theStates);
	}

	@Override
	public Set&lt;Object&gt; visitExpressionBoolean(final ExpressionBooleanContext ctx) {
		System.out.println(&quot;\nvisitExpressionBoolean called ...\n&quot;);
		final TerminalNode node = ctx.BOOL();
		final Set&lt;Object&gt; result;
		if (node.getText().equalsIgnoreCase(&quot;true&quot;)) {
			result = new HashSet&lt;&gt;(states);
			return result;
		}
		result = Collections.emptySet();
		return result;
	}

	@Override
	public Set&lt;Object&gt; visitExpressionBinaryAndOr(final ExpressionBinaryAndOrContext ctx) {
		System.out.println(&quot;\nvisitExpressionBinaryAndOr called ...\n&quot;);
		final Set&lt;Object&gt; result = new HashSet&lt;&gt;(super.visit(ctx.leftFormula));
		switch (ctx.operator.getText()) {
		case &quot;&amp;&quot;:
			result.retainAll(super.visit(ctx.rightFormula));
			return result;
		case &quot;|&quot;:
			result.addAll(super.visit(ctx.rightFormula));
			return result;
		default:
			throw new UnsupportedOperationException();
		}
	}

	@Override
	public Set&lt;Object&gt; visitExpressionBinaryImplies(final ExpressionBinaryImpliesContext ctx) {
		System.out.println(&quot;\nvisitExpressionBinaryImplies called ...\n&quot;);
		final Set&lt;Object&gt; result = new HashSet&lt;&gt;(states);
		result.removeAll(super.visit(ctx.leftFormula));
		result.addAll(super.visit(ctx.rightFormula));
		return result;
	}

	@Override
	protected Set&lt;Object&gt; aggregateResult(Set&lt;Object&gt; aggregate, Set&lt;Object&gt; nextResult) {
		if (aggregate == null) {
			return nextResult;
		}
		if (nextResult == null) {
			return aggregate;
		}
		Set&lt;Object&gt; clone = new HashSet&lt;&gt;(aggregate);
		clone.addAll(nextResult);
		return clone;
	}

}

I use the println-statements in order to see when the different rules will be called. If I test the shown grammar where the

(variable=value_assignment)?

are commented out, the output is as expected:

test evaluation of &lt;true &amp; false =&gt; true&gt;
visitExpressionBinaryImplies called ...
visitExpressionBinaryAndOr called ...
visitExpressionBoolean called ...
visitExpressionBoolean called ...
visitExpressionBoolean called ...

test evaluation of &lt;false &amp; true =&gt; true&gt;
visitExpressionBinaryImplies called ...
visitExpressionBinaryAndOr called ...
visitExpressionBoolean called ...
visitExpressionBoolean called ...
visitExpressionBoolean called ...

PASSED: testEvaluation(&quot;true &amp; false =&gt; true&quot;, [0, 1, 2], [0, 1, 2])
PASSED: testEvaluation(&quot;false &amp; true =&gt; true&quot;, [0, 1, 2], [0, 1, 2])

But, when I include those statements, the precedence changes:

test evaluation of &lt;true &amp; false =&gt; true&gt;
visitExpressionBinaryAndOr called ...
visitExpressionBoolean called ...
visitExpressionBinaryImplies called ...
visitExpressionBoolean called ...
visitExpressionBoolean called ...

test evaluation of &lt;false &amp; true =&gt; true&gt;
visitExpressionBinaryAndOr called ...
visitExpressionBoolean called ...
visitExpressionBinaryImplies called ...
visitExpressionBoolean called ...
visitExpressionBoolean called ...

PASSED: testEvaluation(&quot;true &amp; false =&gt; true&quot;, [0, 1, 2], [0, 1, 2])
FAILED: testEvaluation(&quot;false &amp; true =&gt; true&quot;, [0, 1, 2], [0, 1, 2])
java.lang.AssertionError: Sets differ: expected [0, 1, 2] but got []

As you can see, the implication will be called AFTER the conjunction, which is not what I want. Also, the first test case passes the test by accident, since the intended operator precedence will not be met. Can anyone explain to me why the operator precedence changes because of using the value_assignment-rule (I just deleted the comment symbols around it)?

Thank you very much for your help!

答案1

得分: 1

经过一些尝试,我成功地通过以下方式解决了这个问题:

grammar TestGrammar;

@header {
package some.package;
}

fragment A : ('A'|'a') ;
fragment E : ('E'|'e') ;
fragment F : ('F'|'f') ;
fragment L : ('L'|'l') ;
fragment R : ('R'|'r') ;
fragment S : ('S'|'s') ;
fragment T : ('T'|'t') ;
fragment U : ('U'|'u') ;
BOOL : (T R U E | F A L S E) ;

AND : '&' ;
OR : '|' ;
IMPLIES : '=>' ;

AS : 'als' ;

ID : [a-zA-Z_][a-zA-Z0-9_]+ ;

formula  :
  BOOL #ExpressionBoolean
  | leftFormula=formula operator=(AND | OR) rightFormula=formula #ExpressionBinaryAndOr
  | leftFormula=formula operator=IMPLIES rightFormula=formula #ExpressionBinaryImplies
  | innerFormula=formula AS storageName=ID  #ExpressionAssignment
  | identifier=ID #ExpressionIdentifier
;

因此,我将处理存储能力作为一个单独的公式。虽然这不完全是我想做的(它迫使我为每个子公式提供存储选项,并且如果需要特定子公式的存储行为,我必须在访问者中进行管理)。但是,我可以接受这种解决方法。

英文:

After some attempts, I managed to ship around the problem as follows:

grammar TestGrammar;

@header {
package some.package;
}

fragment A : (&#39;A&#39;|&#39;a&#39;) ;
fragment E : (&#39;E&#39;|&#39;e&#39;) ;
fragment F : (&#39;F&#39;|&#39;f&#39;) ;
fragment L : (&#39;L&#39;|&#39;l&#39;) ;
fragment R : (&#39;R&#39;|&#39;r&#39;) ;
fragment S : (&#39;S&#39;|&#39;s&#39;) ;
fragment T : (&#39;T&#39;|&#39;t&#39;) ;
fragment U : (&#39;U&#39;|&#39;u&#39;) ;
BOOL : (T R U E | F A L S E) ;

AND : &#39;&amp;&#39; ;
OR : &#39;|&#39; ;
IMPLIES : &#39;=&gt;&#39; ;

AS : &#39;als&#39; ;

ID : [a-zA-Z_][a-zA-Z0-9_]+ ;

formula  :
  BOOL #ExpressionBoolean
  | leftFormula=formula operator=(AND | OR) rightFormula=formula #ExpressionBinaryAndOr
  | leftFormula=formula operator=IMPLIES rightFormula=formula #ExpressionBinaryImplies
  | innerFormula=formula AS storageName=ID  #ExpressionAssignment
  | identifier=ID #ExpressionIdentifier
;

So, I will handle the storage ability as a separate formula. This is not exactly what I wanted to do (it forces me to provide the storage option to each sub-formula, and I have to manage it in the visitor if a storing behaviour is desired or not for a specific sub-formula). But, I can live with that work-around.

huangapple
  • 本文由 发表于 2020年10月8日 15:07:41
  • 转载请务必保留本文链接:https://go.coder-hub.com/64257414.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定