英文:
How to test a token in JavaCC
问题
TOKEN: // 我想让这个令牌只接受偶数
{
<TOKENPARANUMEROSPARES: (["0"-"9"])* ("0"|"2"|"4"|"6"|"8") > {System.out.println("Soy un numero par");}
}
void llamarNumeroPar ():
{}
{
// 我想让我的程序能够从主程序中发出下面显示的消息
< TOKENPARANUMEROSPARES > {System.out.println ("soy un numeroPar");}
}
<details>
<summary>英文:</summary>
I want to create a token that accepts only even numbers, and I need to test it in javacc, I'm just starting to use it and I can't think of any way to test this token, can anyone help me, thanks!
TOKEN: //I want this token to only accept even numbers
{
<TOKENPARANUMEROSPARES: (["0"-"9"])* ("0"|"2"|"4"|"6"|"8") > {System.out.println("Soy un numero par");}
}
void llamarNumeroPar ():
{}
{
//I want my program to be able to launch a message from my main with the message shown below
< TOKENPARANUMEROSPARES > {System.out.println ("soy un numeroPar");}
}
To print something, what do you recommend? Do I do it from the main or what could I correct in this part of my code?
</details>
# 答案1
**得分**: 2
让我们假设你有以下的 `parser.jj` 文件：
```java
options {
STATIC = false;
IGNORE_CASE = false;
}
PARSER_BEGIN(Parser)
package test14;
public class Parser {
}
PARSER_END(Parser)
TOKEN:
{
<EVEN: (["0"-"9"])* ("0"|"2"|"4"|"6"|"8") >
}
void llamarNumeroPar ():
{}
{
< EVEN > {System.out.println ("soy un numeroPar");}
}
现在你可以编写一个主类：
package test14;
import java.io.StringReader;
public class Test14 {
public static void main(String[] args) {
Parser parser = new Parser(new StringReader("1234567"));
Token token = parser.getNextToken();
if (token.kind == Parser.EVEN) {
System.out.println("Even");
} else {
System.out.println("Something else");
}
}
}
然后... 惊喜：输出结果是 "Even"。这是因为它已经读取了 "123456"，这确实是一个偶数。
现在添加第二个标记 ODD：
TOKEN:
{
<EVEN: (["0"-"9"])* ("0"|"2"|"4"|"6"|"8") >
| <ODD: (["0"-"9"])* ("1"|"3"|"5"|"7"|"9") >
}
并且修改主类如下：
if (token.kind == Parser.EVEN) {
System.out.println("Even: " + token.image);
} else if (token.kind == Parser.ODD) {
System.println("Odd: " + token.image);
} else {
System.out.println("Something else: " + token.image);
}
这次，输出结果是预期的：
Odd: 1234567
考虑定义一个标记 NUMBER 并在 Java 代码中处理它的奇偶性。
英文:
Let's say you have the following parser.jj file:
options {
STATIC = false;
IGNORE_CASE = false;
}
PARSER_BEGIN(Parser)
package test14;
public class Parser {
}
PARSER_END(Parser)
TOKEN:
{
<EVEN: (["0"-"9"])* ("0"|"2"|"4"|"6"|"8") >
}
void llamarNumeroPar ():
{}
{
< EVEN > {System.out.println ("soy un numeroPar");}
}
You can now write a main class:
package test14;
import java.io.StringReader;
public class Test14 {
public static void main(String[] args) {
Parser parser = new Parser(new StringReader("1234567"));
Token token = parser.getNextToken();
if (token.kind == Parser.EVEN) {
System.out.println("Even");
} else {
System.out.println("Something else");
}
}
}
And... surprise: the output is "Even". This is because it has read "123456" which is indeed an even number.
Now add a second token ODD:
TOKEN:
{
<EVEN: (["0"-"9"])* ("0"|"2"|"4"|"6"|"8") >
| <ODD: (["0"-"9"])* ("1"|"3"|"5"|"7"|"9") >
}
And modify the main class as follows:
if (token.kind == Parser.EVEN) {
System.out.println("Even: " + token.image);
} else if (token.kind == Parser.ODD) {
System.out.println("Odd: " + token.image);
} else {
System.out.println("Something else: " + token.image);
}
This time, the output is as expected:
Odd: 1234567
Consider defining a token NUMBER and dealing with its parity in the Java code.
答案2
得分: 2
以下是翻译好的部分：
"Maurice's answer is excellent. I'm going to come at it from a slightly different angle by answering a slightly broader question.
我认为莫里斯的回答很出色。我将从略微不同的角度回答一个略微广泛的问题。
I don't think it makes a lot of sense to test one token definition at a time. This is because the tricky part of writing token managers is usually not the individual regular expressions, but rather the interactions between them, especially when there are multiple states. So the broader question is how to test a token manager as a whole.
我不认为逐个测试令牌定义有多大意义。这是因为编写令牌管理器的棘手部分通常不是单个正则表达式，而是它们之间的相互作用，特别是当存在多个状态时。因此，更广泛的问题是如何测试整个令牌管理器。
I think something like the following could be quite useful.
我认为类似以下内容可能非常有用。
import your.favourite.test.framework ;
导入您喜欢的测试框架；
class TokenTests implements XYZConstants {
TokenTests类，实现XYZConstants接口：
private final int TME = -99 ; // -99 because this should not be already in use as a token kind.
私有常量TME等于-99；-99是因为不应该已经用作令牌种类。
private void doOneTokenTest( String input, int[] expectation ) {
执行一个令牌测试的私有方法，传入输入字符串和期望的令牌数组；
XYZ parser = new XYZ(new StringReader(input));
创建XYZ解析器并传入输入字符串；
for( int k = 0 ; k < expectation.length; ++k ) {
循环遍历期望的令牌数组；
int state = parser.token_source.curLexState ;
获取解析器的当前状态；
try {
尝试执行以下操作：
Token token = parser.getNextToken();
获取下一个令牌；
checkThat( expectation[k] != TME,
检查期望值是否不等于TME；
"Expected a TokenMgrErr, but got a" +
"Expected a TokenMgrErr, but got a" +
tokenImage[ token.kind ] + " at token " +
tokenImage[token.kind]在令牌处的字符串表示 + " at token " +
k + " of input <<" + input + ">>" +
k在输入中的位置 + " of input <<" + input + ">>" +
" state is " + state ) ;
" state is " + state ) ;
checkThat( token.kind == expectation[k],
检查令牌的种类是否等于期望的种类；
"Expected a " +
"Expected a " +
tokenImage[ expectation[k] ] +
tokenImage[期望的种类] +
" but got a " +
" but got a " +
tokenImage[ token.kind ] + " at token " +
tokenImage[令牌的种类] + " at token " +
k + " of input <<" + input + ">>" +
k在输入中的位置 + " of input <<" + input + ">>" +
" state is " + state ) ;
" state is " + state ) ;
} catch( TokenMgrErr e ) {
捕获TokenMgrErr异常，执行以下操作：
checkThat( expectation[k] == TME,
检查期望值是否等于TME；
"Expected a " +
"Expected a " +
tokenImage[ expectation[k] ] +
tokenImage[期望的种类] +
" but got a TokenMgrErr at token " +
" but got a TokenMgrErr at token " +
k + " of input <<" + input + ">>" +
k在输入中的位置 + " of input <<" + input + ">>" +
" state is " + state ) ; } } }
" state is " + state ) ; } } }
So now you can create and run a bunch of tests
现在，您可以创建并运行一系列测试：
@Test
@Test注解，表示下面是一个测试方法；
public void testTokenMatching() {
public void testTokenMatching()方法：
record Pair( String input, int[] expectation ) ;
定义一个记录类型Pair，包含输入字符串和期望的令牌数组；
Pair[] tests = {
Pair[] tests数组，包含多个测试：
new Pair( "1234", int[]{ EVEN, EOF } ),
new Pair( "1233", int[]{ ODD, EOF } ),
new Pair( "". int[] { EOF },
new Pair( "123.3", int[]{ ODD, TME } }
new Pair( "1234", int[]{ EVEN, EOF } ),
new Pair( "1233", int[]{ ODD, EOF } ),
new Pair( "". int[] { EOF },
new Pair( "123.3", int[]{ ODD, TME } }
for( Pair t : tests ) {
遍历测试数组，执行以下操作：
doOneTokenTest( t.input(), t.expectation() ) ; } }
doOneTokenTest方法，传入输入和期望的令牌数组；
这部分已经完成了对代码的翻译。
英文:
Maurice's answer is excellent. I'm going to come at it from a slightly different angle by answering a slightly broader question.
I don't think it makes a lot of sense to test one token definition at a time. This is because the tricky part of writing token managers is usually not the individual regular expressions, but rather the interactions between them, especially when there are multiple states. So the broader question is how to test a token manager as a whole.
I think something like the following could be quite useful.
import your.favourite.test.framework ;
class TokenTests implements XYZConstants {
private final int TME = -99 ; // -99 because this should not be already in use as a token kind.
private void doOneTokenTest( String input, int[] expectation ) {
XYZ parser = new XYZ(new StringReader(input));
for( int k = 0 ; k < expectation.length; ++k ) {
int state = parser.token_source.curLexState ;
try {
Token token = parser.getNextToken();
checkThat( expectation[k] != TME,
"Expected a TokenMgrErr, but got a" +
tokenImage[ token.kind ] + " at token " +
k + " of input <<" + input + ">>" +
" state is " + state ) ;
checkThat( token.kind == expectation[k],
"Expected a " +
tokenImage[ expectation[k] ] +
" but got a " +
tokenImage[ token.kind ] + " at token " +
k + " of input <<" + input + ">>" +
" state is " + state ) ;
} catch( TokenMgrErr e ) {
checkThat( expectation[k] == TME,
"Expected a " +
tokenImage[ expectation[k] ] +
" but got a TokenMgrErr at token " +
k + " of input <<" + input + ">>" +
" state is " + state ) ; } } }
So now you can create and run a bunch of tests
@Test
public void testTokenMatching() {
record Pair( String input, int[] expectation ) ;
Pair[] tests = {
new Pair( "1234", int[]{ EVEN, EOF } ),
new Pair( "1233", int[]{ ODD, EOF } ),
new Pair( "". int[] { EOF },
new Pair( "123.3", int[]{ ODD, TME } } ;
for( Pair t : tests ) {
doOneTokenTest( t.input(), t.expectation() ) ; } }
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论