JavaCC 创建自定义 Token 类

huangapple go评论90阅读模式
英文:

JavaCC creating custom Token class

问题

我正在为我的编译器和解释器课程的学术任务而努力,我们当前的任务是使用JavaCC创建一个扫描器和一组标记。我对JavaCC的工作原理有很好的理解,但我的问题是在遇到困难时找到在线资源来帮助我。我正在努力创建一个自定义的Token类,让我们称其为NewToken.java。我知道基本的Token类有一个image变量和一个kind变量,但我想实现我自己的变量"value"。此外,我想弄清楚如何赋值这个值。我希望value变量保存我扫描的文字值,例如,我的NewToken与以下内容匹配:

<IDENTIFIER:(<LETTER>)+ (<LETTER> | <DIGIT>)* >
<#LETTER:["a" - "z"] >
<#DIGIT: ["0" - "9"] >

因此,类似"Name123Name"的内容将被捕获,当发生时,我希望将字符串"Name123Name"存储到我的NewToken对象的'value'变量中。我希望这是有意义的,我对JavaCC还不熟悉,可能会用错误的名字来称呼事物。

public NewToken() {}
public NewToken(int kind) {
  this(kind, null);
}
public NewToken(int kind, String image) {
  this.kind = kind;
  this.image = image;
  this.value = image;
}
public String toString() {
  return image;
}
public static Token newToken(int ofKind, String image) {
  switch (ofKind) {
    default: return new Token(ofKind, image);
  }
}
public static Token newToken(int ofKind) {
  return newToken(ofKind, null);
}

以上是我的NewToken类的部分代码,我让它扩展了Token并实现了java.io.Serializable。我使用生成的Token.java代码创建了它。我还有我的变量声明和getValue()函数,这里没有列出来以节省空间。我不是在寻求任何人为我完成工作,我只需要一些关于如何使这个工作正常运行的指导,非常感谢你的帮助。

英文:

I'm working on a school assignment for my compiler and interpreters course and our current task is to create a scanner and a set of tokens using JavaCC. I have a pretty solid understanding of how JavaCC works but my problem is finding resources online to help me out when I get stuck. I am working on creating a custom Token class, let's call it NewToken.Java. I know that the base Token class has an image variable and a kind variable but I want to implement my own variable "value". Furthermore I want to figure out how I can assign this value. I want the value variable to hold the literal value of what I scan, for example, my NewToken is being matched to the following

&lt; IDENTIFIER:(&lt; LETTER &gt;)+ ( &lt; LETTER &gt; | &lt; DIGIT &gt;)* &gt;
&lt; #LETTER:[&quot;a&quot; - &quot;z&quot;] &gt;
&lt; #DIGIT: [&quot;0&quot; - &quot;9&quot;] &gt;

so something along the lines of Name123Name would get caught and when it does I want to store the string "Name123Name" into the 'value' variable of my NewToken object. I hope this makes sense, I am still new to JavaCC and may be calling things by there wrong name here.

 public NewToken(){}
  public NewToken(int kind){
    this(kind,null);
  }
  public NewToken(int kind, String image){
    this.kind=kind;
    this.image=image;
    this.value=image;
  }
  public String toString(){
    return image;
  }
  public static Token newToken(int ofKind, String image){
    switch(ofKind){
      default : return new Token(ofKind, image);
    }
  }
  public static Token newToken(int ofKind){
    return newToken(ofKind, null);
  }
}

Above is part of my code for the NewToken class, I have it extending Token and implementing java.io.serializable. I created by using the code generated for Token.java. I also have my variable declarations and my getValue() function which are not listed here to save space. I'm not looking for anyone to do my work for me I just need some guidance on how I would get this working, thank you in advance.

答案1

得分: 2

首先,我认为 newToken 程序应该返回类型为 NewToken 的对象,而不是 Token

public static NewToken newToken(int ofKind, String image){
  return new NewToken(ofKind, image);
}

我认为你可能不需要第二个方法。但是,我不能完全确定,所以我会保留它。

我对于 valueimage 之间的区别还有些不太清楚,但我假设你可以从 imagekind 计算出所需的 value 值。我进一步假设你已经将这个函数实现为一个静态方法。

private static String computeValue(int kind, String image) {...}

删除前两个构造函数,只保留剩下的一个:

private NewToken(int kind, String image){
  this.kind = kind;
  this.image = image;
  this.value = computeValue(kind, image);
}
英文:

First off, I think the newToken routine should return objects of type NewToken rather than Token.

  public static Token newToken(int ofKind, String image){
    return new NewToken(ofKind, image);
  }
  public static Token newToken(int ofKind){
    return new NewToken(ofKind, null);
  } 

(I don’t think you need that second method. But, I’m not completely sure, so I’ll leave it.)

It’s a bit unclear to me how you want value to differ from image, but I’m going to assume that you can compute the desired value for value from the image and the kind. And I’ll further assume that you have implemented this function as a static method.

private static String computeValue(int kind, String image) {...}

Delete the first two constructors and the remaining one should be:

  private NewToken(int kind, String image){
    this.kind = kind;
    this.image = image;
    this.value = computeValue( kind, image );
  }

答案2

得分: 0

诺维尔教授给您的答案基于使用非常旧且已过时的 JavaCC 版本。他建议您采取的方法可能是在使用传统的 JavaCC 中做得最好的方式。

然而,JavaCC 的最新版本是 JavaCC 21,它可以在非常干净、优雅的方式下直接处理这种用例。具体信息请参阅这里

正如您所见,您可以在语法文件中添加注释,导致生成并使用各种 Token 子类。

此外,JavaCC 21 具有 代码注入 功能,允许您直接将代码注入到任何生成的文件中,包括 Token 子类。这个功能在传统的 JavaCC 中完全不存在。但是通过使用这个功能,您可以将 computeValue 方法直接注入到适当的 Token 子类中。

INJECT NewToken:
{
    private static String computeValue(int kind, String image) {...}
}

您将其放入您的语法中,computeValue 方法将会被插入到生成的 NewToken.java 文件中。

顺便提一下,最近在 dzone.com 上出现了一篇关于 JavaCC 21 的文章

英文:

The answer that Professor Norvell is giving you is based on using a very old, obsolete version of JavaCC. The way he's suggesting you go about things is probably about the best way of doing it, if you were going to use the legacy JavaCC.

However, the most advanced version of JavaCC is JavaCC 21 and it handles this sort of use case straight out of the box in a very clean, elegant manner. See here for information on this specifically.

As you can see, you can put annotations in your grammar file that cause the various Token subclasses to be generated and used.

Also, JavaCC 21 has code injection that allows you to inject code directly into the any generated files, including the Token subclasses. That feature is not at all present in legacy JavaCC either. But using that, you could just inject your computeValue method right into the appropriate Token subclass.

INJECT NewToken :
{
   private static String computeValue(int kind, String image) {...}
}

You put that in your grammar and the computeValue method just gets inserted into the generated NewToken.java file.

By the way, there is an article about JavaCC 21 that appeared recently on dzone.com.

huangapple
  • 本文由 发表于 2020年10月3日 12:43:55
  • 转载请务必保留本文链接:https://go.coder-hub.com/64180801.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定