Antlr v4线程安全模式

huangapple go评论83阅读模式
英文:

Antlr v4 thread-safe mode

问题

我写了这样的代码:

  1. 首先,我使用蚂蚁创建了一个协程池:
var pool, _ = ants.NewPoolWithFunc(5, func(commitDiff interface{}) {
	AnalyzeCommitDiff(commitDiff.(diffParsedType))
})

*函数AnalyzeCommitDiff()包含了我的antlr_analysis函数

  1. 然后我写了以下代码:
var (
	lexerPool *sync.Pool = &sync.Pool{New: func() interface{} {
		return javaparser.NewJavaLexer(nil)
	}}
	parserPool *sync.Pool = &sync.Pool{New: func() interface{} {
		return javaparser.NewJavaParser(nil)
	}}
	newTreeShapeListener *sync.Pool = &sync.Pool{New: func() interface{} {
		return new(TreeShapeListener)
	}}
)

func executeJava(diffText string) javaparser.AnalysisInfoType {

	input := antlr.NewInputStream(diffText)

	lexer := lexerPool.Get().(*javaparser.JavaLexer)
	defer lexerPool.Put(lexer)
	lexer.SetInputStream(input)

	stream := antlr.NewCommonTokenStream(lexer, 0)

	p := parserPool.Get().(*javaparser.JavaParser)
	defer parserPool.Put(p)
	p.SetTokenStream(stream)

	//p.AddErrorListener(antlr.NewDiagnosticErrorListener(true))

	p.BuildParseTrees = true

	p.GetInterpreter().SetPredictionMode(antlr.PredictionModeSLL)

	tree := p.CompilationUnit()

	listener := newTreeShapeListener.Get().(*TreeShapeListener)
	defer newTreeShapeListener.Put(listener)

	antlr.ParseTreeWalkerDefault.Walk(listener, tree)
	return javaparser.Infos
}

但显然存在线程安全问题。我该如何使我的代码线程安全?有人可以给我一个例子吗?

英文:

I wrote my code like this:

  1. Firstly I create routines pool with ants:
var pool, _ = ants.NewPoolWithFunc(5, func(commitDiff interface{}) {
	AnalyzeCommitDiff(commitDiff.(diffParsedType))
})

*The function AnalyzeCommitDiff() contains my antlr_analysis function

  1. Then I wrote these:
var (
	lexerPool *sync.Pool = &sync.Pool{New: func() interface{} {
		return javaparser.NewJavaLexer(nil)
	}}
	parserPool *sync.Pool = &sync.Pool{New: func() interface{} {
		return javaparser.NewJavaParser(nil)
	}}
	newTreeShapeListener *sync.Pool = &sync.Pool{New: func() interface{} {
		return new(TreeShapeListener)
	}}
)

func executeJava(diffText string) javaparser.AnalysisInfoType {

	input := antlr.NewInputStream(diffText)


	lexer := lexerPool.Get().(*javaparser.JavaLexer)
	defer lexerPool.Put(lexer)
	lexer.SetInputStream(input)

	stream := antlr.NewCommonTokenStream(lexer, 0)

	p := parserPool.Get().(*javaparser.JavaParser)
	defer parserPool.Put(p)
	p.SetTokenStream(stream)

	//p.AddErrorListener(antlr.NewDiagnosticErrorListener(true))

	p.BuildParseTrees = true

	p.GetInterpreter().SetPredictionMode(antlr.PredictionModeSLL)

	tree := p.CompilationUnit()

	listener := newTreeShapeListener.Get().(*TreeShapeListener)
	defer newTreeShapeListener.Put(listener)

	antlr.ParseTreeWalkerDefault.Walk(listener, tree)
	return javaparser.Infos
}

But there's issue obviously because of not thread-safe.
How can I make my code thread-safe?
Can anyone give me an example for that?

答案1

得分: 2

线程池在ANTLR4中无法工作。出于性能原因,除了一个地方,即解析器或词法分析器访问共享的DFA之外,没有额外的线程处理。

因此,您唯一可以使用线程的方式是将词法分析器/解析器实例与一个线程关联,并且只能从该单个线程访问它。每个解析器实例使用一个专用线程,并且不要在线程之间共享词法分析器/解析器。

英文:

Thread pools won't work with ANTLR4. For performance reasons there's no extra thread handling, except for one place where the shared DFA is accessed by a parser or lexer.

So, the only way you can use threads is to associate a lexer/parser instance to a thread and access that only from this single thread. Use a dedicated thread per parser instance and never share lexers/parser between threads.

huangapple
  • 本文由 发表于 2021年7月30日 16:40:02
  • 转载请务必保留本文链接:https://go.coder-hub.com/68587939.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定