英文:
Antlr v4 thread-safe mode
问题
我写了这样的代码:
- 首先,我使用蚂蚁创建了一个协程池:
 
var pool, _ = ants.NewPoolWithFunc(5, func(commitDiff interface{}) {
	AnalyzeCommitDiff(commitDiff.(diffParsedType))
})
*函数AnalyzeCommitDiff()包含了我的antlr_analysis函数
- 然后我写了以下代码:
 
var (
	lexerPool *sync.Pool = &sync.Pool{New: func() interface{} {
		return javaparser.NewJavaLexer(nil)
	}}
	parserPool *sync.Pool = &sync.Pool{New: func() interface{} {
		return javaparser.NewJavaParser(nil)
	}}
	newTreeShapeListener *sync.Pool = &sync.Pool{New: func() interface{} {
		return new(TreeShapeListener)
	}}
)
func executeJava(diffText string) javaparser.AnalysisInfoType {
	input := antlr.NewInputStream(diffText)
	lexer := lexerPool.Get().(*javaparser.JavaLexer)
	defer lexerPool.Put(lexer)
	lexer.SetInputStream(input)
	stream := antlr.NewCommonTokenStream(lexer, 0)
	p := parserPool.Get().(*javaparser.JavaParser)
	defer parserPool.Put(p)
	p.SetTokenStream(stream)
	//p.AddErrorListener(antlr.NewDiagnosticErrorListener(true))
	p.BuildParseTrees = true
	p.GetInterpreter().SetPredictionMode(antlr.PredictionModeSLL)
	tree := p.CompilationUnit()
	listener := newTreeShapeListener.Get().(*TreeShapeListener)
	defer newTreeShapeListener.Put(listener)
	antlr.ParseTreeWalkerDefault.Walk(listener, tree)
	return javaparser.Infos
}
但显然存在线程安全问题。我该如何使我的代码线程安全?有人可以给我一个例子吗?
英文:
I wrote my code like this:
- Firstly I create routines pool with ants:
 
var pool, _ = ants.NewPoolWithFunc(5, func(commitDiff interface{}) {
	AnalyzeCommitDiff(commitDiff.(diffParsedType))
})
*The function AnalyzeCommitDiff() contains my antlr_analysis function
- Then I wrote these:
 
var (
	lexerPool *sync.Pool = &sync.Pool{New: func() interface{} {
		return javaparser.NewJavaLexer(nil)
	}}
	parserPool *sync.Pool = &sync.Pool{New: func() interface{} {
		return javaparser.NewJavaParser(nil)
	}}
	newTreeShapeListener *sync.Pool = &sync.Pool{New: func() interface{} {
		return new(TreeShapeListener)
	}}
)
func executeJava(diffText string) javaparser.AnalysisInfoType {
	input := antlr.NewInputStream(diffText)
	lexer := lexerPool.Get().(*javaparser.JavaLexer)
	defer lexerPool.Put(lexer)
	lexer.SetInputStream(input)
	stream := antlr.NewCommonTokenStream(lexer, 0)
	p := parserPool.Get().(*javaparser.JavaParser)
	defer parserPool.Put(p)
	p.SetTokenStream(stream)
	//p.AddErrorListener(antlr.NewDiagnosticErrorListener(true))
	p.BuildParseTrees = true
	p.GetInterpreter().SetPredictionMode(antlr.PredictionModeSLL)
	tree := p.CompilationUnit()
	listener := newTreeShapeListener.Get().(*TreeShapeListener)
	defer newTreeShapeListener.Put(listener)
	antlr.ParseTreeWalkerDefault.Walk(listener, tree)
	return javaparser.Infos
}
But there's issue obviously because of not thread-safe.
How can I make my code thread-safe?
Can anyone give me an example for that?
答案1
得分: 2
线程池在ANTLR4中无法工作。出于性能原因,除了一个地方,即解析器或词法分析器访问共享的DFA之外,没有额外的线程处理。
因此,您唯一可以使用线程的方式是将词法分析器/解析器实例与一个线程关联,并且只能从该单个线程访问它。每个解析器实例使用一个专用线程,并且不要在线程之间共享词法分析器/解析器。
英文:
Thread pools won't work with ANTLR4. For performance reasons there's no extra thread handling, except for one place where the shared DFA is accessed by a parser or lexer.
So, the only way you can use threads is to associate a lexer/parser instance to a thread and access that only from this single thread. Use a dedicated thread per parser instance and never share lexers/parser between threads.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。


评论