英文:
Antlr v4 thread-safe mode
问题
我写了这样的代码:
- 首先,我使用蚂蚁创建了一个协程池:
var pool, _ = ants.NewPoolWithFunc(5, func(commitDiff interface{}) {
AnalyzeCommitDiff(commitDiff.(diffParsedType))
})
*函数AnalyzeCommitDiff()包含了我的antlr_analysis函数
- 然后我写了以下代码:
var (
lexerPool *sync.Pool = &sync.Pool{New: func() interface{} {
return javaparser.NewJavaLexer(nil)
}}
parserPool *sync.Pool = &sync.Pool{New: func() interface{} {
return javaparser.NewJavaParser(nil)
}}
newTreeShapeListener *sync.Pool = &sync.Pool{New: func() interface{} {
return new(TreeShapeListener)
}}
)
func executeJava(diffText string) javaparser.AnalysisInfoType {
input := antlr.NewInputStream(diffText)
lexer := lexerPool.Get().(*javaparser.JavaLexer)
defer lexerPool.Put(lexer)
lexer.SetInputStream(input)
stream := antlr.NewCommonTokenStream(lexer, 0)
p := parserPool.Get().(*javaparser.JavaParser)
defer parserPool.Put(p)
p.SetTokenStream(stream)
//p.AddErrorListener(antlr.NewDiagnosticErrorListener(true))
p.BuildParseTrees = true
p.GetInterpreter().SetPredictionMode(antlr.PredictionModeSLL)
tree := p.CompilationUnit()
listener := newTreeShapeListener.Get().(*TreeShapeListener)
defer newTreeShapeListener.Put(listener)
antlr.ParseTreeWalkerDefault.Walk(listener, tree)
return javaparser.Infos
}
但显然存在线程安全问题。我该如何使我的代码线程安全?有人可以给我一个例子吗?
英文:
I wrote my code like this:
- Firstly I create routines pool with ants:
var pool, _ = ants.NewPoolWithFunc(5, func(commitDiff interface{}) {
AnalyzeCommitDiff(commitDiff.(diffParsedType))
})
*The function AnalyzeCommitDiff() contains my antlr_analysis function
- Then I wrote these:
var (
lexerPool *sync.Pool = &sync.Pool{New: func() interface{} {
return javaparser.NewJavaLexer(nil)
}}
parserPool *sync.Pool = &sync.Pool{New: func() interface{} {
return javaparser.NewJavaParser(nil)
}}
newTreeShapeListener *sync.Pool = &sync.Pool{New: func() interface{} {
return new(TreeShapeListener)
}}
)
func executeJava(diffText string) javaparser.AnalysisInfoType {
input := antlr.NewInputStream(diffText)
lexer := lexerPool.Get().(*javaparser.JavaLexer)
defer lexerPool.Put(lexer)
lexer.SetInputStream(input)
stream := antlr.NewCommonTokenStream(lexer, 0)
p := parserPool.Get().(*javaparser.JavaParser)
defer parserPool.Put(p)
p.SetTokenStream(stream)
//p.AddErrorListener(antlr.NewDiagnosticErrorListener(true))
p.BuildParseTrees = true
p.GetInterpreter().SetPredictionMode(antlr.PredictionModeSLL)
tree := p.CompilationUnit()
listener := newTreeShapeListener.Get().(*TreeShapeListener)
defer newTreeShapeListener.Put(listener)
antlr.ParseTreeWalkerDefault.Walk(listener, tree)
return javaparser.Infos
}
But there's issue obviously because of not thread-safe.
How can I make my code thread-safe?
Can anyone give me an example for that?
答案1
得分: 2
线程池在ANTLR4中无法工作。出于性能原因,除了一个地方,即解析器或词法分析器访问共享的DFA之外,没有额外的线程处理。
因此,您唯一可以使用线程的方式是将词法分析器/解析器实例与一个线程关联,并且只能从该单个线程访问它。每个解析器实例使用一个专用线程,并且不要在线程之间共享词法分析器/解析器。
英文:
Thread pools won't work with ANTLR4. For performance reasons there's no extra thread handling, except for one place where the shared DFA is accessed by a parser or lexer.
So, the only way you can use threads is to associate a lexer/parser instance to a thread and access that only from this single thread. Use a dedicated thread per parser instance and never share lexers/parser between threads.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论