英文:
Working with Vision text recognition (Swift) on IOS 15.6 and memory grows ~25MB per recognition. Anyway to free up memory?
问题
我需要对数百张图像逐一进行文本识别,但每次内存会增加约25MB+。我在互联网上搜索过,但找不到是什么导致内存保留或如何释放它。通过插入断点,我可以看到每次调用imageRequestHandler.perform()时大小会增加。以下是相关代码。有什么建议吗?
英文:
I need to do text recognition on hundreds of images one at a time, but each time, memory grows by ~25mb+. I've searched the internet but can't find what is causing the memory retention or how to release it. By putting in breaks, I can see that the jump in size occurs with each call to imageRequestHandler.perform(). Below is the relevant code. Any suggestions?
func textRecognition(image:CGImage) {
let textRecognitionRequest = VNRecognizeTextRequest(
completionHandler: self.handleDetectedText)
textRecognitionRequest.recognitionLevel = .accurate
textRecognitionRequest.recognitionLanguages = ["en_US"]
textRecognitionRequest.usesLanguageCorrection = false
// request handler
let textRequest = [textRecognitionRequest]
let imageRequestHandler = VNImageRequestHandler(cgImage: image, orientation: .up, options: [:])
DispatchQueue.global(qos: .userInitiated).async {
do {
// perform request
try imageRequestHandler.perform(textRequest)
} catch let error {
print("Error \(error)")
}
}
}
func handleDetectedText(request: VNRequest?, error:Error?){
if let error = error { print("ERROR: \(error)"); return }
guard let results = request?.results, results.count > 0 else {
DispatchQueue.main.async {
self.result_field.isEnabled=false
self.result_field.text = "Scan failed - Retry"
let desc = NSMutableAttributedString(string: "Retake Photo", attributes: [NSAttributedString.Key.font: UIFont.systemFont(ofSize: 22, weight: .regular)])
self.take_button.setAttributedTitle(desc, for: UIControl.State.normal)
self.take_button.isHidden = false
self.take_button.isEnabled = true
}
return // code to process the text replaced by the 'return' statement
}}}
答案1
得分: 1
我建议进行分析以查看占用内存的原因。然而,一般来说,我看到有两个可能需要探索以进行潜在改进的领域:
-
图像可能导致内存使用较高。我们(开发者)倾向于捕获“最佳”图像。然而,文本识别可能不需要高质量大图像(对较小的图像产生相同的结果)。因此,尝试在捕获和/或处理时减小图像的尺寸:
- 尝试使用
AVCapturePhotoSettings
进行实验,特别是photoQualityPrioritization
和photo format
, - 尝试减小
CGImage
的大小/可能将其转为黑白。然而,这些操作也不是没有成本的。或尝试将CVPixelBuffer
直接传递给VNImageRequestHandler
(而不转换为 CGImage - 如果你从相机中获取图像的话)
- 尝试使用
-
查看
autoreleasepool
是否有助于内存使用。最明显的位置是imageRequestHandler.perform
:尝试在专用队列上执行它,并为该队列设置autoreleaseFrequency: .workItem
:
private static let performQueue = DispatchQueue(
label: "your label",
qos: .userInitiated,
autoreleaseFrequency: .workItem, // <-- 从默认值 .never 切换
target: .global(qos: .userInitiated) // <-- 与你现有的优先级相同
)
// ...
Self.performQueue.async {
try imageRequestHandler.perform(textRequest)
}
它的作用是:
该队列在执行块之前配置自动释放池,并在块执行完成后释放该池中的对象。
由于识别请求可能具有许多临时对象,你可能会从这个设置中受益,因为这些对象将在请求完成后立即释放(而不是“最终释放”)。不能保证它会有帮助,但值得一试。
再次强调,这些只是建议,但如果减少内存使用对你很重要,确实需要进行 性能评估。尽管我认为 25MB 是相当合理的。
英文:
I would recommend profiling to see what takes the memory. However in general I see 2 areas that could be explored for potential improvements:
-
Image may be causing higher memory usage. Our (developers) tendency is to capture "the best" image. However text recognition may not need a huge high quality image (and give the same results for smaller image). So try to make your image smaller on capturing and/or processing:
- experiment with
AVCapturePhotoSettings
, namelyphotoQualityPrioritization
and photoformat
, - try to reduce
CGImage
size / maybe make it B&W. Those operations are not without the cost as well though. Or try to passCVPixelBuffer
directly toVNImageRequestHandler
(without converting to CGImage - that's if you are taking image from camera)
- experiment with
-
See if
autoreleasepool
benefits your memory usage. The most obvious location isimageRequestHandler.perform
: try to perform it on a dedicated queue, and setautoreleaseFrequency: .workItem
for that queue:
private static let performQueue = DispatchQueue(
label: "your label",
qos: .userInitiated,
autoreleaseFrequency: .workItem, // <-- switching from default .never
target: .global(qos: .userInitiated) <-- same priority as the one you have
)
// ...
Self.performQueue.async {
try imageRequestHandler.perform(textRequest)
}
What does it do:
> The queue configures an autorelease pool before the execution of a block, and releases the objects in that pool after the block finishes executing.
Since the recognition request may have a lot of temporary objects, you may benefit from this setting, since the objects will be immediately released after the request completes (instead of "eventually"). There's no guarantee that it will help, but worth a try.
Again, those are suggestions, but really performance evaluation is needed if reducing memory is that important for you. Although I think 25MB is quite reasonable, to be honest.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论