Working with Vision text recognition (Swift) on IOS 15.6 and memory grows ~25MB per recognition. Anyway to free up memory?

huangapple go评论71阅读模式
英文:

Working with Vision text recognition (Swift) on IOS 15.6 and memory grows ~25MB per recognition. Anyway to free up memory?

问题

我需要对数百张图像逐一进行文本识别,但每次内存会增加约25MB+。我在互联网上搜索过,但找不到是什么导致内存保留或如何释放它。通过插入断点,我可以看到每次调用imageRequestHandler.perform()时大小会增加。以下是相关代码。有什么建议吗?

英文:

I need to do text recognition on hundreds of images one at a time, but each time, memory grows by ~25mb+. I've searched the internet but can't find what is causing the memory retention or how to release it. By putting in breaks, I can see that the jump in size occurs with each call to imageRequestHandler.perform(). Below is the relevant code. Any suggestions?

    func textRecognition(image:CGImage) {
                
        let textRecognitionRequest = VNRecognizeTextRequest(
            completionHandler: self.handleDetectedText)
        textRecognitionRequest.recognitionLevel = .accurate
        textRecognitionRequest.recognitionLanguages = ["en_US"]
        textRecognitionRequest.usesLanguageCorrection = false
        
        // request handler
        let textRequest = [textRecognitionRequest]
        let imageRequestHandler = VNImageRequestHandler(cgImage: image, orientation: .up, options: [:])
        DispatchQueue.global(qos: .userInitiated).async {
            do {
                // perform request
                try imageRequestHandler.perform(textRequest)
            } catch let error {
                print("Error \(error)")
            }
        }
    }
    
    func handleDetectedText(request: VNRequest?, error:Error?){
        if let error = error {  print("ERROR: \(error)"); return  }
        guard let results = request?.results, results.count > 0 else {
            DispatchQueue.main.async {
                self.result_field.isEnabled=false
                self.result_field.text = "Scan failed - Retry"
                let desc = NSMutableAttributedString(string: "Retake Photo", attributes: [NSAttributedString.Key.font: UIFont.systemFont(ofSize: 22, weight: .regular)])
                self.take_button.setAttributedTitle(desc, for: UIControl.State.normal)
                self.take_button.isHidden = false
                self.take_button.isEnabled = true
            }
            return // code to process the text replaced by the 'return' statement
        }}}

答案1

得分: 1

我建议进行分析以查看占用内存的原因。然而,一般来说,我看到有两个可能需要探索以进行潜在改进的领域:

  1. 图像可能导致内存使用较高。我们(开发者)倾向于捕获“最佳”图像。然而,文本识别可能不需要高质量大图像(对较小的图像产生相同的结果)。因此,尝试在捕获和/或处理时减小图像的尺寸:

    • 尝试使用 AVCapturePhotoSettings 进行实验,特别是 photoQualityPrioritizationphoto format
    • 尝试减小 CGImage 的大小/可能将其转为黑白。然而,这些操作也不是没有成本的。或尝试将 CVPixelBuffer 直接传递给 VNImageRequestHandler(而不转换为 CGImage - 如果你从相机中获取图像的话)
  2. 查看 autoreleasepool 是否有助于内存使用。最明显的位置是 imageRequestHandler.perform:尝试在专用队列上执行它,并为该队列设置 autoreleaseFrequency: .workItem

private static let performQueue = DispatchQueue(
        label: "your label",
        qos: .userInitiated,
        autoreleaseFrequency: .workItem, // <-- 从默认值 .never 切换
        target: .global(qos: .userInitiated) // <-- 与你现有的优先级相同
    )

// ...

Self.performQueue.async {
    try imageRequestHandler.perform(textRequest)
}

它的作用是:

该队列在执行块之前配置自动释放池,并在块执行完成后释放该池中的对象。

由于识别请求可能具有许多临时对象,你可能会从这个设置中受益,因为这些对象将在请求完成后立即释放(而不是“最终释放”)。不能保证它会有帮助,但值得一试。

再次强调,这些只是建议,但如果减少内存使用对你很重要,确实需要进行 性能评估。尽管我认为 25MB 是相当合理的。

英文:

I would recommend profiling to see what takes the memory. However in general I see 2 areas that could be explored for potential improvements:

  1. Image may be causing higher memory usage. Our (developers) tendency is to capture "the best" image. However text recognition may not need a huge high quality image (and give the same results for smaller image). So try to make your image smaller on capturing and/or processing:

    • experiment with AVCapturePhotoSettings, namely photoQualityPrioritization and photo format,
    • try to reduce CGImage size / maybe make it B&W. Those operations are not without the cost as well though. Or try to pass CVPixelBuffer directly to VNImageRequestHandler (without converting to CGImage - that's if you are taking image from camera)
  2. See if autoreleasepool benefits your memory usage. The most obvious location is imageRequestHandler.perform: try to perform it on a dedicated queue, and set autoreleaseFrequency: .workItem for that queue:

private static let performQueue = DispatchQueue(
        label: &quot;your label&quot;,
        qos: .userInitiated,
        autoreleaseFrequency: .workItem, // &lt;-- switching from default .never
        target: .global(qos: .userInitiated) &lt;-- same priority as the one you have
    )

// ...

Self.performQueue.async {
    try imageRequestHandler.perform(textRequest)
}

What does it do:

> The queue configures an autorelease pool before the execution of a block, and releases the objects in that pool after the block finishes executing.

Since the recognition request may have a lot of temporary objects, you may benefit from this setting, since the objects will be immediately released after the request completes (instead of "eventually"). There's no guarantee that it will help, but worth a try.

Again, those are suggestions, but really performance evaluation is needed if reducing memory is that important for you. Although I think 25MB is quite reasonable, to be honest.

huangapple
  • 本文由 发表于 2023年3月20日 23:19:26
  • 转载请务必保留本文链接:https://go.coder-hub.com/75792122.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定