清除CameraX API中的图像读取缓冲区。

huangapple go评论51阅读模式
英文:

Clear Image Reader Buffer in CameraX API

问题

I am using CameraX API and its ImageAnalysis to analyze each frame. Processing of frames takes some time and imageReader buffer gets filled resulting in this error -

[ImageReader-256x144f23m4-11920-0](id:2e9000000000,api:4,p:651,c:11920) waitForFreeSlotThenRelock: timeout

How can I clear the Image Reader when it gets filled up and carry on processing frames at the same speed.

Here is my analyze Function

        @SuppressLint("UnsafeOptInUsageError")
        override fun analyze(imageProxy: ImageProxy) {
            val image = imageProxy.image

            if (image != null) {
                val inputImage =
                    InputImage.fromMediaImage(image, imageProxy.imageInfo.rotationDegrees)
                processImage(inputImage, imageProxy)
            } else {
                imageProxy.close()
            }
        }

And ProcessImage Function

 @SuppressLint("UnsafeOptInUsageError")
        private fun processImage(image: InputImage, imageProxy: ImageProxy) {
            poseDetector.process(image).addOnSuccessListener { results ->
                posesfromMLKit = getPoseCoordinate(results, imageProxy)
                if (isFullDetectionStarted) {
                    getPoints(posesfromMLKit)
                }
                try {
                    // Convert Image to BitMap
                    val yBuffer = imageProxy.image!!.planes[0].buffer // Y
                    val vuBuffer = imageProxy.image!!.planes[2].buffer // VU

                    val ySize = yBuffer.remaining()
                    val vuSize = vuBuffer.remaining()

                    val nv21 = ByteArray(ySize + vuSize)

                    yBuffer.get(nv21, 0, ySize)
                    vuBuffer.get(nv21, ySize, vuSize)

                    val yuvImage =
                        YuvImage(nv21, ImageFormat.NV21, image!!.width, image!!.height, null)
                    val out = ByteArrayOutputStream()
                    yuvImage.compressToJpeg(Rect(0, 0, yuvImage.width, yuvImage.height), 50, out)
                    val imageBytes = out.toByteArray()
                    base64EncodedFrame = Base64.getEncoder().encodeToString(imageBytes).toString()
                } catch (e: Exception) {
                    imageProxy.close()
                }
                frameratePerSecond++
                imageProxy.close()
            }.addOnFailureListener { e ->
                imageProxy.close()
            }.addOnCompleteListener {
                imageProxy.close()
            }.addOnCanceledListener {
                imageProxy.close()
            }
        }

I tried dropping some frames when processImage function is processing 1 frame. But it resulted in Slow Processing somehow.

but that solved the error, yet it is not a good approach for this project.

英文:

I am using CameraX API and its ImageAnalysis to analyze each frame. Processing of frames takes some time and imageReader buffer gets filled resulting in this error -

[ImageReader-256x144f23m4-11920-0](id:2e9000000000,api:4,p:651,c:11920) waitForFreeSlotThenRelock: timeout

How can I clear the Image Reader when it gets filled up and carry on processing frames at the same speed.

Here is my analyze Function

        @SuppressLint("UnsafeOptInUsageError")
        override fun analyze(imageProxy: ImageProxy) {
            val image = imageProxy.image

            if (image != null) {
                val inputImage =
                    InputImage.fromMediaImage(image, imageProxy.imageInfo.rotationDegrees)
                processImage(inputImage, imageProxy)
            }else {
                imageProxy.close()
            }
        }
       

And ProcessImage Function

 @SuppressLint("UnsafeOptInUsageError")
        private fun processImage(image: InputImage, imageProxy: ImageProxy) {
            poseDetector.process(image).addOnSuccessListener { results ->
                posesfromMLKit = getPoseCoordinate(results, imageProxy)
                if (isFullDetectionStarted) {
                    getPoints(posesfromMLKit)
                }
                try {
                    // Convert Image to BitMap
                    val yBuffer = imageProxy.image!!.planes[0].buffer // Y
                    val vuBuffer = imageProxy.image!!.planes[2].buffer // VU

                    val ySize = yBuffer.remaining()
                    val vuSize = vuBuffer.remaining()

                    val nv21 = ByteArray(ySize + vuSize)

                    yBuffer.get(nv21, 0, ySize)
                    vuBuffer.get(nv21, ySize, vuSize)

                    val yuvImage =
                        YuvImage(nv21, ImageFormat.NV21, image!!.width, image!!.height, null)
                    val out = ByteArrayOutputStream()
                    yuvImage.compressToJpeg(Rect(0, 0, yuvImage.width, yuvImage.height), 50, out)
                    val imageBytes = out.toByteArray()
                    base64EncodedFrame = Base64.getEncoder().encodeToString(imageBytes).toString()
                } catch (e: Exception) {
                    imageProxy.close()
                }
                frameratePerSecond++
                imageProxy.close()
            }.addOnFailureListener { e ->
                imageProxy.close()
            }.addOnCompleteListener {
                imageProxy.close()
//                    CoroutineScope(Dispatchers.IO).launch {
//                        delay(100)
//                        imageProxy.close()
//                    }
            }.addOnCanceledListener {
                imageProxy.close()
            }
        }

I tried dropping some frames when processImage function is processing 1 frame. But it resulted in Slow Processing somehow.

but that solved the error, yet it is not a good approach for this project.

答案1

得分: 0

val options = PoseDetectorOptions.Builder()
.setDetectorMode(PoseDetectorOptions.STREAM_MODE)
.build()

这将确保更快的处理速度,从而获得更快的结果。请确保已设置它。在此处阅读更多信息:https://developers.google.com/ml-kit/vision/pose-detection/android?authuser=1#tips_to_improve_performance

此外,请确保您将setBackpressureStrategy设置为STRATEGY_KEEP_ONLY_LATEST。在此处阅读更多信息:https://developer.android.com/training/camerax/analyze#operating-modes

英文:
val options = PoseDetectorOptions.Builder()
.setDetectorMode(PoseDetectorOptions.STREAM_MODE)
.build()

Would ensure a faster processing hence faster results. Make sure you have set it. Read more here: https://developers.google.com/ml-kit/vision/pose-detection/android?authuser=1#tips_to_improve_performance

Also, make sure the you setBackpressureStrategy to STRATEGY_KEEP_ONLY_LATEST. Read more here: https://developer.android.com/training/camerax/analyze#operating-modes

答案2

得分: 0

The ImageReader queue does not get filled up with the default ImageAnalysis configs. e.g. the STRATEGY_KEEP_ONLY_LATEST mode. With this mode, CameraX keeps dropping incoming frames until the current frame is closed. Could you post your ImageAnalysis configuration?

The resolution is 256x144 which is uncommon for camera. My guess is that it's the queue in MLKit gets filled up.

英文:

The ImageReader queue does not get filled up with the default ImageAnalysis configs. e.g. the STRATEGY_KEEP_ONLY_LATEST mode. With this mode, CameraX keeps dropping incoming frames until the current frame is closed. Could you post your ImageAnalysis configuration?

The resolution is 256x144 which is uncommon for camera. My guess is that it's the queue in MLKit gets filled up.

huangapple
  • 本文由 发表于 2023年6月13日 15:42:44
  • 转载请务必保留本文链接:https://go.coder-hub.com/76462685.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定