Google Vision API检查检测到的条形码是否在区域内。

huangapple go评论41阅读模式
英文:

Barcode Google Vision API check if detected barcode is inside area

问题

我有问题,检测条形码是否在指定的区域内。为了测试,相机源预览和表面视图的大小都是1440x1080,以防止相机和视图之间的缩放。即使我看到二维码不在代表图像的框内,我也会得到正面检查结果。有什么问题吗?

错误的正面检查

Google Vision API检查检测到的条形码是否在区域内。

ScannerActivity

public class ScannerActivity extends AppCompatActivity {
    private static final String TAG = "ScannerActivity";

    private SurfaceView mSurfaceView; // 在XML中,其大小被强制设置为1440x1080
    private CameraSource mCameraSource;
    private ScannerOverlay mScannerOverlay; // 在XML中,其大小被强制设置为1440x1080

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        // .. 创建和初始化视图
        // ...

        BarcodeDetector barcodeDetector = new BarcodeDetector.Builder(this)
                .setBarcodeFormats(Barcode.ALL_FORMATS)
                .build();

        mCameraSource = new CameraSource.Builder(this, barcodeDetector)
                .setRequestedPreviewSize(1440, 1080)
                .setRequestedFps(20.0f)
                .setFacing(CameraSource.CAMERA_FACING_BACK)
                .setAutoFocusEnabled(true)
                .build();

        barcodeDetector.setProcessor(new Detector.Processor<Barcode>() {
            @Override
            public void release() {
            }

            @Override
            public void receiveDetections(Detector.Detections<Barcode> detections) {
                parseDetections(detections.getDetectedItems());
            }
        });

    }

    private void parseDetections(SparseArray<Barcode> barcodes) {
        for (int i = 0; i < barcodes.size(); i++) {
            Barcode barcode = barcodes.valueAt(i);
            if (isInsideBox(barcode)) {
                runOnUiThread(() -> {
                    Toast.makeText(this, "GOT DETECTION: " + barcode.displayValue, Toast.LENGTH_SHORT).show();
                });
            }
        }
    }

    private boolean isInsideBox(Barcode barcode) {
        Rect barcodeBoundingBox = barcode.getBoundingBox();
        Rect scanBoundingBox = mScannerOverlay.getBox();

        boolean checkResult = barcodeBoundingBox.left >= scanBoundingBox.left &&
                barcodeBoundingBox.right <= scanBoundingBox.right &&
                barcodeBoundingBox.top >= scanBoundingBox.top &&
                barcodeBoundingBox.bottom <= scanBoundingBox.bottom;

        Log.d(TAG, "isInsideBox: " + (checkResult ? "YES" : "NO"));
        return checkResult;
    }
}

<details>
<summary>英文:</summary>
I have problem with detecting if barcode is inside specified area. For testing purposes camera source preview and surface view has same size ``1440x1080`` to prevent scaling between camera and view. I get positive checks even if I see QR Code isn&#39;t in box what represents image. Whats wrong?
# False positive check #
[![False positive check][1]][1]
# ScannerActivity #

public class ScannerActivity extends AppCompatActivity {
private static final String TAG = "ScannerActivity";

private SurfaceView mSurfaceView; // Its size is forced to 1440x1080 in XML
private CameraSource mCameraSource;
private ScannerOverlay mScannerOverlay; // Its size is forced to 1440x1080 in XML
@Override
protected void onCreate(Bundle savedInstanceState) {
// .. create and init views
// ...
BarcodeDetector barcodeDetector = new BarcodeDetector.Builder(this)
.setBarcodeFormats(Barcode.ALL_FORMATS)
.build();
mCameraSource = new CameraSource.Builder(this, barcodeDetector)
.setRequestedPreviewSize(1440, 1080)
.setRequestedFps(20.0f)
.setFacing(CameraSource.CAMERA_FACING_BACK)
.setAutoFocusEnabled(true)
.build();
barcodeDetector.setProcessor(new Detector.Processor&lt;Barcode&gt;() {
@Override
public void release() {
}
@Override
public void receiveDetections(Detector.Detections&lt;Barcode&gt; detections) {
parseDetections(detections.getDetectedItems());
}
});
}
private void parseDetections(SparseArray&lt;Barcode&gt; barcodes) {
for (int i = 0; i &lt; barcodes.size(); i++) {
Barcode barcode = barcodes.valueAt(i);
if (isInsideBox(barcode)) {
runOnUiThread(() -&gt; {
Toast.makeText(this, &quot;GOT DETECTION: &quot; + barcode.displayValue, Toast.LENGTH_SHORT).show();
});
}
}
}
private boolean isInsideBox(Barcode barcode) {
Rect barcodeBoundingBox = barcode.getBoundingBox();
Rect scanBoundingBox = mScannerOverlay.getBox();
boolean checkResult = barcodeBoundingBox.left &gt;= scanBoundingBox.left &amp;&amp;
barcodeBoundingBox.right &lt;= scanBoundingBox.right &amp;&amp;
barcodeBoundingBox.top &gt;= scanBoundingBox.top &amp;&amp;
barcodeBoundingBox.bottom &lt;= scanBoundingBox.bottom;
Log.d(TAG, &quot;isInsideBox: &quot;+(checkResult ? &quot;YES&quot; : &quot;NO&quot;));
return checkResult;
}

}


[1]: https://i.stack.imgur.com/OfNi6.png
</details>
# 答案1
**得分**: 1
问题的解释很简单,但解决方案不容易解释。
来自您用户界面的框的坐标通常不会与每个预览帧上的虚拟框相同。您必须将坐标从用户界面框转换为scanBoundingBox。
我开源了一个[示例][1],实现了您尝试完成的相同用例。在这个示例中,我采用了另一种方法,首先从每个帧中剪切框,然后再将其提供给Google Vision,这也更高效,因为Google Vision不必分析整个图像并浪费大量CPU...
[1]: https://github.com/minkiapps/Firebase-ML-Kit-Scanner-Demo
<details>
<summary>英文:</summary>
Explanation to your issue is simple, but the solution is not trivial to explain.
The coordinates of the box from your UI will mostly not the be the same like the imaginary box on each preview frame. You must transform the coordinates from the UI box to scanBoundingBox. 
I open sourced an [example][1] which implement the same usecase you are trying to accomplish. In this example I took another approach, I cut the box out of each frame first before feeding it to Google Vision, which is also more efficient, since Google Vision don&#39;t have to analyse the whole picture and waste tons of CPU...
[1]: https://github.com/minkiapps/Firebase-ML-Kit-Scanner-Demo
</details>
# 答案2
**得分**: 0
我决定通过包装条形码检测器来裁剪帧,但我不知道为什么裁剪后的帧会旋转90度,即使智能手机是竖直方向。
## Box Detector 类 ##
```java
public class BoxDetector extends Detector<Barcode> {
private Detector<Barcode> mDelegate;
private int mBoxWidth;
private int mBoxHeight;
// 调试信息
private CroppedFrameListener mCroppedFrameListener;
public BoxDetector(Detector<Barcode> delegate, int boxWidth, int boxHeight) {
mDelegate = delegate;
mBoxWidth = boxWidth;
mBoxHeight = boxHeight;
}
public void setCroppedFrameListener(CroppedFrameListener croppedFrameListener) {
mCroppedFrameListener = croppedFrameListener;
}
@Override
public SparseArray<Barcode> detect(Frame frame) {
int frameWidth = frame.getMetadata().getWidth();
int frameHeight = frame.getMetadata().getHeight();
// 我假设框是居中的。
int left = (frameWidth / 2) - (mBoxWidth / 2);
int top = (frameHeight / 2) - (mBoxHeight / 2);
int right = (frameWidth / 2) + (mBoxWidth / 2);
int bottom = (frameHeight / 2) + (mBoxHeight / 2);
YuvImage yuvImage = new YuvImage(frame.getGrayscaleImageData().array(), ImageFormat.NV21, frameWidth, frameHeight, null);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
yuvImage.compressToJpeg(new Rect(left, top, right, bottom), 100, outputStream);
byte[] jpegArray = outputStream.toByteArray();
Bitmap bitmap = BitmapFactory.decodeByteArray(jpegArray, 0, jpegArray.length);
Frame croppedFrame = new Frame.Builder()
.setBitmap(bitmap)
.setRotation(frame.getMetadata().getRotation())
.build();
if (mCroppedFrameListener != null) {
mCroppedFrameListener.onNewCroppedFrame(croppedFrame.getBitmap(), croppedFrame.getMetadata().getRotation());
}
return mDelegate.detect(croppedFrame);
}
public interface CroppedFrameListener {
void onNewCroppedFrame(Bitmap bitmap, int rotation);
}
}

Box Detector 使用

BarcodeDetector barcodeDetector = new BarcodeDetector.Builder(this)
        .setBarcodeFormats(Barcode.ALL_FORMATS)
        .build();

BoxDetector boxDetector = new BoxDetector(
        barcodeDetector,
        mBoxSize.getWidth(),
        mBoxSize.getHeight());

boxDetector.setCroppedFrameListener(new BoxDetector.CroppedFrameListener() {
    @Override
    public void onNewCroppedFrame(final Bitmap bitmap, int rotation) {
        Log.d(TAG, "onNewCroppedFrame: new bitmap, rotation: " + rotation);
        runOnUiThread(new Runnable() {
            @Override
            public void run() {
                mPreview.setImageBitmap(bitmap);
            }
        });
    }
});

裁剪后的帧被旋转

Google Vision API检查检测到的条形码是否在区域内。

英文:

I decied to cropp frame by wrapping barcode detector however I don't know why but cropped frame is rotated by 90 degress even smarphone is upright orientation.

Box Detector class

public class BoxDetector extends Detector&lt;Barcode&gt; {
private Detector&lt;Barcode&gt; mDelegate;
private int mBoxWidth;
private int mBoxHeight;
// Debugging
private CroppedFrameListener mCroppedFrameListener;
public BoxDetector(Detector&lt;Barcode&gt; delegate, int boxWidth, int boxHeight) {
mDelegate = delegate;
mBoxWidth = boxWidth;
mBoxHeight = boxHeight;
}
public void setCroppedFrameListener(CroppedFrameListener croppedFrameListener) {
mCroppedFrameListener = croppedFrameListener;
}
@Override
public SparseArray&lt;Barcode&gt; detect(Frame frame) {
int frameWidth = frame.getMetadata().getWidth();
int frameHeight = frame.getMetadata().getHeight();
// I assume that box is centered.
int left = (frameWidth / 2) - (mBoxWidth / 2);
int top = (frameHeight / 2) - (mBoxHeight / 2);
int right = (frameWidth / 2) + (mBoxWidth / 2);
int bottom = (frameHeight / 2) + (mBoxHeight / 2);
YuvImage yuvImage = new YuvImage(frame.getGrayscaleImageData().array(), ImageFormat.NV21, frameWidth, frameHeight, null);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
yuvImage.compressToJpeg(new Rect(left, top, right, bottom), 100, outputStream);
byte[] jpegArray = outputStream.toByteArray();
Bitmap bitmap = BitmapFactory.decodeByteArray(jpegArray, 0, jpegArray.length);
Frame croppedFrame = new Frame.Builder()
.setBitmap(bitmap)
.setRotation(frame.getMetadata().getRotation())
.build();
if(mCroppedFrameListener != null) {
mCroppedFrameListener.onNewCroppedFrame(croppedFrame.getBitmap(), croppedFrame.getMetadata().getRotation());
}
return mDelegate.detect(croppedFrame);
}
public interface CroppedFrameListener {
void onNewCroppedFrame(Bitmap bitmap, int rotation);
}
}

Box Detector usuage

        BarcodeDetector barcodeDetector = new BarcodeDetector.Builder(this)
.setBarcodeFormats(Barcode.ALL_FORMATS)
.build();
BoxDetector boxDetector = new BoxDetector(
barcodeDetector,
mBoxSize.getWidth(),
mBoxSize.getHeight());
boxDetector.setCroppedFrameListener(new BoxDetector.CroppedFrameListener() {
@Override
public void onNewCroppedFrame(final Bitmap bitmap, int rotation) {
Log.d(TAG, &quot;onNewCroppedFrame: new bitmap, rotation: &quot;+rotation);
runOnUiThread(new Runnable() {
@Override
public void run() {
mPreview.setImageBitmap(bitmap);
}
});
}
});

Cropped frame is rotated

Google Vision API检查检测到的条形码是否在区域内。

huangapple
  • 本文由 发表于 2020年7月22日 19:46:58
  • 转载请务必保留本文链接:https://go.coder-hub.com/63033458.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定