英文:
Is there a way to do background substraction on video of moving cameras? based on optical flow mainly?
问题
-
我的问题是：现在我可以获取视频的密集光流。由于摄像机的旋转，物体和背景同时移动，因此目标和背景都有光流的结果。寻找一种消除背景的方法。
-
我正在使用的摄像机在录制时以固定速度旋转。是否有办法将摄像机的速度与背景的光流相关联？这样我可以根据摄像机的参数来区分背景的光流和前景的光流吗？深度信息是否必要？
-
我使用的摄像机（事件相机）输出二进制图像，这使得难以确定何时目标和背景混在一起。
-
我尝试为所有像素的光流设置了阈值。也就是说，遮挡所有具有相对较低光流值的像素。但问题是，当目标（一个小型无人机）移动缓慢时，这种方法不起作用，而且这种方法无法消除所有背景。
英文:
My problem is: Now i can get the dense optical flow for the video. The object and background are moving at the same time due to the rotation of the camera, therefore both the target and backgroud have OF results. Looking for a way to eliminate backgrounds.
1.The camera I'm using rotates at a fix speed when recording. Is there any way to relate the velocity of the camera to the OF of the background? So i can tell background flow from foreground flow based on the parameter of the camera? Is depth informaton necessary?
2.The camera(event camera) i'm using outputs binary images, which makes it harder to tell when target and background mixed up.
I tried put a threshold for the optical flow for all the pixels. That is, mask off all the pixels that have relatively low OF values. But the problem is when the target(a small drone) is moving slowly, this doesn't work, and this method can't eliminate all backgrounds.
答案1
得分: 1
背景减除 是一类低级算法。不要期望太多。您将不得不执行一些计算量较大的操作。基本上，您需要像处理动作相机一样稳定视频镜头。
假设摄像头和场景之间没有平移，距离就变得无关紧要了。只需补偿旋转，这就少了些麻烦。
您需要跟踪摄像机的旋转/姿态。这个主题称为运动结构或SLAM。光流可以成为其中的一个组成部分。通常，它涉及特征跟踪。可以通过匹配或者对特征点进行稀疏光流来完成。
一个好的方法可能是结合两者（传感器融合）。光流相对低噪声，但当累积时会漂移。特征匹配存在噪声，但不会漂移。
当您知道摄像机的运动方式后，您可以将图像变换为初始（或任何其他）摄像机姿态。通过“摄像机姿态”，我指的是投影。这实际上只是您投影的特定方向。关于这一点的数学常常在“图像拼接”中讨论。
英文:
Background subtraction is a category of low level algorithms. Don't expect too much out of that. You'll have to deploy some computationally expensive operations. You'll basically have to stabilize the footage like it were an action cam.
Assuming no translation between cam and scene, distances become irrelevant. That leaves just rotation to compensate for. That is less of a headache.
You'll have to track the camera rotation/pose. That topic is called Structure from Motion or SLAM. Optical flow can be a component in that. Usually it involves feature tracking. That can be done via matching, or via sparse optical flow on the feature points.
A good approach would probably combine both (sensor fusion). Optical flow is fairly low-noise but when you accumulate it, it'll drift. The feature matching is noisy but doesn't drift.
When you know how your camera moves, you can then warp the image into the initial (or any other) camera pose. By "camera pose" I mean projection. It's really just a specific orientation of your projection. The math for this is commonly discussed in "image stitching".
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论