Opencv SeamFinder 返回斑块状的遮罩。

huangapple go评论111阅读模式
英文:

Image Stitching: Opencv SeamFinder returns patchy masks

问题

I have a setup of 6 cameras (60° horizontal and vertical FoV) with the same center looking outwards with a horizontal rotation of 30 degrees. I am projecting the images onto a sphere and the cameras are located in the center of which, see sketch below.

我的摄像机设置包括6台摄像机(水平和垂直视场角均为60°),它们都朝外以30度的水平旋转。我将图像投影到一个球面上,这些摄像机位于球面的中心,如下图所示:

Opencv SeamFinder 返回斑块状的遮罩。

I am stitching the images together close to this diagram from opencv, but because I know the orientation of the cameras I skip the "registration" part and (for now) omit the blending of the images.

我正在使用OpenCV将图像拼接在一起,与该图示接近,但因为我了解摄像机的方向,所以我跳过了“配准”部分,并(暂时)省略了图像的混合。

Opencv SeamFinder 返回斑块状的遮罩。

My problem is, that the masks returned by the SeamFinder are nonsense. I think it should work, because if I use NoSeamfinder in the simulation the results look good, but I tried every other Seamfinder OpenCV provides and the results are as shown below:

我的问题是,由SeamFinder返回的掩模是无意义的。我认为它应该能够正常工作,因为在模拟中使用NoSeamfinder时结果看起来很好,但我尝试了OpenCV提供的每种其他Seamfinder,结果如下所示:

Opencv SeamFinder 返回斑块状的遮罩。

I have worked with the stitching_detailed.cpp example from opencv to set up the pipeline but I have no idea what might be the issue. I thought that maybe, because I am working within a simulation, the gradients in the images are too sharp to work with, so I applied a Gaussian blur, which did not change anything.

我已经使用OpenCV的stitching_detailed.cpp示例来建立流程,但我不知道问题可能出在哪里。我认为可能是因为我在模拟环境中工作,图像中的梯度太陡峭,所以我应用了高斯模糊,但这并没有改变什么。

I work with cv::Mat in the code and transform them to cv::UMat for the SeamFinder.

我在代码中使用cv::Mat,并将它们转换为cv::UMat以供SeamFinder使用。


<details>
<summary>英文:</summary>

I have a setup of 6 cameras (60&#176; horizontal and vertical FoV) with the same center looking outwards with a horizontal rotation of 30 degrees. I am projecting the images onto a sphere and the cameras are located in the center of which, see sketch below.

[![simplified camera positioning sketch, look from on top of sphere](https://i.stack.imgur.com/pf2Rd.png)](https://i.stack.imgur.com/pf2Rd.png)

I am stitching the images together close to this diagram from opencv, but because I know the orientation of the cameras I skip the &quot;registration&quot; part and (for now) omit the blending of the images.

[![image stitching pipeline from opencv](https://i.stack.imgur.com/V50sM.png)](https://i.stack.imgur.com/V50sM.png)

My problem is, that the masks returned by the SeamFinder are nonsense. I think it should work, because if I use `NoSeamfinder` in the simulation the results look good, but I tried every other Seamfinder OpenCV provides and the results are as shown below:
[![SeamFinder Results](https://i.stack.imgur.com/mWjvQ.jpg)](https://i.stack.imgur.com/mWjvQ.jpg)

I have worked with the stitching_detailed.cpp example from opencv to set up the pipeline but I have no idea what might be the issue. I thought that maybe, because I am working within a simulation, the gradients in the images are too sharp to work with, so I applied a Gaussian blur, which did not change anything.

I have some code snippets below and can post more on demand, but I want to avoid just dumping a whole load here.

I work with `cv::Mat` in the code and transform them to `cv::UMat` for the SeamFinder.

std::vector<cv::Mat> warped_images_mat; // the single images are warped according to the sphere
std::vector<cv::UMat> warped_images_umat;

std::vector<cv::Mat> warped_masks_mat; // Masks of the warped images within the sphere image
std::vector<cv::UMat> warped_masks_umat;

std::vector<cv::Mat> warped_masks_fixed_mat; // These masks are calculated once and used to refresh the masks for the SeamFinder
std::vector<cv::UMat> warped_masks_fixed_umat;

std::vector<cv::Point> corners; // position of the imgs and masks in the sphere image

cv::Ptr<cv::detail::SeamFinder> seam_finder = cv::makePtr<cv::detail::GraphCutSeamFinder>(cv::detail::GraphCutSeamFinderBase::COST_COLOR_GRAD);

// within the ROS2 callback:

this->warped_masks_umat.clear(); // clear the old masks
for(size_t cam_idx = 0; cam_idx < 6; cam_idx++){
int cols = this->warped_images_mat.at(cam_idx).cols;
int rows = this->warped_images_mat.at(cam_idx).rows;

    // std::cout &lt;&lt; &quot;rows: &quot; &lt;&lt; rows &lt;&lt; &quot;; cols: &quot; &lt;&lt; cols &lt;&lt; std::endl;
    for (int col = 0; col &lt; cols; col++)
        for (int row = 0; row &lt; rows; row++)
            if (this-&gt;warped_masks_fixed_mat.at(cam_idx).at&lt;u_char&gt;(row, col) != 0)
                // code to refresh warped image
    this-&gt;warped_images_mat.at(cam_idx).getUMat(cv::ACCESS_RW).convertTo(this-&gt;warped_images_umat.at(cam_idx), CV_32F);
    this-&gt;warped_masks_umat.push_back(this-&gt;warped_masks_fixed_umat.at(cam_idx).clone());

}

seam_finder->find(this->warped_images_umat, this->corners, this->warped_masks_umat);


</details>


# 答案1
**得分**: 0

我混淆了OpenCV中的符号。`cv::Point` 中的 `.x` 和 `.y` 被交换了。用于刷新传入图像时它们是这样的,但对于 `SeamFinder` 来说则是相反的。

当我尝试重现收到的评论中的示例时,我注意到了掩膜中的一个独特形状。现在它也可以在完整的掩膜上工作,而不需要裁剪任何内容。

<details>
<summary>英文:</summary>

I mixed up the notations from opencv. The `.x` and `.y` from `cv::Point` were exchanged. It worked like this for refreshing the incoming images but were the wrong way round for the `SeamFinder`.

I noticed this when I tried to reproduce an example from the comment I got and noticed a distinctive shape in the masks. It works now also with the full masks, without cropping anything.

</details>



huangapple
  • 本文由 发表于 2023年3月9日 19:17:55
  • 转载请务必保留本文链接:https://go.coder-hub.com/75683870.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定