为什么使用模板匹配进行目标检测的代码不能成功运行?

huangapple go评论68阅读模式
英文:

Why is this code of Object detection using Template Matching does not run successfully

问题

我得到的输出只有参考图像,输出中没有看到边界框。

我已经尝试了来自这个网站的代码: https://www.sicara.fr/blog-technique/object-detection-template-matching

这是参考图像:
参考图像

这是模板:
1:

为什么使用模板匹配进行目标检测的代码不能成功运行?

模板2:

为什么使用模板匹配进行目标检测的代码不能成功运行?

模板3:

为什么使用模板匹配进行目标检测的代码不能成功运行?

与网站相比,使用该代码,输出应该看起来像这样:

期望的输出:

为什么使用模板匹配进行目标检测的代码不能成功运行?

我期望得到像网站上讨论的输出,然而,当我尝试运行这段代码时,似乎什么都没有被检测到。这是我复制的代码:

import cv2
import numpy as np

DEFAULT_TEMPLATE_MATCHING_THRESHOLD = 0.9

class Template:
    """
    定义模板的类
    """

    def __init__(self, image_path, label, color, matching_threshold=DEFAULT_TEMPLATE_MATCHING_THRESHOLD):
        """
        Args:
            image_path (str): 模板图像路径
            label (str): 与模板相关的标签
            color (List[int]): 与标签关联的颜色(用于绘制检测结果)
            matching_threshold (float): 考虑对象被模板匹配检测的最低相似度分数
        """
        self.image_path = image_path
        self.label = label
        self.color = color
        self.template = cv2.imread(image_path)
        self.template_height, self.template_width = self.template.shape[:2]
        self.matching_threshold = matching_threshold

image = cv2.imread("reference.jpg")

templates = [
    Template(image_path="Component1.jpg", label="1", color=(0, 0, 255), matching_threshold=0.99),
    Template(image_path="Component2.jpg", label="2", color=(0, 255, 0,), matching_threshold=0.91),
    Template(image_path="Component3.jpg", label="3", color=(0, 191, 255), matching_threshold=0.99),
]

detections = []
for template in templates:
    template_matching = cv2.matchTemplate(template.template, image, cv2.TM_CCORR_NORMED)
    match_locations = np.where(template_matching >= template.matching_threshold)

    for (x, y) in zip(match_locations[1], match_locations[0]):
        match = {
            "TOP_LEFT_X": x,
            "TOP_LEFT_Y": y,
            "BOTTOM_RIGHT_X": x + template.template_width,
            "BOTTOM_RIGHT_Y": y + template.template_height,
            "MATCH_VALUE": template_matching[y, x],
            "LABEL": template.label,
            "COLOR": template.color
        }
        detections.append(match)

def compute_iou(boxA, boxB):
    xA = max(boxA["TOP_LEFT_X"], boxB["TOP_LEFT_X"])
    yA = max(boxA["TOP_LEFT_Y"], boxB["TOP_LEFT_Y"])
    xB = min(boxA["BOTTOM_RIGHT_X"], boxB["BOTTOM_RIGHT_X"])
    yB = min(boxA["BOTTOM_RIGHT_Y"], boxB["BOTTOM_RIGHT_Y"])
    interArea = max(0, xB - xA + 1) * max(0, yB - yA + 1)
    boxAArea = (boxA["BOTTOM_RIGHT_X"] - boxA["TOP_LEFT_X"] + 1) * (boxA["BOTTOM_RIGHT_Y"] - boxA["TOP_LEFT_Y"] + 1)
    boxBArea = (boxB["BOTTOM_RIGHT_X"] - boxB["TOP_LEFT_X"] + 1) * (boxB["BOTTOM_RIGHT_Y"] - boxB["TOP_LEFT_Y"] + 1)
    iou = interArea / float(boxAArea + boxBArea - interArea)
    return iou

def non_max_suppression(objects, non_max_suppression_threshold=0.5, score_key="MATCH_VALUE"):
    sorted_objects = sorted(objects, key=lambda obj: obj[score_key], reverse=True)
    filtered_objects = []
    for object_ in sorted_objects:
        overlap_found = False
        for filtered_object in filtered_objects:
            iou = compute_iou(object_, filtered_object)
            if iou > non_max_suppression_threshold:
                overlap_found = True
                break
        if not overlap_found:
            filtered_objects.append(object_)
    return filtered_objects

NMS_THRESHOLD = 0.2
detections = non_max_suppression(detections, non_max_suppression_threshold=NMS_THRESHOLD)
image_with_detections = image.copy()

for detection in detections:
    cv2.rectangle(
        image_with_detections,
        (detection["TOP_LEFT_X"], detection["TOP_LEFT_Y"]),
        (detection["BOTTOM_RIGHT_X"], detection["BOTTOM_RIGHT_Y"]),
        detection["COLOR"],
        2,
    )
    cv2.putText(
        image_with_detections,
        f"{detection['LABEL']} - {detection['MATCH_VALUE']}",
        (detection["TOP_LEFT_X"] + 2, detection["TOP_LEFT_Y"] + 20),
        cv2.FONT_HERSHEY_SIMPLEX, 0.5,
        detection["COLOR"], 1,
        cv2.LINE_AA,
    )

print("Image written to file-system: ", status)
cv2.imshow("res", image_with_detections)
cv2.waitKey(0)

这是他的最终输出:
5

这是我在检测较大的组件时的尝试,代码能够检测到它们,这是结果:
结果

这是我想要检测的调整后的模板和原始组件,但不幸的是不能:

第一 第二 第三

英文:

The output that I get is just the reference image and no bounding box is seen in the output.
I have tried this code from this website: https://www.sicara.fr/blog-technique/object-detection-template-matching

Here's the reference image
Reference Image

Here are the templates:
1:

为什么使用模板匹配进行目标检测的代码不能成功运行?

templates 2:

为什么使用模板匹配进行目标检测的代码不能成功运行?

templates 3:

为什么使用模板匹配进行目标检测的代码不能成功运行?

As compared to the website, using the code the output should look like this:

Expected Output:

为什么使用模板匹配进行目标检测的代码不能成功运行?

I am expecting to have this output as discussed in the website, however, when I tried to run this code, nothing seems to be detected. Here is the code that I copied:

import cv2
import numpy as np
DEFAULT_TEMPLATE_MATCHING_THRESHOLD = 0.9
class Template:
"""
A class defining a template
"""
def __init__(self, image_path, label, color, matching_threshold=DEFAULT_TEMPLATE_MATCHING_THRESHOLD):
"""
Args:
image_path (str): path of the template image path
label (str): the label corresponding to the template
color (List[int]): the color associated with the label (to plot detections)
matching_threshold (float): the minimum similarity score to consider an object is detected by template
matching
"""
self.image_path = image_path
self.label = label
self.color = color
self.template = cv2.imread(image_path)
self.template_height, self.template_width = self.template.shape[:2]
self.matching_threshold = matching_threshold
image = cv2.imread("reference.jpg")
templates = [
Template(image_path="Component1.jpg", label="1", color=(0, 0, 255), matching_threshold=0.99),
Template(image_path="Component2.jpg", label="2", color=(0, 255, 0,) , matching_threshold=0.91),
Template(image_path="Component3.jpg", label="3", color=(0, 191, 255), matching_threshold=0.99),
detections = []
for template in templates:
template_matching = cv2.matchTemplate(template.template, image, cv2.TM_CCORR_NORMED)
match_locations = np.where(template_matching >= template.matching_threshold)
for (x, y) in zip(match_locations[1], match_locations[0]):
match = {
"TOP_LEFT_X": x,
"TOP_LEFT_Y": y,
"BOTTOM_RIGHT_X": x + template.template_width,
"BOTTOM_RIGHT_Y": y + template.template_height,
"MATCH_VALUE": template_matching[y, x],
"LABEL": template.label,
"COLOR": template.color
}
detections.append(match)
def compute_iou(boxA, boxB):
xA = max(boxA["TOP_LEFT_X"], boxB["TOP_LEFT_X"])
yA = max(boxA["TOP_LEFT_Y"], boxB["TOP_LEFT_Y"])
xB = min(boxA["BOTTOM_RIGHT_X"], boxB["BOTTOM_RIGHT_X"])
yB = min(boxA["BOTTOM_RIGHT_Y"], boxB["BOTTOM_RIGHT_Y"])
interArea = max(0, xB - xA + 1) * max(0, yB - yA + 1)
boxAArea = (boxA["BOTTOM_RIGHT_X"] - boxA["TOP_LEFT_X"] + 1) * (boxA["BOTTOM_RIGHT_Y"] - boxA["TOP_LEFT_Y"] + 1)
boxBArea = (boxB["BOTTOM_RIGHT_X"] - boxB["TOP_LEFT_X"] + 1) * (boxB["BOTTOM_RIGHT_Y"] - boxB["TOP_LEFT_Y"] + 1)
iou = interArea / float(boxAArea + boxBArea - interArea)
return iou
def non_max_suppression(objects, non_max_suppression_threshold=0.5, score_key="MATCH_VALUE"):
"""
Filter objects overlapping with IoU over threshold by keeping only the one with maximum score.
Args:
objects (List[dict]): a list of objects dictionaries, with:
{score_key} (float): the object score
{top_left_x} (float): the top-left x-axis coordinate of the object bounding box
{top_left_y} (float): the top-left y-axis coordinate of the object bounding box
{bottom_right_x} (float): the bottom-right x-axis coordinate of the object bounding box
{bottom_right_y} (float): the bottom-right y-axis coordinate of the object bounding box
non_max_suppression_threshold (float): the minimum IoU value used to filter overlapping boxes when
conducting non-max suppression.
score_key (str): score key in objects dicts
Returns:
List[dict]: the filtered list of dictionaries.
"""
sorted_objects = sorted(objects, key=lambda obj: obj[score_key], reverse=True)
filtered_objects = []
for object_ in sorted_objects:
overlap_found = False
for filtered_object in filtered_objects:
iou = compute_iou(object_, filtered_object)
if iou > non_max_suppression_threshold:
overlap_found = True
break
if not overlap_found:
filtered_objects.append(object_)
return filtered_objects
NMS_THRESHOLD = 0.2
detections = non_max_suppression(detections, non_max_suppression_threshold=NMS_THRESHOLD)
image_with_detections = image.copy()
for detection in detections:
cv2.rectangle(
image_with_detections,
(detection["TOP_LEFT_X"], detection["TOP_LEFT_Y"]),
(detection["BOTTOM_RIGHT_X"], detection["BOTTOM_RIGHT_Y"]),
detection["COLOR"],
2,
)
cv2.putText(
image_with_detections,
f"{detection['LABEL']} - {detection['MATCH_VALUE']}",
(detection["TOP_LEFT_X"] + 2, detection["TOP_LEFT_Y"] + 20),
cv2.FONT_HERSHEY_SIMPLEX, 0.5,
detection["COLOR"], 1,
cv2.LINE_AA,
)
# NMS_THRESHOLD = 0.2
# detection = non_max_suppression(detections, non_max_suppression_threshold=NMS_THRESHOLD)
print("Image written to file-system: ", status)
cv2.imshow("res", image_with_detections)
cv2.waitKey(0)

this is how his final output looks like:
5

Here's my attempt in detecting the larger components, the code was able to detect them and here is the result:
Result

Here are the resize templates and the original components that I wanted to detect but unfortunately can't:

1st 2nd 3rd

答案1

得分: 0

以下是使用Python/OpenCV在模板匹配中查找多个匹配项的方法,使用了您的参考图像和最小模板。我已经删除了您在模板周围的所有白色填充。我的方法简单地在匹配的相关图像上绘制一个黑色矩形,然后在修改后的相关图像中继续寻找下一个最佳匹配项。

我使用了cv2.TM_CCORR_NORMED和匹配阈值为0.90。您的参考图像中有4个这样的模板,所以我将我的搜索数量设置为4,并设置了10的间隔来进行非极大值抑制。您还有其他形状和大小相同的小物品,但上面的文字不同。因此,您需要为每个不同的模板。

import cv2
import numpy as np

# 读取图像
img = cv2.imread('circuit_board.jpg')

# 读取模板
tmplt = cv2.imread('circuit_item.png')
hh, ww, cc = tmplt.shape

# 设置参数
match_thresh = 0.90               # 停止匹配值的阈值
num_matches = 4                   # 停止匹配的数量阈值
match_radius = 10                 # 匹配峰值的近似半径
match_radius2 = match_radius//2

# 从模板匹配中获取相关性图像
corrimg = cv2.matchTemplate(img,tmplt,cv2.TM_CCORR_NORMED)
hc, wc = corrimg.shape

# 获取所有峰值的位置,它们的值高于match_thresh,最多匹配num_matches次
imgcopy = img.copy()
corrcopy = corrimg.copy()
for i in range(0, num_matches):
    # 获取最大值和最大值的位置
    min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(corrcopy)
    x1 = max_loc[0]
    y1 = max_loc[1]
    x2 = x1 + ww
    y2 = y1 + hh
    loc = str(x1) + "," + str(y1)
    if max_val > match_thresh:
        print("匹配次数:", i+1, "匹配值:", max_val, "匹配位置:", loc)
        # 画出白色边界框以定义匹配位置
        cv2.rectangle(imgcopy, (x1,y1), (x2,y2), (255,255,255), 1)
        # 在相关图像的副本上插入黑色矩形,以免再次找到该匹配项
        corrcopy[y1-match_radius2:y1+match_radius2, x1-match_radius2:x1+match_radius2] = 0
        i = i + 1
    else:
        break

# 保存结果
# 对相关图像进行4次幂运算,以强调峰值
cv2.imwrite('circuit_board_multi_template_corr.png', (255*cv2.pow(corrimg,4)).clip(0,255).astype(np.uint8))
cv2.imwrite('circuit_board_multi_template_corr_masked.png', (255*cv2.pow(corrcopy,4)).clip(0,255).astype(np.uint8))
cv2.imwrite('circuit_board_multi_template_match.png', imgcopy)

# 显示结果
# 对相关图像进行4次幂运算,以强调峰值
cv2.imshow('图像', img)
cv2.imshow('模板', tmplt)
cv2.imshow('相关性', cv2.pow(corrimg,4))
cv2.imshow('相关性遮罩', cv2.pow(corrcopy,4))
cv2.imshow('结果', imgcopy)
cv2.waitKey(0)
cv2.destroyAllWindows()

原始相关图像:

为什么使用模板匹配进行目标检测的代码不能成功运行?

经过4次匹配后的修改后的相关图像:

为什么使用模板匹配进行目标检测的代码不能成功运行?

在输入上标记的匹配项为白色矩形:

为什么使用模板匹配进行目标检测的代码不能成功运行?

匹配位置:

匹配次数: 1 匹配值: 0.9982172250747681 匹配位置: 128,68
匹配次数: 2 匹配值: 0.9762057065963745 匹配位置: 128,90
匹配次数: 3 匹配值: 0.9755787253379822 匹配位置: 128,48
匹配次数: 4 匹配值: 0.963689923286438 匹配位置: 127,107
英文:

Here is a method of finding multiple matches in template matching in Python/OpenCV using your reference and smallest template. I have remove all the white padding you had around your template. My method simply draws a black rectangle over the correlation image where it matches and then repeats looking for the next best match in the modified correlation image.

I have used cv2.TM_CCORR_NORMED and a match threshold of 0.90. You have 4 of these templates showing in your reference image, so I set my search number to 4 and spacing of 10 for the non-maximum suppression by masking. You have other small items of the same shape and size, but the text on them is different. So you will need different templates for each.

Reference:

为什么使用模板匹配进行目标检测的代码不能成功运行?

Template:

为什么使用模板匹配进行目标检测的代码不能成功运行?

import cv2
import numpy as np
# read image
img = cv2.imread('circuit_board.jpg')
# read template
tmplt = cv2.imread('circuit_item.png')
hh, ww, cc = tmplt.shape
# set arguments
match_thresh = 0.90               # stopping threshold for match value
num_matches = 4                   # stopping threshold for number of matches
match_radius = 10                 # approx radius of match peaks
match_radius2 = match_radius//2
# get correlation surface from template matching
corrimg = cv2.matchTemplate(img,tmplt,cv2.TM_CCORR_NORMED)
hc, wc = corrimg.shape
# get locations of all peaks higher than match_thresh for up to num_matches
imgcopy = img.copy()
corrcopy = corrimg.copy()
for i in range(0, num_matches):
# get max value and location of max
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(corrcopy)
x1 = max_loc[0]
y1 = max_loc[1]
x2 = x1 + ww
y2 = y1 + hh
loc = str(x1) + "," + str(y1)
if max_val > match_thresh:
print("match number:", i+1, "match value:", max_val, "match x,y:", loc)
# draw draw white bounding box to define match location
cv2.rectangle(imgcopy, (x1,y1), (x2,y2), (255,255,255), 1)
# insert black rectangle over copy of corr image so not find that match again
corrcopy[y1-match_radius2:y1+match_radius2, x1-match_radius2:x1+match_radius2] = 0
i = i + 1
else:
break
# save results
# power of 4 exaggeration of correlation image to emphasize peaks
cv2.imwrite('circuit_board_multi_template_corr.png', (255*cv2.pow(corrimg,4)).clip(0,255).astype(np.uint8))
cv2.imwrite('circuit_board_multi_template_corr_masked.png', (255*cv2.pow(corrcopy,4)).clip(0,255).astype(np.uint8))
cv2.imwrite('circuit_board_multi_template_match.png', imgcopy)
# show results
# power of 4 exaggeration of correlation image to emphasize peaks
cv2.imshow('image', img)
cv2.imshow('template', tmplt)
cv2.imshow('corr', cv2.pow(corrimg,4))
cv2.imshow('corr masked', cv2.pow(corrcopy,4))
cv2.imshow('result', imgcopy)
cv2.waitKey(0)
cv2.destroyAllWindows()

Original Correlation Image:

为什么使用模板匹配进行目标检测的代码不能成功运行?

Modified Correlation Image after 4 matches:

为什么使用模板匹配进行目标检测的代码不能成功运行?

Matches Marked on Input as White Rectangles:

为什么使用模板匹配进行目标检测的代码不能成功运行?

Match Locations:

match number: 1 match value: 0.9982172250747681 match x,y: 128,68
match number: 2 match value: 0.9762057065963745 match x,y: 128,90
match number: 3 match value: 0.9755787253379822 match x,y: 128,48
match number: 4 match value: 0.963689923286438 match x,y: 127,107

huangapple
  • 本文由 发表于 2023年2月8日 14:05:45
  • 转载请务必保留本文链接:https://go.coder-hub.com/75381916.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定