在TensorFlow中画一条线。

huangapple go评论58阅读模式
英文:

draw a line in tensorflow

问题

我想创建一个人体姿势骨架估计网络,为此,我有一个两部分的网络,第一部分生成16个热图作为输出(每个热图对应不同的关节,因此可以提取关键点),我希望使用这16个关键点创建一个人体骨架并将其馈送到网络的第二半部分。我的问题是,如何在关键点之间绘制线条以创建骨架?我无法找到在tensorflow或keras上对张量对象执行此操作的方法。

英文:

I want to create a human pose skeleton estimation network and for this, I have a two-part network, first part generates 16 heatmaps as output(each heatmap for different joint and hence a key point can be extracted), using these 16 key points I wish to create a human skeleton and feed it to second half of my network. My problem is, how do I draw lines between the key points to create the skeleton? I couldn't find a way to do it on a tensor object using tensorflow or keras.

答案1

得分: 1

以下是您提供的代码的翻译部分:

我知道我有点晚了但这里有一些代码我认为可以实现您想要的功能在TFv2.3)。希望将来可以帮助到某人

它仅使用了TensorFlow操作因此您可以在数据加载器等地方使用它这里真正的痛点是Tensorflow不允许即时分配”,因此无法通过索引来更新张量此代码通过创建两个稀疏张量来解决这个问题一个用于掩码”(表示要应用的线位置),另一个用于new_values”(表示要应用在线上的值)。关于设计线的代码可能在您的情况下不适用基于https://stackoverflow.com/a/47381058),但已经从numpy移植过来

import tensorflow as tf

def trapez(y, y0, w):
return tf.clip_by_value(tf.minimum(y + 1 + w/2 - y0, -y + 1 + w/2 + y0), 0, 1)

def apply_output(img, yy, xx, val):
stack = tf.stack([yy, xx], axis=1)
stack = tf.cast(stack, tf.int64)
values = tf.ones(stack.shape[0], tf.float32)
mask = tf.sparse.SparseTensor(indices=stack, values=values, dense_shape=img.shape)
mask = tf.sparse.reorder(mask)
mask = tf.sparse.to_dense(mask)
mask = tf.cast(mask, tf.float32)

new_values = tf.sparse.SparseTensor(indices=stack, values=val, dense_shape=img.shape)
new_values = tf.sparse.reorder(new_values)
new_values = tf.sparse.to_dense(new_values)

img = img * (1 - mask) + new_values * mask
img = tf.cast(tf.expand_dims(img * 255, axis=-1), tf.uint8)
return img

def weighted_line(img, r0, c0, r1, c1, w):
output = img
x = tf.range(c0, c1 + 1, dtype=tf.float32)
slope = (r1-r0) / (c1-c0)
w = tf.sqrt(1 + tf.abs(slope)) / 2
y = x * slope + (c1
r0-c0*r1) / (c1-c0)

thickness = tf.math.ceil(w/2)
yy = (tf.reshape(tf.math.floor(y), [-1, 1]) + tf.reshape(tf.range(-thickness-1, thickness+2), [1, -1]))
xx = tf.repeat(x, yy.shape[1])
values = tf.reshape(trapez(yy, tf.reshape(y, [-1, 1]), w), [-1])
yy = tf.reshape(yy, [-1])

limits_y = tf.math.logical_and(yy >= 0, yy < img.shape[0])
limits_x = tf.math.logical_and(xx >= 0, xx < img.shape[1])
limits = tf.math.logical_and(limits_y, limits_x)
limits = tf.math.logical_and(limits, values > 0)
yy = tf.cast(yy[limits], tf.float32)
xx = tf.cast(xx[limits], tf.float32)

return yy, xx, values[limits], apply_output(output, yy, xx, values[limits])

只是为了进行健全性检查,您可以使用以下方式调用它,并使用OpenCV显示它:

if name == "main":
IMG = tf.zeros((500, 500), tf.float32)
yy, xx, vals, FINAL_IMG = weighted_line(IMG, 10, 20, 100, 200, 5)
jpeg_string = tf.io.encode_jpeg(FINAL_IMG)
tf.io.write_file("output.jpg", jpeg_string)
import cv2
img = cv2.imread("output.jpg")
cv2.imshow("Output", img)
cv2.waitKey(0)

希望这能帮助到您!如果您有其他翻译需求,请告诉我。

英文:

I know i'm a bit late but here is some code that I think does what you're after (in TFv2.3). Hopefully it will save someone time in the future!

It uses solely tensorflow ops, so you can use it in data loaders etc. The real pain here is that Tensorflow doesn't allow Eager Assignment, so you can't just update tensors by index. This works around that by creating two sparse tensors, one for the mask (where to apply the line) and another for the new_values (what value to apply at the line). The code for simply designing the line might not be applicable in your case (based on https://stackoverflow.com/a/47381058) but ported away from numpy.

import tensorflow as tf


def trapez(y, y0, w):
    return tf.clip_by_value(tf.minimum(y + 1 + w/2 - y0, -y + 1 + w/2 + y0), 0, 1)


def apply_output(img, yy, xx, val):
    stack = tf.stack([yy, xx], axis=1)
    stack = tf.cast(stack, tf.int64)
    values = tf.ones(stack.shape[0], tf.float32)
    mask = tf.sparse.SparseTensor(indices=stack, values=values, dense_shape=img.shape)
    mask = tf.sparse.reorder(mask)
    mask = tf.sparse.to_dense(mask)
    mask = tf.cast(mask, tf.float32)

    new_values = tf.sparse.SparseTensor(indices=stack, values=val, dense_shape=img.shape)
    new_values = tf.sparse.reorder(new_values)
    new_values = tf.sparse.to_dense(new_values)

    img = img * (1 - mask) + new_values * mask
    img = tf.cast(tf.expand_dims(img * 255, axis=-1), tf.uint8)
    return img


def weighted_line(img, r0, c0, r1, c1, w):
    output = img
    x = tf.range(c0, c1 + 1, dtype=tf.float32)
    slope = (r1-r0) / (c1-c0)
    w *= tf.sqrt(1 + tf.abs(slope)) / 2
    y = x * slope + (c1*r0-c0*r1) / (c1-c0)

    thickness = tf.math.ceil(w/2)
    yy = (tf.reshape(tf.math.floor(y), [-1, 1]) + tf.reshape(tf.range(-thickness-1, thickness+2), [1, -1]))
    xx = tf.repeat(x, yy.shape[1])
    values = tf.reshape(trapez(yy, tf.reshape(y, [-1, 1]), w), [-1])
    yy = tf.reshape(yy, [-1])

    limits_y = tf.math.logical_and(yy >= 0, yy < img.shape[0])
    limits_x = tf.math.logical_and(xx >= 0, xx < img.shape[1])
    limits = tf.math.logical_and(limits_y, limits_x)
    limits = tf.math.logical_and(limits, values > 0)
    yy = tf.cast(yy[limits], tf.float32)
    xx = tf.cast(xx[limits], tf.float32)

    return yy, xx, values[limits], apply_output(output, yy, xx, values[limits])

Just for a sanity check, you can call it with the following, and display it using opencv

if __name__ == "__main__":
    IMG = tf.zeros((500, 500), tf.float32)
    yy, xx, vals, FINAL_IMG = weighted_line(IMG, 10, 20, 100, 200, 5)
    jpeg_string = tf.io.encode_jpeg(FINAL_IMG)
    tf.io.write_file("output.jpg", jpeg_string)
    import cv2
    img = cv2.imread("output.jpg")
    cv2.imshow("Output", img)
    cv2.waitKey(0)

huangapple
  • 本文由 发表于 2020年1月3日 23:12:00
  • 转载请务必保留本文链接:https://go.coder-hub.com/59580964.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定