英文:
During training, how to get value of a tensor defined during building time?
问题
从这个教程中,可以看到作为参数传递给tape.gradient()的loss_value变量可以有如下值:
tf.Tensor(0.36069894, shape=(), dtype=float32)
我正在构建类似的东西(代码非常长且复杂,不可能在这里放出),但我访问的损失元素是在构建时计算的(即在训练循环之前),如下所示:
x = Input(shape=(2,*input_shape))
z = SomeLayer(x)
y = SomeOtherLayer(z)
total_loss = TensorOperations(z)
net = Model(x,y)
其中TensorOperations
可以是不同的操作,如K.mean
、K.abs
等。
但当我在训练期间打印出这个total_loss变量时:
with tf.GradientTape() as tape:
net(input_data)
print(total_loss)
我得到类似以下的内容:
Tensor("truediv_8:0", shape=(?,), dtype=float32)
我的问题是:如何获得一个具有实际值的张量(类似于第一个张量)?
[编辑] 我所指的“真正的”代码要复杂得多,这里是total_loss变量和这里是net = Model()的链接。
英文:
From this tutorial, one sees that the loss_value variable given as param to tape.gradient() can have a value like so:
tf.Tensor(0.36069894, shape=(), dtype=float32)
I am constructing something similar (code is very long and complex, not possible to put it here), but the loss element I access to is computed during building time (so, before the training loop), in this fashion:
x = Input(shape=(2,*input_shape))
z = SomeLayer(x)
y = SomeOtherLayer(z)
total_loss = TensorOperations(z)
net = Model(x,y)
Where TensorOperations
can be different operations such as K.mean
, K.abs
etc.
But when I print out this total_loss variable during training:
with tf.GradientTape() as tape:
net(input_data)
print(total_loss)
I get something like:
Tensor("truediv_8:0", shape=(?,), dtype=float32)
My question is: how do I get a tensor with an actual value (like the first tensor) ?
[EDIT] the "real" code I am referring to is much more complex, here is the total_loss variable and here is the net = Model()
答案1
得分: 1
最简单的方法可能是将您的损失层包括在模型的输出中:
net = Model(x, [y, total_loss])
然后,在训练循环中:
with tf.GradientTape() as tape:
y_pred, loss_value = net(input_data)
print(loss_value)
然而,我建议尝试将损失与模型声明解耦,这样更灵活。
英文:
The easiest is probably to include your loss layer as an output of your model:
net = Model(x,[y, total_loss])
And then, during the training loop:
with tf.GradientTape() as tape:
y_pred, loss_value = net(input_data)
print(loss_value)
However, I would advise to try to decouple the loss from the model declaration, it's just more flexible.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论