英文:
Which axis to use to normalise 3 dimensional data?
问题
我想知道在三维数据上进行归一化时，最好使用哪个轴（默认是最后一个轴）。例如，我有一个形状为 3 x 4 x 5（批量大小 x 元素数量 x 特征数量）的张量。在这种情况下，我应该使用轴 1 来对每个特征进行归一化，还是使用默认的轴 -1（2）？:
data = tf.constant(np.arange(3 * 4 * 5).reshape(3, 4, 5) * 10, dtype=tf.float32)print(data)tf.Tensor([[[ 0. 10. 20. 30. 40.][ 50. 60. 70. 80. 90.][100. 110. 120. 130. 140.][150. 160. 170. 180. 190.]][[200. 210. 220. 230. 240.][250. 260. 270. 280. 290.][300. 310. 320. 330. 340.][350. 360. 370. 380. 390.]][[400. 410. 420. 430. 440.][450. 460. 470. 480. 490.][500. 510. 520. 530. 540.][550. 560. 570. 580. 590.]]], shape=(3, 4, 5), dtype=float32)layer = tf.keras.layers.LayerNormalization(axis=1)output = layer(data)print(output)tf.Tensor([[[-1.3416405 -1.3416405 -1.3416406 -1.3416405 -1.3416405 ][-0.44721347 -0.44721353 -0.44721353 -0.44721353 -0.4472134 ][ 0.44721353 0.44721353 0.4472134 0.44721353 0.44721365][ 1.3416405 1.3416405 1.3416405 1.3416406 1.3416407 ]][[-1.3416407 -1.3416407 -1.3416407 -1.3416405 -1.3416405 ][-0.44721365 -0.44721365 -0.44721365 -0.44721317 -0.44721317][ 0.44721317 0.44721317 0.44721317 0.44721365 0.44721365][ 1.3416405 1.3416405 1.3416405 1.341641 1.3416405 ]][[-1.341641 -1.341641 -1.341641 -1.341641 -1.34164 ][-0.44721413 -0.44721413 -0.44721413 -0.44721413 -0.44721317][ 0.44721317 0.44721317 0.44721317 0.44721317 0.44721413][ 1.3416405 1.3416405 1.3416405 1.3416405 1.3416414 ]]], shape=(3, 4, 5), dtype=float32)
英文:
I want to know which axis is better to use for normalisation for a 3 dimensional data (it is the last axis by default). For example, I have this tensor with the shape 3 x 4 x 5 (batch_size x number_of_elements x number_of_features). Should I use axis 1 in this case so that I normalise each feature or the default axis -1 (2)?:
data = tf.constant(np.arange(3 * 4 * 5).reshape(3, 4, 5) * 10, dtype=tf.float32)print(data)tf.Tensor([[[ 0. 10. 20. 30. 40.][ 50. 60. 70. 80. 90.][100. 110. 120. 130. 140.][150. 160. 170. 180. 190.]][[200. 210. 220. 230. 240.][250. 260. 270. 280. 290.][300. 310. 320. 330. 340.][350. 360. 370. 380. 390.]][[400. 410. 420. 430. 440.][450. 460. 470. 480. 490.][500. 510. 520. 530. 540.][550. 560. 570. 580. 590.]]], shape=(3, 4, 5), dtype=float32)layer = tf.keras.layers.LayerNormalization(axis=1)output = layer(data)print(output)tf.Tensor([[[-1.3416405 -1.3416405 -1.3416406 -1.3416405 -1.3416405 ][-0.44721347 -0.44721353 -0.44721353 -0.44721353 -0.4472134 ][ 0.44721353 0.44721353 0.4472134 0.44721353 0.44721365][ 1.3416405 1.3416405 1.3416405 1.3416406 1.3416407 ]][[-1.3416407 -1.3416407 -1.3416407 -1.3416405 -1.3416405 ][-0.44721365 -0.44721365 -0.44721365 -0.44721317 -0.44721317][ 0.44721317 0.44721317 0.44721317 0.44721365 0.44721365][ 1.3416405 1.3416405 1.3416405 1.341641 1.3416405 ]][[-1.341641 -1.341641 -1.341641 -1.341641 -1.34164 ][-0.44721413 -0.44721413 -0.44721413 -0.44721413 -0.44721317][ 0.44721317 0.44721317 0.44721317 0.44721317 0.44721413][ 1.3416405 1.3416405 1.3416405 1.3416405 1.3416414 ]]], shape=(3, 4, 5), dtype=float32)
答案1
得分: 2
层归一化是在特征轴/轴上执行的。我不确定你的 axis=1 是什么意思，但由于你有多维数据，我假设这是一个（空间？）特征。所以在你的情况下，你应该使用将列表传递给函数的选项
tf.keras.layers.LayerNormalization(axis=[1, 2])
一些背景信息可以在这里找到，其中提到：
> [...] 假设我们有一个表示（B，H，W，C）的4D张量，表示图像的批次、高度、宽度和通道。
> 批次归一化 → 对通道（1,1,1,c）求均值和方差
> 层归一化 → 对批次（b,1,1,1）求均值和方差
> 实例归一化 → 对批次/通道（b,1,1,c）求均值和方差
英文:
Layer Normalization is performed across feature axis/axes. I am not sure what your axis=1 is, but, since you have multidimensional data, I assume this is a (spatial?) feature. So in your case you should use the option to pass a list to the function
tf.keras.layers.LayerNormalization(axis=[1, 2])
some background can be found here where it says:
> [...] Assuming we have 4D tensor that represents (B,H,W,C), which
> stands for Batch, Height, Width, and Channel of image.
> Batch Norm → Take mean and variance respect to channel (1,1,1,c)
> Layer Norm → Take mean and variance respect to batch (b,1,1,1)
> Instance Norm → Take mean and variance respect to batch/channel (b,1,1,c)
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论