英文:
Placing examples in a mini-batch on the rows or columns of a matrix?
问题
我正在使用Tensorflow 2.0来训练一个模型。我正在考虑是否应该将多个示例放入一个批次中的行或列中。显然，这将影响我设计模型的方式。是否有关于哪种方式更好的实际建议？
英文:
I am using tensorflow 2.0 to train a model. I am deciding about whether I should put multiple examples in a batch along the rows or columns of a matrix. Obviously this will affect how I design the model as well. Is there any practical advice on which is better?
答案1
得分: 1
以下是您要翻译的内容：
引用 @ShanqingCai
将示例放置在矩阵的行中（即输入张量的第一个轴）是训练深度学习模型的主要方式。几乎可以找到的任何tensorflow2 / keras示例都遵循这种模式。将它们放在任何非第一个轴上的情况要少得多。
正如所说，行维度是存储样本的首选方式。我可以想到这样做的两个原因，
-
TF进行了涉及数据批次的许多矩阵乘法。因此，通过将批次维度保持为第一个维度，您可以不断使用矩阵乘法生成张量，其中批次维度也是第一个维度。（例如
[batch size, 10] . [10, 2]生成[batch size, 2]） -
另一个原因是行维度是变化最慢的维度。因此，通过将单个连续内存块中的访问个体样本，这在磁盘读取/内存读取时始终是首选的。
英文:
Quoting @ShanqingCai
> Putting examples in a batch along the rows of a matrix (i.e., the first axis of the input tensor) is the prevailing way of training deep learning models. Virtually any tensorflow2 / keras example you can find follows that pattern. Putting them along any non-first axis is much rarer.
As said, the row dimension is the preferred way of storing samples. I can think of two reasons why this is the case,
-
TF does a lot of matrix multiplications involving batches of data. Therefore, by keeping batch dimension as the first dimension, you are able to continuously produce tensors using matrix multiplication, which also has the batch dimension as the first dimension. (e.g.
[batch size, 10] . [10, 2]produces[batch size, 2]) -
The other reason is that the row dimension is the slowest changing dimension. Therefore by you can access an individual sample by taking a single contiguous chunk of memory which is always preferred when it comes to disk reading/ memory reading.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论