英文:
Batchwise convolutions – for each batch element a different weight
问题
给定形状为(32, 4, 16)的输入 x。需要对该输入应用一维卷积（conv1d）。请注意，有4个输入通道，每个通道的维度为16，批处理大小为32。假设要输出8个通道，因此weight的形状为(8, 4, 1)。
总的来说，以下是代码：
batch_size = 32
input_channels = 4
output_channels = 8
dim = 16
x = torch.zeros(size=(batch_size, input_channels, dim))
weight = torch.rand(size=(output_channels, input_channels, 1))
y = F.conv1d(x, weight)
现在来回答您的问题：
与其对每个批次元素应用相同的权重，我想对每个批次元素应用不同的权重。换句话说，weight必须具有形状(32, 8, 4, 1)。
我如何实现这种批处理卷积操作？
如果将weight分配为torch.rand(size=(batch_size, output_channels, input_channels, 1))，那么代码不起作用。
当然，存在一种基于for循环的简单解决方案。我正在寻找一种无需for循环的解决方案。
翻译部分结束，不包括问题回答。
英文:
Given an input x which has a shape of (32, 4, 16). To this input, a convolution (conv1d) shall be applied. Note that, there are 4 input channels. Each of these channels has a dimensionality of 16. The batch size is 32. In the following, it is assumed that the number of output channels shall be 8. Therefore, the weight has the shape (8, 4, 1)
Overall, we got the following code:
batch_size = 32
input_channels = 4
output_channels = 8
dim = 16
x = torch.zeros(size=(batch_size, input_channels, dim))
weight = torch.rand(size=(output_channels, input_channels, 1))
y = F.conv1d(x, weight)
Now to my question:
Instead of applying the same weight to each batch element, I want to apply to each batch element a different weight. In other words, the weight must have the shape (32, 8, 4, 1).
How I can implement this batchwise convolution operation?
If weight is assigned to torch.rand(size=(batch_size, output_channels, input_channels, 1)) then the code does not work.
Ofcourse there exists a simple solution which is based on a for-loop.
I am looking for a solution without a for-loop.
答案1
得分: 0
shape 是 (1) 的 Conv1d 与广播乘法和求和缩减相同。具体来说
# x.shape (32, 4, 16)
# weight.shape (8, 4, 1)
y = (x.unsqueeze(1) * weight.unsqueeze(0)).sum(dim=2)
# 等同于 y = F.conv1d(x, weight)
# 等同于 y = torch.einsum('bin,oin->bon', x, weight)
所以如果我们假设 weight 的形状是 (32, 8, 4, 1)，那么我们就不需要在第一维上进行广播。
# x.shape (32, 4, 16)
# weight.shape (32, 8, 4, 1)
y = (x.unsqueeze(1) * weight).sum(dim=2)
# 等同于 y = torch.einsum('bin,boin->bon', x, weight)
英文:
Conv1d with a kernel of shape (1) is the same as broadcasted multiply with sum-reduction. Specifically
# x.shape (32, 4, 16)
# weight.shape (8, 4, 1)
y = (x.unsqueeze(1) * weight.unsqueeze(0)).sum(dim=2)
# equivalent to y = F.conv1d(x, weight)
# equivalent to y = torch.einsum('bin,oin->bon', x, weight)
So if we assume weight has shape (32, 8, 4, 1) then we just don't need to broadcast over the first dimension.
# x.shape (32, 4, 16)
# weight.shape (32, 8, 4, 1)
y = (x.unsqueeze(1) * weight).sum(dim=2)
# equivalent to y = torch.einsum('bin,boin->bon', x, weight)
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论