英文:
Neural Net: Loss decreasing, but accuracy stays exactly the same
问题
以下是您提供的代码的中文翻译部分：
我对PyTorch还很陌生，刚尝试在MINST数据集上构建了我的第一个神经网络。特别是，我想遵循这个教程：https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html
我的训练代码如下：
for epoch in range(5): # 一个epoch是对所有数据的一次迭代
running_loss = 0.0
for i, data in enumerate(train_loader, 0):
# 获取输入数据（这是一个批次，包括图像和标签的列表）
inputs, labels = data
# 重置梯度
optimizer.zero_grad()
# 前向传播 + 反向传播 + 优化
outputs = net(inputs) # 前向传播
loss = criterion(outputs, labels) # 损失
loss.backward() # 反向传播
optimizer.step() # 更新权重
# 打印统计信息
running_loss += loss.item()
if i % 2000 == 1999: # 每2000个小批次打印一次
print(f'[{epoch + 1}, {i + 1:5d}] loss: {running_loss / 2000:.3f}')
running_loss = 0.0
# 每个epoch后计算准确率
correct = 0
total = 0
with torch.no_grad():
for data in train_loader:
images, labels = data
output = net(images)
_, predicted = torch.max(outputs.data, 1)
total += labels.size(0)
correct += (predicted == labels).sum().item()
print(correct/total)
print('训练结束')
输出现在让我感到困惑，因为损失似乎显著减小，但准确度在数千个样本上保持完全不变。至少我期望准确度会有所变化（而不是增加），因为损失似乎表明权重正在变化。
完整的笔记本可以在这里找到：https://www.kaggle.com/code/adrianhayler/building-my-first-neural-net-with-pytorch
如有任何帮助，将不胜感激。
英文:
I am new to PyTorch and just tried to build my first Neural Network on the MINST dataset. In particular, I wanted to followed this tutorial: https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html
My training code looks like this:
for epoch in range(5): # a epoch is one iteration over all the data
running_loss = 0.0
for i, data in enumerate(train_loader, 0):
# get the inputs (this is a batch, so lists of images & labels)
inputs, labels = data
# reset the gradient
optimizer.zero_grad()
# forward + backward + optimize
outputs = net(inputs) # forward
loss = criterion(outputs, labels) # loss
loss.backward() # backward
optimizer.step() # update
# print statistics
running_loss += loss.item()
if i % 2000 == 1999: # print every 2000 mini-batches
print(f'[{epoch + 1}, {i + 1:5d}] loss: {running_loss / 2000:.3f}')
running_loss = 0.0
# compute accuracy after each epoch
correct = 0
total = 0
with torch.no_grad():
for data in train_loader:
images, labels = data
output = net(images)
_, predicted = torch.max(outputs.data, 1)
total += labels.size(0)
correct += (predicted == labels).sum().item()
print(correct/total)
print('Training finished')
The output now confuses me, as the loss seems to decrease significantly, but the accuracy remains completely unchanged on thousands of samples. I would at least expect a change in the accuracy (not an increase), as the weights seem to change judging by the loss.
[1, 2000] loss: 1.902
[1, 4000] loss: 1.413
[1, 6000] loss: 1.188
[1, 8000] loss: 0.844
[1, 10000] loss: 0.666
[1, 12000] loss: 0.631
[1, 14000] loss: 0.614
[1, 16000] loss: 0.534
[1, 18000] loss: 0.557
[1, 20000] loss: 0.518
[1, 22000] loss: 0.448
[1, 24000] loss: 0.450
[1, 26000] loss: 0.466
[1, 28000] loss: 0.493
[1, 30000] loss: 0.412
[1, 32000] loss: 0.404
0.09857142857142857
[2, 2000] loss: 0.442
[2, 4000] loss: 0.421
[2, 6000] loss: 0.429
[2, 8000] loss: 0.423
[2, 10000] loss: 0.411
[2, 12000] loss: 0.426
[2, 14000] loss: 0.425
[2, 16000] loss: 0.400
[2, 18000] loss: 0.430
[2, 20000] loss: 0.376
[2, 22000] loss: 0.385
[2, 24000] loss: 0.376
[2, 26000] loss: 0.388
[2, 28000] loss: 0.433
[2, 30000] loss: 0.346
[2, 32000] loss: 0.357
0.09857142857142857
[3, 2000] loss: 0.393
[3, 4000] loss: 0.356
[3, 6000] loss: 0.396
[3, 8000] loss: 0.381
[3, 10000] loss: 0.350
[3, 12000] loss: 0.368
[3, 14000] loss: 0.405
[3, 16000] loss: 0.355
[3, 18000] loss: 0.367
[3, 20000] loss: 0.355
[3, 22000] loss: 0.357
[3, 24000] loss: 0.366
[3, 26000] loss: 0.362
[3, 28000] loss: 0.393
[3, 30000] loss: 0.336
[3, 32000] loss: 0.333
0.09857142857142857
[4, 2000] loss: 0.372
[4, 4000] loss: 0.323
[4, 6000] loss: 0.362
[4, 8000] loss: 0.368
[4, 10000] loss: 0.346
[4, 12000] loss: 0.345
[4, 14000] loss: 0.381
[4, 16000] loss: 0.363
[4, 18000] loss: 0.357
[4, 20000] loss: 0.337
[4, 22000] loss: 0.363
[4, 24000] loss: 0.343
[4, 26000] loss: 0.353
[4, 28000] loss: 0.390
[4, 30000] loss: 0.298
[4, 32000] loss: 0.343
0.09857142857142857
[5, 2000] loss: 0.350
[5, 4000] loss: 0.324
[5, 6000] loss: 0.361
[5, 8000] loss: 0.350
[5, 10000] loss: 0.307
[5, 12000] loss: 0.347
[5, 14000] loss: 0.372
[5, 16000] loss: 0.347
[5, 18000] loss: 0.356
[5, 20000] loss: 0.302
[5, 22000] loss: 0.339
[5, 24000] loss: 0.345
[5, 26000] loss: 0.340
[5, 28000] loss: 0.405
[5, 30000] loss: 0.310
[5, 32000] loss: 0.352
0.09857142857142857
Training finished
The full notebook can be found here: https://www.kaggle.com/code/adrianhayler/building-my-first-neural-net-with-pytorch
Any help is greatly appreciated.
答案1
得分: 0
我找到了问题：
_, predicted = torch.max(outputs.data, 1)
必须更改为：
_, predicted = torch.max(output.data, 1)
outputs 是前向传递的输出，而不是我们迭代的样本。
英文:
I found the problem:
_, predicted = torch.max(outputs.data, 1)
has to be changed to:
_, predicted = torch.max(output.data, 1)
outputs is the output of the forward pass and not the samples we iterate over.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论