英文:
Pytorch: Receiving same test evaluations each round
问题
我有一个与我的联邦学习设置有关的问题。我有一些权重,我想要使用PyTorch在每一轮中进行评估(测试)。目前,我对模型的损失和准确性很感兴趣。
这些权重是一个numpy数组的列表。
我使用的模型如下:
class Net(nn.Module):
"""模型(从 'PyTorch:60分钟快速入门' 中简单修改的CNN)"""
def __init__(self) -> None:
super(Net, self).__init__()
self.conv1 = nn.Conv2d(3, 6, 5)
self.pool = nn.MaxPool2d(2, 2)
self.conv2 = nn.Conv2d(6, 16, 5)
self.fc1 = nn.Linear(16 * 5 * 5, 120)
self.fc2 = nn.Linear(120, 84)
self.fc3 = nn.Linear(84, 10)
def forward(self, x: torch.Tensor) -> torch.Tensor:
x = self.pool(F.relu(self.conv1(x)))
x = self.pool(F.relu(self.conv2(x)))
x = x.view(-1, 16 * 5 * 5)
x = F.relu(self.fc1(x))
x = F.relu(self.fc2(x))
return self.fc3(x)
和生成结果的方法:
def load_testset():
"""加载CIFAR-10(测试集)"""
trf = Compose([ToTensor(), Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
testset = CIFAR10("./data", train=False, download=True, transform=trf)
return DataLoader(testset), testset
def get_AccuracyAndLoss(weights):
# 加载现有的权重列表
weights_list = weights
for i, weights in enumerate(weights_list):
layer_name = 'layer_' + str(i)
setattr(Net(), layer_name, nn.Parameter(torch.from_numpy(weights)))
# 加载模型和数据(简单CNN,CIFAR-10)
net = Net().to(DEVICE)
testloader, test_set = load_testset()
criterion = torch.nn.CrossEntropyLoss()
correct, total, loss = 0, 0, 0.0
net.eval()
with torch.no_grad():
for images, labels in testloader:
images, labels = images.to(DEVICE), labels.to(DEVICE)
outputs = net(images)
loss += criterion(outputs, labels).item()
_, predicted = torch.max(outputs.data, 1)
total += labels.size(0)
correct += (predicted == labels).sum().item()
print('Test_loss: %.3f, accuracy: %.2f' % (loss/len(testloader), correct / total))
问题是,我每一轮得到相同的结果,大约是Test_loss: 2.306,accuracy: 0.10,我不知道是模型不正确还是权重插入方式有问题。我对深度学习相当新,欢迎提供不太复杂的答案。
大部分代码都基于PyTorch深度学习教程:https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html
我使用Python 3.10。
英文:
I have a problem with my federated learning setup. I have some weights which I want to evaluate (test) each round using PyTorch. For now im interested in the loss and accuracy of the model.
The weights are a list of numpy arrays.
The model im using is the following:
class Net(nn.Module):
"""Model (simple CNN adapted from 'PyTorch: A 60 Minute Blitz')"""
def __init__(self) -> None:
super(Net, self).__init__()
self.conv1 = nn.Conv2d(3, 6, 5)
self.pool = nn.MaxPool2d(2, 2)
self.conv2 = nn.Conv2d(6, 16, 5)
self.fc1 = nn.Linear(16 * 5 * 5, 120)
self.fc2 = nn.Linear(120, 84)
self.fc3 = nn.Linear(84, 10)
def forward(self, x: torch.Tensor) -> torch.Tensor:
x = self.pool(F.relu(self.conv1(x)))
x = self.pool(F.relu(self.conv2(x)))
x = x.view(-1, 16 * 5 * 5)
x = F.relu(self.fc1(x))
x = F.relu(self.fc2(x))
return self.fc3(x)
and the methods which produce the results:
def load_testset():
"""Load CIFAR-10 (test set)."""
trf = Compose([ToTensor(), Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
testset = CIFAR10("./data", train=False, download=True, transform=trf)
return DataLoader(testset), testset
def get_AccuracyAndLoss(weights):
# Load the existing weights list
weights_list = weights
for i, weights in enumerate(weights_list):
layer_name = 'layer_' + str(i)
setattr(Net(), layer_name, nn.Parameter(torch.from_numpy(weights)))
# Load model and data (simple CNN, CIFAR-10)
net = Net().to(DEVICE)
testloader, test_set = load_testset()
criterion = torch.nn.CrossEntropyLoss()
correct, total, loss = 0, 0, 0.0
net.eval()
with torch.no_grad():
for images, labels in testloader:
images, labels = images.to(DEVICE), labels.to(DEVICE)
outputs = net(images)
loss += criterion(outputs, labels).item()
_, predicted = torch.max(outputs.data, 1)
total += labels.size(0)
correct += (predicted == labels).sum().item()
print('Test_loss: %.3f, accuracy: %.2f' % (loss/len(testloader), correct / total))
The issue is that i get the same results each round , which is around Test_loss: 2.306, accuracy: 0.10 and i dont know whether the model is incorrect or the way the weights get inserted. Im fairly new to deep learning so unsophisticated answers are appreciated.
Most of the code is based of the pytorch deep learning tutorial here : https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html
Im using python 3.10
答案1
得分: 0
根据您提供的代码,我认为您的模型未正确初始化。因为每次使用Net()
时,您都在创建一个具有随机权重的全新模型,因此您的for
循环只是为每一层创建一个全新的模型,然后正确初始化一个模型的一层,但之后该模型就没有被再次使用。在for
循环之后,您又创建了一个具有全新随机权重的模型,用于验证。
要解决这个问题,您应该:
- 首先使用
net = Net().to(DEVICE)
创建一个模型。 - 然后,在
for
循环中,使用setattr(net, layer_name, nn.Parameters(...))
正确初始化该模型的每一层。
按照这个顺序进行操作,结果应该更好,假设您在每次验证之间对模型进行训练。
如果您愿意,也可以使用load_state_dict
(请参阅pytorch网站上的此教程)来避免必须浏览模型的每一层。
英文:
From the code you provided, I think your model is not correctly initialized.
Since each time you use Net()
you are creating a new model with random weights, so your for
loop is just creating a completely new model for each layer, initialize one layer of each model correctly, and then drop the model since it is not used again. After the for
loop, you are creating another new model with all random weights and are using it for validation.
To fix it, you should :
- First create a model with
net = Net().to(DEVICE)
- Then, do your
for
loop to initialize correctly each layer of this model withsetattr(net, layer_name, nn.Parameters(...))
By doing it in this order, the result should be better, assuming you are training your model between each validation.
If you want, it could also be easier to use load_state_dict
(see this tutorial on pytorch website) to avoid having to browse each layer of your model.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论