Pytorch: 每轮接收相同的测试评估

huangapple go评论76阅读模式
英文:

Pytorch: Receiving same test evaluations each round

问题

我有一个与我的联邦学习设置有关的问题。我有一些权重,我想要使用PyTorch在每一轮中进行评估(测试)。目前,我对模型的损失和准确性很感兴趣。

这些权重是一个numpy数组的列表。

我使用的模型如下:

class Net(nn.Module):
    """模型(从 'PyTorch:60分钟快速入门' 中简单修改的CNN)"""

    def __init__(self) -> None:
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16 * 5 * 5, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = x.view(-1, 16 * 5 * 5)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        return self.fc3(x)

和生成结果的方法:

def load_testset():
    """加载CIFAR-10(测试集)"""
    trf = Compose([ToTensor(), Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
    testset = CIFAR10("./data", train=False, download=True, transform=trf)
    return DataLoader(testset), testset

def get_AccuracyAndLoss(weights):
    # 加载现有的权重列表
    weights_list = weights

    for i, weights in enumerate(weights_list):
        layer_name = 'layer_' + str(i)
        setattr(Net(), layer_name, nn.Parameter(torch.from_numpy(weights)))

    # 加载模型和数据(简单CNN,CIFAR-10)
    net = Net().to(DEVICE)
    testloader, test_set = load_testset()

    criterion = torch.nn.CrossEntropyLoss()
    correct, total, loss = 0, 0, 0.0
    net.eval()
    with torch.no_grad():
        for images, labels in testloader:
            images, labels = images.to(DEVICE), labels.to(DEVICE)
            outputs = net(images)

            loss += criterion(outputs, labels).item()
            _, predicted = torch.max(outputs.data, 1)
            total += labels.size(0)
            correct += (predicted == labels).sum().item()

    print('Test_loss: %.3f, accuracy: %.2f' % (loss/len(testloader), correct / total))

问题是,我每一轮得到相同的结果,大约是Test_loss: 2.306,accuracy: 0.10,我不知道是模型不正确还是权重插入方式有问题。我对深度学习相当新,欢迎提供不太复杂的答案。

大部分代码都基于PyTorch深度学习教程:https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html

我使用Python 3.10。

英文:

I have a problem with my federated learning setup. I have some weights which I want to evaluate (test) each round using PyTorch. For now im interested in the loss and accuracy of the model.
The weights are a list of numpy arrays.
The model im using is the following:

class Net(nn.Module):
    """Model (simple CNN adapted from 'PyTorch: A 60 Minute Blitz')"""

    def __init__(self) -> None:
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16 * 5 * 5, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = x.view(-1, 16 * 5 * 5)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        return self.fc3(x)

and the methods which produce the results:

def load_testset():
    """Load CIFAR-10 (test set)."""
    trf = Compose([ToTensor(), Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
    testset = CIFAR10("./data", train=False, download=True, transform=trf)
    return DataLoader(testset), testset

def get_AccuracyAndLoss(weights):
    # Load the existing weights list
    weights_list = weights

    for i, weights in enumerate(weights_list):
        layer_name = 'layer_' + str(i)
        setattr(Net(), layer_name, nn.Parameter(torch.from_numpy(weights)))

    # Load model and data (simple CNN, CIFAR-10)
    net = Net().to(DEVICE)
    testloader, test_set = load_testset()

    criterion = torch.nn.CrossEntropyLoss()
    correct, total, loss = 0, 0, 0.0
    net.eval()
    with torch.no_grad():
        for images, labels in testloader:
            images, labels = images.to(DEVICE), labels.to(DEVICE)
            outputs = net(images)

            loss += criterion(outputs, labels).item()
            _, predicted = torch.max(outputs.data, 1)
            total += labels.size(0)
            correct += (predicted == labels).sum().item()

    print('Test_loss: %.3f, accuracy: %.2f' % (loss/len(testloader), correct / total))

The issue is that i get the same results each round , which is around Test_loss: 2.306, accuracy: 0.10 and i dont know whether the model is incorrect or the way the weights get inserted. Im fairly new to deep learning so unsophisticated answers are appreciated.

Most of the code is based of the pytorch deep learning tutorial here : https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html

Im using python 3.10

答案1

得分: 0

根据您提供的代码,我认为您的模型未正确初始化。因为每次使用Net()时,您都在创建一个具有随机权重的全新模型,因此您的for循环只是为每一层创建一个全新的模型,然后正确初始化一个模型的一层,但之后该模型就没有被再次使用。在for循环之后,您又创建了一个具有全新随机权重的模型,用于验证。

要解决这个问题,您应该:

  • 首先使用net = Net().to(DEVICE)创建一个模型。
  • 然后,在for循环中,使用setattr(net, layer_name, nn.Parameters(...))正确初始化该模型的每一层。

按照这个顺序进行操作,结果应该更好,假设您在每次验证之间对模型进行训练。

如果您愿意,也可以使用load_state_dict(请参阅pytorch网站上的此教程)来避免必须浏览模型的每一层。

英文:

From the code you provided, I think your model is not correctly initialized.
Since each time you use Net() you are creating a new model with random weights, so your for loop is just creating a completely new model for each layer, initialize one layer of each model correctly, and then drop the model since it is not used again. After the for loop, you are creating another new model with all random weights and are using it for validation.

To fix it, you should :

  • First create a model with net = Net().to(DEVICE)
  • Then, do your for loop to initialize correctly each layer of this model with setattr(net, layer_name, nn.Parameters(...))

By doing it in this order, the result should be better, assuming you are training your model between each validation.

If you want, it could also be easier to use load_state_dict (see this tutorial on pytorch website) to avoid having to browse each layer of your model.

huangapple
  • 本文由 发表于 2023年4月4日 03:37:32
  • 转载请务必保留本文链接:https://go.coder-hub.com/75923182.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定