def train():
#1.initilazing network
network = ResNet18()
network = network.to(device)
#state=torch.load('E:\LearningStuff\DLcode\Pytorch\CIFAR10\Trained_models01\Model21.pkl')
#param=state['network']
#opti=state['optimizer']
#optimizer=optim.Adam(network.parameters(),lr=learning_rate)
#optimizer.load_state_dict(opti)
print("Initializing Network")
#2.prepare data
trainloader = get_train_data_loader(batch_size=batch_size)
testloader = get_test_data_loader(batch_size=batch_size)
#3.prepare optimizer
optimizer = optim.Adam(network.parameters(), lr=learning_rate)
#optimizer=optim.SGD(network.parameters(),lr=0.1,momentum=0.9,weight_decay=5e-4)
#可选项：衰减学习率
scheduler = torch.optim.lr_scheduler.MultiStepLR(
optimizer, milestones=[150, 250, 300], gamma=0.1) #每10次训练衰减10倍
#4.initilazing tensorboard
comment = f'ResNet18 batch_size={batch_size} lr={learning_rate} device={device}'
tb = SummaryWriter(comment=comment)
best_acc = 0.0
for epoch in range(num_epochs):
train_loss = 0
train_correct = 0
for images, labels in trainloader:
images = images.to(device)
labels = labels.to(device)
optimizer.zero_grad()
preds = network(images)
loss = F.cross_entropy(preds, labels)
loss.backward()
optimizer.step()
train_loss += loss.item()
train_correct += preds.argmax(dim=1).eq(labels).sum().item()
present_trainset_acc = train_correct / TRAINSET_LENGTH
test_loss, present_testset_acc = test(network, testloader)
tb.add_scalar('Train Loss', train_loss, epoch)
tb.add_scalar('Accuracy on Trainset', present_trainset_acc, epoch)
tb.add_scalar('Test Loss', test_loss, epoch)
tb.add_scalar('Accuracy on Testset', present_testset_acc, epoch)
scheduler.step()
'''只保存测试集准确率不断上升的epoch'''
if present_testset_acc > best_acc:
state = {
'network': network.state_dict(),
'accuracy': present_testset_acc,
'optimizer': optimizer.state_dict(),
'epoch': epoch
}
torch.save(
state, 'E:\LearningStuff\DLcode\Pytorch\CIFAR10\Models\model' +
str(epoch) + '.pkl')
best_acc = present_testset_acc
print("epoch", epoch, "loss", train_loss, "Train acc",
present_trainset_acc, "Test acc ", present_testset_acc)
tb.close()
def main():
print("Initalizing Network")
for run in RunBuilder.get_runs(params):
comment = f'-{run}'
device = torch.device(run.device)
cnn = CNN()
cnn = cnn.to(device)
#cnn.load_state_dict(torch.load("E:\LearningStuff\DLcode\Pytorch\Mnist\CNN3MODEL9967.pkl"))
#extending_loader=get_expanding_data_loader([8],length=2000,batch_size=run.batch_size)
train_loader = get_train_data_loader(run.batch_size)
test_loader = get_test_data_loader(run.batch_size)
optimizer = optim.Adam(cnn.parameters(), lr=run.learning_rate)
''' Initializing tensorboard '''
tb = SummaryWriter(comment=comment, flush_secs=1)
images, labels = next(iter(train_loader))
grid = torchvision.utils.make_grid(images)
'''begin to train'''
for epoch in range(num_epochs):
total_loss = 0
total_correct = 0
adjust_learning_rate(optimizer, epoch, run.learning_rate)
for batch in train_loader:
images = batch[0].to(device)
labels = batch[1].to(device)
preds = cnn(images)
loss = F.cross_entropy(preds, labels)
optimizer.zero_grad()
loss.backward()
optimizer.step()
total_loss += loss.item()
total_correct += get_num_correct(preds, labels)
tb.add_scalar('Loss', total_loss, epoch)
tb.add_scalar('Number Correct', total_correct, epoch)
tb.add_scalar('Accuracy', total_correct / 60000, epoch)
path = SAVED_PATH + "\MODEL" + str(epoch) + ".pkl"
torch.save(cnn.state_dict(), path)
print("epoch", epoch, "loss", total_loss, "Accuracy",
total_correct / 60000)
cnn.to('cpu').eval()
for epoch in range(num_epochs):
cnn.load_state_dict(
torch.load(SAVED_PATH + "\MODEL" + str(epoch) + ".pkl"))
total_correct = 0
for batchs in test_loader:
images, labels = batchs
preds = cnn(images)
total_correct += get_num_correct(preds, labels)
print("The Accuracy of the ", epoch, "model on test-set is:",
total_correct / 60000)
tb.add_scalar('Accuracy on testset', total_correct / 10000, epoch)
tb.close()
def CombinationTest():
'''将九个预训练的模型级联进行预测，以达到99.7%的准确率'''
cnn1 = CNN()
cnn1.load_state_dict(torch.load(path + "\Model1.pkl"))
cnn1.eval()
cnn2 = CNN()
cnn2.load_state_dict(torch.load(path + "\Model2.pkl"))
cnn2.eval()
cnn3 = CNN()
cnn3.load_state_dict(torch.load(path + "\Model3.pkl"))
cnn3.eval()
cnn4 = CNN()
cnn4.load_state_dict(torch.load(path + "\Model4.pkl"))
cnn4.eval()
cnn5 = CNN()
cnn5.load_state_dict(torch.load(path + "\Model5.pkl"))
cnn5.eval()
cnn6 = CNN()
cnn6.load_state_dict(torch.load(path + "\Model6.pkl"))
cnn6.eval()
cnn7 = CNN()
cnn7.load_state_dict(torch.load(path + "\Model7.pkl"))
cnn7.eval()
cnn8 = CNN()
cnn8.load_state_dict(torch.load(path + "\Model8.pkl"))
cnn8.eval()
cnn9 = CNN()
cnn9.load_state_dict(torch.load(path + "\Model9.pkl"))
cnn9.eval()
total_correct = 0
test_loader = get_test_data_loader(256)
for batchs in test_loader:
images, labels = batchs
preds1 = cnn1(images).argmax(dim=1).unsqueeze(dim=1)
preds2 = cnn2(images).argmax(dim=1).unsqueeze(dim=1)
preds3 = cnn3(images).argmax(dim=1).unsqueeze(dim=1)
preds4 = cnn4(images).argmax(dim=1).unsqueeze(dim=1)
preds5 = cnn5(images).argmax(dim=1).unsqueeze(dim=1)
preds6 = cnn6(images).argmax(dim=1).unsqueeze(dim=1)
preds7 = cnn7(images).argmax(dim=1).unsqueeze(dim=1)
preds8 = cnn8(images).argmax(dim=1).unsqueeze(dim=1)
preds9 = cnn9(images).argmax(dim=1).unsqueeze(dim=1)
preds = torch.cat((preds1, preds2, preds3, preds4, preds5, preds6,
preds7, preds8, preds9),
dim=1)
preds = preds.numpy()
pred = []
for i in range(preds.shape[0]):
temp = preds[i]
pred.append(sorted(temp)[len(temp) // 2])
preds = torch.tensor(pred)
total_correct += new_get_num_correct(preds, labels)
print("The Accuracy of the combination model on test-set is:",
total_correct / 10000)
def read_dataset(self):
self.label_2.setText(" ")
batch = get_test_data_loader(1)
image, label = next(iter(batch))
self.image = image #将tensor传走
self.label_3.setText(str(label.numpy()))
image = image.squeeze(dim=0)
unloader = transforms.ToPILImage()
image = unloader(image)
pixmap = ImageQt.toqpixmap(image)
jpg = QtGui.QPixmap(pixmap).scaled(self.label.width(),
self.label.height())
self.label.setPixmap(jpg)
def SingleTest():
test_loader = get_test_data_loader(batch_size=256)
cnn = CNN()
''' 防止权重改变 '''
cnn.eval()
cnn.load_state_dict(torch.load('newMODEL9915.pkl'))
print("load cnn net.")
total_correct = 0
for batchs in test_loader:
images, labels = batchs
preds = cnn(images)
total_correct += get_num_correct(preds, labels)
print('The Accuracy of the model on test-set is:', total_correct / 10000)
def read_dataset(self):
self.label_2.setText(" ")
batch = get_test_data_loader(1)
image, label = next(iter(batch))
self.image = image #将tensor传走
self.label_3.setText(CLASSES[label])
image = image.squeeze(dim=0)
std = [0.2023, 0.1994, 0.2010]
mean = [0.4914, 0.4822, 0.4465]
image[0] = image[0] * std[0] + mean[0]
image[1] = image[1] * std[1] + mean[1]
image[2] = image[2] * std[2] + mean[2]
unloader = transforms.ToPILImage()
image = unloader(image)
pixmap = ImageQt.toqpixmap(image)
jpg = QtGui.QPixmap(pixmap).scaled(self.label.width(),
self.label.height())
self.label.setPixmap(jpg)
def test():
test_loader = get_test_data_loader(100)
network = ResNet18().to(device)
state = torch.load(MODEL_PATH)
param = state['network']
network.load_state_dict(param)
network.eval()
total_correct = 0
total_loss = 0
with torch.no_grad():
for images, labels in test_loader:
images = images.to(device)
labels = labels.to(device)
preds = network(images)
loss = F.cross_entropy(preds, labels)
a = preds.argmax(dim=1).equal(labels)
total_loss += loss.item()
total_correct += preds.argmax(dim=1).eq(labels).sum().item()
test_acc = total_correct / TESTSET_LENGTH
return test_acc
def ConfusionMatrix(network, loader):
all_preds = torch.tensor([])
all_labels = torch.tensor([])
network.eval()
for batch in loader:
images, labels = batch
preds = network(images)
all_preds = torch.cat((all_preds, preds), dim=0)
all_labels = torch.cat((all_labels, labels), dim=0)
stacked = torch.stack((all_labels, all_preds.argmax(dim=1)), dim=1)
cm = torch.zeros(CATEGORY_Num, CATEGORY_Num, dtype=torch.int64)
for p in stacked:
j, k = p.tolist()
cm[j, k] = cm[j, k] + 1
plot_confusion_matrix(cm,
CLASSES,
normalize=False,
title='Confusion matrix',
cmp=plt.cm.Blues)
if __name__ == '__main__':
loader = get_test_data_loader(256)
network = ResNet18()
state = torch.load(
'E:\LearningStuff\DLcode\Pytorch\CIFAR10\Models\BestModel.pkl')
param = state['network']
network.load_state_dict(param)
ConfusionMatrix(network, loader)
def plot_confusion_matrix():
cnn = CNN()
cnn.load_state_dict(torch.load(path + "\Model1.pkl"))
loader = get_test_data_loader(10)
ConfusionMatrix(cnn, loader)
def predict(image):
if not torch.is_tensor(image):
image = image.resize((28, 28))
'''将图像转为tensor'''
loader = transforms.Compose(
[transforms.ToTensor(),
transforms.Normalize(0.1307, 0.3081)])
image = loader(image).unsqueeze(dim=0)
''' 预测 '''
network = CNN()
network.eval()
network.load_state_dict(torch.load(PATH + "\Model1.pkl"))
pred = network(image).argmax(dim=1)
return pred
if __name__ == '__main__':
Load_from_file = True
if Load_from_file:
image = Image.open('Images/new2.jpg').convert('L')
pred = predict(image)
else:
batch = get_test_data_loader(batch_size=1)
image, label = next(iter(batch))
pred = predict(image)
print("the Prediction of ", str(label.numpy()), " is:",
str(pred.numpy()))
print('The predition is:{}'.format(pred.item()))