機械学習のお勉強(pytorchのtutorialを眺めてみる)
install
DOCS
PyTorch documentation — PyTorch master documentation
Tutorial
すごくわかりやすい
What is PyTorch? — PyTorch Tutorials 0.2.0_4 documentation
Pytorchで遊ぼう【データ成形からFNNまで】 - HELLO CYBERNETICS
GitHub - yunjey/pytorch-tutorial: PyTorch Tutorial for Deep Learning Researchers
気になった箇所だけ下にまとめる👇
Data Loading and Processing Tutorial
Data Loading and Processing Tutorial — PyTorch Tutorials 0.2.0_4 documentation
Transform
Let’s create three transforms:
Rescale: to scale the image
RandomCrop: to crop from image randomly. This is data augmentation.
ToTensor: to convert the numpy images to torch images (we need to swap axes).
Iterating through the dataset
Batching the data
Shuffling the data
Load the data in parallel using multiprocessing workers
num_workersでいくつのコアでデータをロードするか指定(デフォルトはメインのみ)
dataloader = DataLoader(transformed_dataset, batch_size=4, shuffle=True, num_workers=4) # Helper function to show a batch def show_landmarks_batch(sample_batched): """Show image with landmarks for a batch of samples.""" images_batch, landmarks_batch = \ sample_batched['image'], sample_batched['landmarks'] batch_size = len(images_batch) im_size = images_batch.size(2) grid = utils.make_grid(images_batch) plt.imshow(grid.numpy().transpose((1, 2, 0))) for i in range(batch_size): plt.scatter(landmarks_batch[i, :, 0].numpy() + i * im_size, landmarks_batch[i, :, 1].numpy(), s=10, marker='.', c='r') plt.title('Batch from dataloader') for i_batch, sample_batched in enumerate(dataloader): print(i_batch, sample_batched['image'].size(), sample_batched['landmarks'].size()) # observe 4th batch and stop. if i_batch == 3: plt.figure() show_landmarks_batch(sample_batched) plt.axis('off') plt.ioff() plt.show() break
output
0 torch.Size([4, 3, 224, 224]) torch.Size([4, 68, 2]) 1 torch.Size([4, 3, 224, 224]) torch.Size([4, 68, 2]) 2 torch.Size([4, 3, 224, 224]) torch.Size([4, 68, 2]) 3 torch.Size([4, 3, 224, 224]) torch.Size([4, 68, 2])
torchvision
これは便利すぎる
One of the more generic datasets available in torchvision is ImageFolder.
It assumes that images are organized in the following way:
root/ants/xxx.png
root/ants/xxy.jpeg
root/ants/xxz.png
.
.
.
root/bees/123.jpg
root/bees/nsdf3.png
root/bees/asd932_.png
where ‘ants’, ‘bees’ etc. are class labels. Similarly generic transforms which operate on PIL.Image like RandomHorizontalFlip, Scale, are also avaiable. You can use these to write a dataloader like this:
import torch from torchvision import transforms, datasets data_transform = transforms.Compose([ transforms.RandomSizedCrop(224), transforms.RandomHorizontalFlip(), transforms.ToTensor(), transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]) ]) hymenoptera_dataset = datasets.ImageFolder(root='hymenoptera_data/train', transform=data_transform) dataset_loader = torch.utils.data.DataLoader(hymenoptera_dataset, batch_size=4, shuffle=True, num_workers=4)
これを使えば自作データセットのCNNのデータの用意がすぐできるイエイ
Transfer Learning Tutorial
Transfer Learning tutorial — PyTorch Tutorials 0.2.0_4 documentation
These two major transfer learning scenarios looks as follows:
Finetuning the convnet: Instead of random initializaion, we initialize the network with a pretrained network, like the one that is trained on imagenet 1000 dataset. Rest of the training looks as usual.
ConvNet as fixed feature extractor: Here, we will freeze the weights for all of the network except that of the final fully connected layer. This last fully connected layer is replaced with a new one with random weights and only this layer is trained.
Finetuning the convnet
pretrainされたモデルを読み込んだ上で学習
model_ft = models.resnet18(pretrained=True) num_ftrs = model_ft.fc.in_features model_ft.fc = nn.Linear(num_ftrs, 2) if use_gpu: model_ft = model_ft.cuda() criterion = nn.CrossEntropyLoss() # Observe that all parameters are being optimized optimizer_ft = optim.SGD(model_ft.parameters(), lr=0.001, momentum=0.9) # Decay LR by a factor of 0.1 every 7 epochs exp_lr_scheduler = lr_scheduler.StepLR(optimizer_ft, step_size=7, gamma=0.1)
ConvNet as fixed feature extractor
for param in model_conv.parameters(): param.requires_grad = False
ここでconv層の重みは固定をして全結合層だけ学習
model_conv = torchvision.models.resnet18(pretrained=True) for param in model_conv.parameters(): param.requires_grad = False # Parameters of newly constructed modules have requires_grad=True by default num_ftrs = model_conv.fc.in_features model_conv.fc = nn.Linear(num_ftrs, 2) if use_gpu: model_conv = model_conv.cuda() criterion = nn.CrossEntropyLoss() # Observe that only parameters of final layer are being optimized as # opoosed to before. optimizer_conv = optim.SGD(model_conv.fc.parameters(), lr=0.001, momentum=0.9) # Decay LR by a factor of 0.1 every 7 epochs exp_lr_scheduler = lr_scheduler.StepLR(optimizer_conv, step_size=7, gamma=0.1)
GPU化
Training a classifier — PyTorch Tutorials 0.2.0_4 documentation
Multi-GPU examples — PyTorch Tutorials 0.2.0_4 documentation
net.cuda() inputs, labels = Variable(inputs.cuda()), Variable(labels.cuda())
上記のように基本的にinputとラベルをすべてcuda()とつけてgpu化する。
.cpu()とするとcpu化。
pytorchのdebianファイル
GitHub - CDLuminate/pytorch: PyTorch Debian packaging
参考
pytorch.learning/mnist1.py at master · moskomule/pytorch.learning · GitHub
Inferring shape via flatten operator - PyTorch Forums
ChainerとPyTorchのコードを比較する - 線形回帰編 - Qiita
PyTorch : Tutorial 初級 : 分類器を訓練する – CIFAR-10 – PyTorch