November 7, 2023
1 Coursework 1: Learning Learning, From Scratch
The total points for this exercise is 100.
Please refer to Moodle submission page for the due date.
1.1 Submission instructions:
Coursework submissions must comprise of a zipped folder. Inside this zip, you should have two things.
1. This Jupyter notebook, containing your code and written comments. 2. A “models/” folder, containing your trained model weights.
In the notebook, cells that need to be changed are clearly stated with a symbol and you are not allowed to change the rest of the code in any way. Descriptive answers must be contained in the markdown blocks starting with “Your reply:”. Code must only be written in the space inside the #begin_solution … #end_solution blocks. Code written outside these blocks will not be processed or evaluated.
Please, do not change #begin_test … #end_test blocks, these blocks are for TAs.
Each exercise must be implemented from scratch. Unless differently specified, only numpy and
matplotlib are allowed. The libraries are imported in the package cell and cannot be modified.
Questions:
Question 1: Linear Fitting (10 points)
Question 2: PCA (12 points)
Question 3: Linear Classification (18 points)
Question 4: Image Denoising (30 points)
Question 5: Neural Implicit Representation (30 points)
1.2 Datasets
We will be using several datasets for the coursework: IRIS , MNIST , CelebA , NoisyOCR.
IRIS and MNIST datasets will be downloaded directly in the notebook using the skdataset library. Keep in mind that you will need to be connected to Internet to be able to download the datasets. If you want to work offline, you are free to save your dataset to npy file locally and load them while offline although this is not officially supported.
CelebA and NoisyOCR are provided separately in a zip file; make sure to unzip it and the unzipped folder is placed in the same directory as the notebook.
IRIS: The IRIS dataset contains the following features in order: sepal length, sepal width, petal length, petal width. Classes names are: Iris Setosa for label 0, Iris Versicolour for label 1, and Iris Virginica for label 2.
MNIST: MNIST is a dataset composed of images of handwritten digits. CelebA: CelebA is a dataset composed of RGB images of human faces.
NoisyOCR: NoisyOCR is a dataset composed of images of noisy OCR scans, along with corre- sponding clean target images.
The script will generate two subsets for each of the two datasets, a training subset (X_dataset and Y_dataset with dataset the name of the dataset) and a test subset (X_dataset_test and Y_dataset_test).
We will test correctness of your code on Hidden set.
Warning: as Hidden may have different dimensions from IRIS and MNIST, hard-coded solutions may not work, thus resulting in lower grades. You need to make sure that your code would work if applied on a different number of samples and a different number of features/pixels.
1.3 Instructions for the other datasets zip file:
File cw_datasets.zip is provided.
You can download this zip locally, unzip it and keep the folder in the same directory as your Notebook. There is code provided below to download and unzip it. If you’re using Colab, the downloaded content will remain only while the session is active. When you start a new session, you’d have to download it again.
Google Drive: For a more permanent solution, you can upload the unzipped folder to your own Google Drive, and mount your drive in the notebook. Code is provided below, to mount the drive.
You can use either options, and a variable drive_mount is provided to toggle between the two. Set this variable accordingly.
The following cell imports all packages needed in the coursework. You are not allowed to use any other packages than the ones listed below.
# Importing packages
import numpy as np
import random
import matplotlib.pyplot as plt
from scipy import optimize
from sklearn import datasets as skdataset
from sklearn.model_selection import train_test_split from pathlib import Path
from PIL import Image
import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader import torch.nn.functional as F
from torch.utils.data import Dataset
import imageio
from PIL import Image import cv2
from tqdm import tqdm
from skimage.metrics import peak_signal_noise_ratio from IPython.display import Video
%matplotlib inline
The following code imports the drive module from Colab. This code will only run on Colab. If you’re not using Colab, comment this code out, or don’t run this cell.
from google.colab import drive
#Unmount any previously mounted drive if needed.
#Feel free to uncomment below line when you need to unmount Drive
#drive.flush_and_unmount()
The following cell sets the drive_mount variable. Set it to True if your Dataset folder is on drive. If you plan to use the Dataset folder locally, then set this to False
[ ]: drive_mount = False [ ]:
# Setting the seed
RAND_ST = 42 random.seed(RAND_ST) #begin_test
程序代写 CS代考 加微信: cstutorcs
1.4 Data Loading
The following cells are used to load IRIS and MNIST datasets.
def load_iris_dataset(num_classes=2): # Load the datadet from SkDataset iris = skdataset.load_iris()
X = iris.data
Y = iris.target
# Reduce the number of classes idx = Y < num_classes
X = X[idx]
Y = Y[idx]
return X, Y
def load_mnist_dataset(num_classes=2):
# Load the datadet from SkDataset
X, Y = skdataset.fetch_openml('mnist_784', version=1,␣
↪return_X_y=True,as_frame=False) Y = Y.astype(np.int64)
# Reduce the number of classes idx = Y < num_classes
X = X[idx] Y = Y[idx] return X, Y
def load_mnist_dataset_onevsall(class_id=7):
# Load the datadet from SkDataset
X, Y = skdataset.fetch_openml('mnist_784', version=1,␣
↪return_X_y=True,as_frame=False) Y = Y.astype(np.int64)
# One versus all
idx = Y == class_id
Y[~idx] = 0 Y[idx] = 1 return X, Y
1.5 Functions for visualization
def plot_correlation(X1, X2): # Plot both variables plt.figure(figsize=(8, 6)) plt.clf()
plt.scatter(X1, X2, edgecolor='k')
line = np.linspace(min(X1.min(), X2.min()), max(X1.max(), X2.max()), 20)
plt.plot(line, line, 'r')
plt.xlabel('Variable 1')
plt.ylabel('Variable 2')
plt.show()
def plot_eigenvecs(stacked_images, n_rows, n_columns, img_shape=None,␣ ↪channels=1):
n_images, n_dims = stacked_images.shape plt.figure()
for i in range(n_rows * n_columns):
plt.subplot(n_rows, n_columns, i + 1) if channels == 1:
# Normalize eigen vector to [0, 255] range
eigenvector = stacked_images[i] - np.min(stacked_images[i])
eigenvector /= np.max(eigenvector) + 1e-8
eigenvector = (eigenvector * 255).astype(np.uint8)
plt.imshow(np.reshape(eigenvector, [int(np.sqrt(n_dims)), int(np.
↪sqrt(n_dims))]), cmap='gray') elif channels == 3:
# Normalize eigen vector to [0, 255] range for each channel
eigenvector = stacked_images[i].reshape(img_shape[0], img_shape[1],␣
eigenvector -= np.min(eigenvector)
eigenvector /= np.max(eigenvector) + 1e-8
eigenvector = (eigenvector * 255).astype(np.uint8)
plt.imshow(eigenvector)
plt.axis('off')
plt.show()
def plot_images(stacked_images, n_rows, n_columns, titles, num_channels=1): n_images, n_dims = stacked_images.shape
plt.figure()
for i in range(n_rows * n_columns):
plt.subplot(n_rows, n_columns, i + 1)
if num_channels == 1:
# Grayscale image
plt.imshow(np.reshape(stacked_images[i], [int(np.sqrt(n_dims)),␣
↪int(np.sqrt(n_dims))]), cmap='gray', vmin=0, vmax=1) elif num_channels == 3:
# RGB image
image = np.reshape(stacked_images[i], [int(np.sqrt(n_dims // 3)),␣ ↪int(np.sqrt(n_dims // 3)), 3])
image = np.clip(image, 0, 1) # Ensure pixel values are within [0,␣
plt.imshow(image) else:
raise ValueError("Unsupported number of channels. Use␣ ↪num_channels=1 for grayscale or num_channels=3 for RGB.")
plt.axis('off')
if len(titles) == n_rows * n_columns:
plt.title(titles[i])
plt.show()
def plot_images_with_text(images, titles, texts=None): N_cols = len(images)
N_rows = len(images[0])
fig, axs = plt.subplots(N_rows, N_cols, figsize=(20, 15))
if titles is not None:
for i, ax in enumerate(axs[0]):
ax.set_title(titles[i])
for i, img_col in enumerate(images): for j, img in enumerate(img_col):
if len(img.shape) == 2:
axs[j,i].imshow(img, interpolation='nearest', cmap='gray')
axs[j,i].imshow(img, interpolation='nearest')
axs[j,i].axis('off') if texts is not None:
axs[j,i].text(0.5,-0.1, str(texts[j][i])[:4], size=10,␣ ↪ha="center", transform=axs[j,i].transAxes)
plt.show()
def plot_images_horizontal(imgs): num_imgs = len(imgs)
f = plt.figure(figsize=(40, 15)) for i in range(num_imgs):
f.add_subplot(1, num_imgs, i+1)
plt.imshow(imgs[i])
plt.show()
def true_positive(Y_test, y_pred):
mask = (Y_test == 1)
tp = (Y_test[mask] == y_pred[mask]).sum() return tp.item()
def true_negative(Y_test, y_pred):
mask = (Y_test == 0) | (Y_test == -1)
tn = (Y_test[mask] == y_pred[mask]).sum() return tn.item()
def false_negative(Y_test, y_pred):
mask = (y_pred == 0) | (y_pred == -1)
tn = (Y_test[mask] != y_pred[mask]).sum() return tn.item()
def false_positive(Y_test, y_pred):
mask = (y_pred == 1)
tn = (Y_test[mask] != y_pred[mask]).sum() return tn.item()
def plot_confusion_matrix(Y_test, y_pred): tp = true_positive(Y_test, y_pred)
tn = true_negative(Y_test, y_pred)
fp = false_positive(Y_test, y_pred) fn = false_negative(Y_test, y_pred) cf = np.array([[tn, fp], [fn, tp]]) fig, ax = plt.subplots() ax.matshow(cf, cmap=plt.cm.Blues)
for i in range(2): for j in range(2):
c = cf[i,j]
ax.text(j, i, str(c), va='center', ha='center')
plt.xlabel('Prediction')
plt.ylabel('Target')
plt.show()
1.6 Data Loading
In the following cells, the dataset is created with proper splits between training and test set.
IRIS dataset [ ]:
MNIST dataset [ ]:
Code for handling the Dataset zip file
The following cell mounts your drive if drive_mount is True
X, Y = load_iris_dataset(num_classes=3)
X_iris, X_iris_test, Y_iris, Y_iris_test = train_test_split(X, Y, test_size=0.
↪1, random_state=RAND_ST) # 90% training and 10% test
X, Y = load_mnist_dataset(num_classes=10)
X = X / 255.0
X_mnist, X_mnist_test, Y_mnist, Y_mnist_test = train_test_split(X, Y,␣
↪test_size=0.1, random_state=RAND_ST) # 90% training and 10% test
if drive_mount: drive.mount('/content/drive')
Set dataset_path to Drive or local directory. Also load the CelebA dataset for PCA [ ]:
if drive_mount:
dataset_path = '/content/drive/MyDrive/cw_datasets/' assert(os.path.exists(dataset_path))
# Set this path to your appropriate drive path
dataset_path = "./cw_datasets"
# In this case, the datasets path is assumed to be in the same directory as␣
↪this notebook
if not os.path.exists("./cw_datasets"):
!unzip -q -o cw_datasets.zip
def load_face_images_array(dataset_path="./cw_datasets/",test_fraction=0.05): images_path = os.path.join(dataset_path,"face_pca/")
flattened_images_list = []
images_paths = [os.path.join(images_path,p) for p in os.listdir(images_path)␣
↪if p.endswith(".jpg")]
for image_file in images_paths:
image_np = np.array(Image.open(image_file))/255.0
image_np_flattened = np.reshape(image_np,(1,-1))
flattened_images_list.append(image_np_flattened)
flattened_images = np.concatenate(flattened_images_list,0)
split_at = int(len(flattened_images)*test_fraction)
X_CelebA_test = flattened_images[:split_at]
X_CelebA = flattened_images[split_at:]
return X_CelebA, X_CelebA_test
X_CelebA,X_CelebA_test = load_face_images_array(dataset_path)
print(X_CelebA.shape,X_CelebA_test.shape)
1.7 1. Linear Fitting (10 points)
a) Implement the normal equation solver function nsolve, which takes as input the matrix X and the target vector y and returns the optimized weights w. Test your code with your own mockup/randomly created data. (5 points)
b) Implement line_fit(X,y) which should fit a linear function to the input data. Test your implementation on the following task: predict with linear fitting the petal length (cm) of the Iris dataset using the three remaining variables as inputs (sepal length (cm), sepal width (cm) and petal width (cm)). Report the L2 loss on the validation set and plot a graph showing the correlation between y and your prediction on the test set. (2 points)
c) Implement poly_fit(X,y) which should fit a 2nd degree polynomial to the input data. Test 8
your implementation on the following task: predict with the polynomial the petal width (cm) of the IRIS dataset using the three remaining variables as inputs (sepal length (cm), sepal width (cm), petal length (cm)). The 2nd degree polynomial should consider all possible pairwise terms, i.e. 𝑤1𝑥2 +𝑤2𝑥𝑦+𝑤3𝑦2 +𝑤4𝑥+𝑤5𝑦+𝑤6 in the case of two input variables 𝑥 and 𝑦. Report the L2 loss on the validation set and plot a graph showing the correlation between 𝑦 and your prediction on the test set. (3 points)
Question 1a
Implement the nsolve function
You need to modify the following cell with your own code.
def nsolve(X,y): """
Write your implementation of nsolve here.
Arguments:
X : Data matrix
y : Labels
w : Weights
#begin_solution
#end_solution
Testing your code on mockup/randomly created data.
You need to modify the following cell with your own code.
After implementing nsolve, test it below on some mock data using np.random
#begin_solution
#end_solution
Question 1b
Implement the line_fit function and test
You need to modify the following cell with your own code.
def line_fit(X,y): """
Write your implementation of line_fit here.
Arguments:
X : Data matrix
y : Labels
w : Weights
l2_error : L2 Prediction error using learned w
#begin_solution
#end_solution
return w, l2_error
Testing your code on IRIS
You need to modify the following cell with your own code.
After implementing line_fit, test it below on IRIS Train set.
Print the L2 error on the Training set.
#begin_solution
#end_solution
You need to modify the following cell with your own code.
After implementing line_fit, test it below on IRIS Test set.
Print the L2 error with respect to Test set.
#begin_solution
#end_solution
Showing the correlation between X and y.
You need to modify the following cell with your own code.
Below, plot and display the correlation between true y and predicted values.
#begin_solution
#end_solution
Do not write anything in the cell below, it is for TAs
#begin_test
Question 1c
Implement the poly_fit function and test
You need to modify the following cell with your own code.
def poly_fit(X,y): """
Write your implementation of poly_fit here.
Arguments:
X : Data matrix
y : Labels
w : Weights
l2_error : L2 prediction error using learned w
#begin_solution
#end_solution
return w, l2_error
Testing your code on IRIS.
You need to modify the following cell with your own code.
After implementing poly_fit, test it below on IRIS Training set.
Print L2 error with respect to training set.
#begin_solution
#end_solution
You need to modify the following cell with your own code.
After implementing poly_fit, test it below on IRIS Test set.
Print L2 error with respect to test set.
#begin_solution
#end_solution
Showing the correlation between X and y.
You need to modify the following cell with your own code.
Below, plot and display the correlation between true y and predicted values
#begin_solution
#end_solution
Do not write anything the cell below, it is for TAs
#begin_test
1.8 2. PCA (12 points)
a) Implement a function pca(X, ndims) that performs PCA over the input data X and returns both the mean vector ̄X and the ndims top components. The top components are the eigen vectors linked to the top eigen values computed from the covariance matrix. Try your function on the MNIST dataset (which is composed of 10 digit classes) and on the CELEBA dataset (which contains face RGB images). Display the top 10 components fitted on the train dataset as images.(2 points + 2 points + 2 points)
b) Next, check that you can reconstruct perfectly an input image (digit or face) from the test set using all components, by implementing pca_projection(X, mean_vec, eig_vecs) and pca_reconstruction(weights, mean_vec, eig_vecs). (2 points + 2 points + 2 points on hid- den dataset)
Question 2a
Implement PCA and plot Eigenvectors
You need to modify the following cell with your own code.
def pca(X, ndims, num_image_channels=1): """
Compute PCA on a batch of flattened grayscale or RGB images with dynamic␣ ↪dimensions.
Arguments:
X : Data matrix where each row is a flattened image
ndims : Number of reduced dimensions
mean_vec : Data mean
top_eig_vecs : Selected eigen vectors, a matrix where each column␣
↪corresponds """
#begin_solution
#end_solution
to an eigen vector
return mean_vec, top_eig_vecs
Testing your code on MNIST
You need to modify the following cell with your own code.
Below, test your PCA on MNIST dataset.
Plot and display the selected eigen vectors returned by PCA.
Use the plot_eigenvecs function provided above.
Read that function and pass arguments as needed.
#begin_solution
#end_solution
Testing your code on CelebA [ ]:
Below, test your PCA on CelebA dataset.
Plot and display the selected eigen vectors returned by PCA.
浙大学霸代写 加微信 cstutorcs
Use the plot_eigenvecs function provided above.
Read that function and pass arguments as needed.
#begin_solution
#end_solution
Do not write anything in the cell below, it is for the TAs
#begin_test
Question 2b
Now, we can evaluate if the code is working properly by projecting the first image of the test set on the eigen vectors.
You need to modify the following cell with your own code.
def pca_projection(X, mean_vec, eig_vecs): """
Write your implenetation of PCA projection here.
Arguments:
X : Data matrix
mean_vec : Data mean
eig_vecs : A numpy array where each column corresponds to an eigen vector
weights : Weights corresponding to the eigen vectors
#begin_solution
#end_solution
return weights
You need to modify the following cell with your own code.
def pca_reconstruction(weights, mean_vec, eig_vecs): """
Write your implementation of PCA reconstruction here.
Arguments:
weights : Weights obtained from pca_projection
mean_vec : Data mean
eig_vecs : A numpy array where each column corresponds to an eigen vector
#begin_solution
#end_solution
return reconstruction
You need to modify the following cell with your own code.
1. Perform PCA on MNIST training set, without reducing dimensions. 2. Project the first image of Test set to the eigen vectors using␣
↪pca_projection.
3. Reconstruct that image using pca_reconstruction.
4. Display side-by-side, the Test image, its projection and the
pixelwise difference between the two. Use the plotting functions provided.
#begin_solution
#end_solution
You need to modify the following cell with your own code.
1. Perform PCA on CelebA training set, without reducing dimensions. 2. Project the first image of Test set to the eigen vectors using␣
↪pca_projection.
3. Reconstruct that image using pca_reconstruction.
4. Display side-by-side, the Test image, its projection and the
pixelwise difference between the two. Use the plotting functions provided.
#begin_solution
#end_solution
If your PCA reconstruction for faces is imperfect, discuss why that is the case. Edit the cell below with “Your reply” Your reply here:
Don’t write anything in the block below, its for TAs
#begin_test
1.9 3. Linear Classification (18 points)
a) Implement the normal equation-based binary linear classifier lclass(examplesA, examplesB, tes- tExamples) where the first two arguments are the set of samples from class A and class B respec- tively and the third is the test. The function should return a vector of 0s and 1s, 0 if test is in A and 1 otherwise. It should, for simplicity, both train and test in one function call. (3 points) Test this on all the samples in IRIS, Setosa vs non-Setosa, and plot the confusion matrix. (4 points on hidden dataset)
b) Extend the implementation in 3a to do multi-class classification. Implement lclass_prob(examplesA, examplesB, testExamples) and lmclass(examples, class, testExamples) that together perform multi-class classification of the examples examples according to the vec- tor of labels class of the same size and test it with testExamples by returning a matrix, where each row expresses the probability of a sample in testExamples to belong to each class. Give the accuracy of your model. (5 points + 6 points on hidden dataset)
Present findings applying multi-class classification on IRIS dataset with 3 classes. Question 3a
Implement Linear, binary classifier and test
You need to modify the following cell with your own code.
def lclass(examplesA, examplesB, testExample): """
Write your implementation of normal equation based linear classifier here.
Arguments:
examplesA : Matrix of samples from class A
examplesB : Matrix of samples from class B
testExample : Matrix of test samples to predict on
preds : Predicted 0/1 labels on testExample
#begin_solution
#end_solution
return preds
You need to modify the following cell with your own code.
After implementing lclass, test it below on the IRIS dataset.
#begin_solution
#end_solution
You need to modify the following cell with your own code.
Below, plot and display the confusion matrix of your prediction with respect to
true values. Use provided functions for plotting.
#begin_solution
#end_solution
Do not write anything in the cell below, it is for the TAs
#begin_test
Question 3b
Implement Linear, multi-class classifier and test
You need to modify the following cell with your own code.
def softmax(preds): """
Write your implementation of the softmax activation here.
Arguments:
preds : The output predicted classes from your classifier
activation : The outputs after applying softmax activation
#begin_solution
#end_solution
程序代写 CS代考 加QQ: 749389476
You need to modify the following cell with your own code.
def lclass_prob(examplesA, examplesB, testExample): """
Write your implementation of lclass_prob here.
Arguments:
examplesA : Matrix of samples from class A
examplesB : Matrix of samples from class B
testExample : Matrix of test samples to predict on
preds : Predicted probabilities of samples in testExample belonging to a␣ ↪class.
#begin_solution
#end_solution
return preds
return activation
You need to modify the following cell with your own code.
def lmclass(examples, labels, testExample): """
Write your implementation of multiclass classifier lmclass here.
Use your lclass_prob implementation here to solve this.
Arguments:
examples : The training data matrix
labels : The training labels
testExample : The testing data matrix
preds_prob : For each sample in testExample, their predicted probabilities
of belonging to each class
#begin_solution
#end_solution
return preds_prob
Print the accuracy of the prediction below
You need to modify the following cell with your own code.
#begin_solution
#end_solution
Do not write anything in the cell below, it is for the TAs
#begin_test
1.10 4. Denoising (30 points)
a) Implement denoiseGauss(image) to denoise the image noisy_image.png under cw_datasets/Filtering/ using a 5×5 Gaussian filter. (5 points)
b) Implement a convolutional neural network in Pytorch to denoise an image. We provide you pairs of noisy and noise-free image patches of size 128×128 for training and testing, under cw_datasets/Denoising/. You may use all the infrastructure of Pytorch. The network should have sufficient depth and complexity to be able to converge well. Please use ReLU non-linearities after each layer (20 points).
c) Given the comparison between the Gaussian-kernel denoising and the neural network denoising methods, discuss which method performs better and why. You should write no more than 5 sentences. (5 points)
Question 4a
Implement the below functions to denoise the image
You need to modify the following cell with your own code.
def gkern(l=5, sig=1.): #begin_solution
#end_solution
return out_kernel def denoise_gauss(image):
#begin_solution
#end_solution
return denoised_image
Load input image from file
if drive_mount:
dataset_path = "/content/drive/MyDrive/cw_datasets/"
dataset_path = "./cw_datasets/"
noisy_image_filename = os.path.join(dataset_path,"Filtering/noisy_image.png")
Filter the input image
noisy_image = np.asarray(Image.open(noisy_image_filename).convert('RGB'))
denoised_image = denoise_gauss(noisy_image)
plot_images_horizontal([noisy_image, denoised_image])
Question 4b
Implement a neural denoiser using Pytorch
Implement dataset class below. Note that the images output by the DenoisingDB dataset should be of size 128 x 128. Crop the original images at the top left corner to get the aforementioned size.
You need to modify the following cell with your own code.
class DenoisingDB(Dataset):
def __init__(self, input_imgs_path, cleaned_imgs_path):
super().__init__() #begin_solution #end_solution
def __len__(self): #begin_solution
#end_solution
return length
def __getitem__(self, idx):
#begin_solution
#end_solution
return (input_img, cleaned_img)
Implement the Denoising network:
Note that the network need to be sufficiently complex for performing the denoising task successfully. Feel free to experiment with different layers, blocks and operations.
You need to modify the following cell with your own code.
class DenoisingCNN(nn.Module): def __init__(self):
super(DenoisingCNN, self).__init__() #begin_solution
#end_solution
def forward(self, x): #begin_solution
#end_solution
# Create an instance of the network
model = DenoisingCNN()
Implement the reconstruction loss
You need to modify the following cell with your own code.
def loss_function(prediction, target): """
Calculate the Mean Squared Error (MSE) loss between the prediction and the␣ ↪target.
Arguments:
prediction : torch.Tensor
The predicted image.
target : torch.Tensor
The target image for reconstruction.
loss : torch.Tensor
The computed loss.
#begin_solution
#end_solution
return loss
Paths to input data
if drive_mount:
dataset_path = "/content/drive/MyDrive/cw_datasets/"
dataset_path = "./cw_datasets/"
input_imgs_path = os.path.join(dataset_path,"Denoising/input_noisy_images/")
cleaned_imgs_path = os.path.join(dataset_path,"Denoising/target_clean_images/")
Write the training loop
You need to modify the following cell with your own code.
#begin_solution
#end_solution
Save the model (needed for marking)
if not os.path.exists("models/"): os.makedirs("./models/")
torch.save({"denoiser":denoiser}, "models/denoiser.pth")
Load trained model
state_dict = torch.load("models/denoiser.pth")
denoiser = state_dict["denoiser"].to(device)
Plot 5 input images and their relative denoised images side by side. Populate the 3 lists of input, gt and output with 5 images each.
You need to modify the following cell with your own code.
input_imgs, gt_imgs, output_imgs = [], [], [] #begin_solution
#end_solution
titles = ['Input', 'GroundTruth', 'Output'] plot_images_with_text([input_imgs, gt_imgs, output_imgs],titles,texts=None)
Compute images for both the AE and Gaussian Kernels algorithm from images. You need to modify the following cell with your own code.
dataset = DenoisingDB(input_imgs_path,cleaned_imgs_path)
images = [dataset[i][0].to(device) for i in range(5)] targets = [dataset[i][1].to(device) for i in range(5)] images_ae = []
images_gauss = []
#begin_solution
#end_solution
Now we plot images and PSNR, to compare the two methods
imgs_plot = [ el.detach().squeeze().cpu().numpy() for el in images] tgts_plot = [ el.detach().squeeze().cpu().numpy() for el in targets]
psnrs = [ 0.0 for el in targets ] for i in range(len(tgts_plot)):
psnr_ae = peak_signal_noise_ratio(tgts_plot[i], images_ae[i])
pnsr_gauss = peak_signal_noise_ratio(tgts_plot[i], images_gauss[i])
psnrs[i] = ['', psnr_ae, pnsr_gauss, '']
img = [imgs_plot, images_ae, images_gauss, tgts_plot]
titles = ['Input', 'AE', 'Gaussian', 'Targets']
texts = psnrs
plot_images_with_text(img, titles, texts)
plt.show()
Question 4c
Discuss which method performs better, and why, edit the cell below with “Your reply”
Your reply here:
1.11 5. Implicit Neural Representation (30 points)
The objective here is to learn an implicit neural function to predict frames of an animated 2D object. That is, learn a function 𝑓 (𝑥, 𝑦, 𝑡; 𝜃), where 𝜃 are learnable MLP weights, that takes as input pixel coordinates 𝑥, 𝑦 and time 𝑡 and predicts the pixel value at that location and time. The
learned function should then be able to generate frames of the video at any given instance 𝑡. The task is divided into multiple incremental blocks as below:
a) Fit an MLP model to a single image. (5 points)
b) Add positional encoding of pixels to improve prediction. (5 points)
c) Adapt the framework to