Skip to content
Advertisement

why my model does not learn? Same on every epoch (Pytorch)

from sklearn import datasets
import pandas as pd
import numpy as np
import torch
from torch import nn

#loading the dataset
(data, target) = datasets.load_diabetes(as_frame=True,return_X_y=True) #with the as_frame=True data: pd.DataFrame


# converting data,target to tensors
data = torch.tensor(data.values,dtype=torch.float)
target = torch.tensor(target.values,dtype=torch.float)


#split the data 80% train 20% testing
a = 0.8
train_data , train_target = data[:int(a*len(data))] , data[:int(a*len(data))]
test_data , test_target = data[int(a*len(data)):] , data[int(a*len(data)):]


#constructing the model
# for this dataset dimentionality is 10 so the in_features will be 10
model = nn.Sequential(
    nn.Linear(in_features=10, out_features=128),
    nn.Linear(in_features=128, out_features=128),
    nn.Linear(in_features=128, out_features=1)
)

#loss fn , optimizer
loss_fn = nn.L1Loss() #binary cross entropy
optimizer = torch.optim.SGD(params = model.parameters(),lr=0.001) #stochastic gradient descent

#training loop
epochs = 1000

for epoch in range(epochs):
    #1. make prediction
    model.train()
    train_pred = model(train_data)
    
    loss = loss_fn(train_pred, train_target)

    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    model.eval()
    with torch.inference_mode():
        test_pred = model(test_data)
        loss_test = loss_fn(test_pred, test_target)

    if epoch%(epochs//min(10,epochs))==0: print(f"{epoch} - training loss: {round(float(loss),4)} | test loss: {round(float(loss_test),4)}")- training loss: {loss} | test loss: {loss_test}")

Output

0 – training loss: 0.0837 | test loss: 0.0806

100 – training loss: 0.0433 | test loss: 0.0431

200 – training loss: 0.0426 | test loss: 0.0425

300 – training loss: 0.042 | test loss: 0.0419

400 – training loss: 0.0414 | test loss: 0.0414

500 – training loss: 0.0408 | test loss: 0.0408

600 – training loss: 0.0403 | test loss: 0.0403

700 – training loss: 0.0398 | test loss: 0.0398

800 – training loss: 0.0393 | test loss: 0.0394

900 – training loss: 0.0388 | test loss: 0.0389

Advertisement

Answer

First, as it was mentioned in the comments, you probably meant:

train_data, train_target = data[:int(a*len(data))] , target[:int(a*len(data))]
test_data, test_target = data[int(a*len(data)):] , target[int(a*len(data)):]

Next, your target size is not consistent with the output size (this should give a warning). Using

loss = loss_fn(train_pred, train_target.unsqueeze(1))

and

loss_test = loss_fn(test_pred, test_target.unsqueeze(1))

should give you some traction.

User contributions licensed under: CC BY-SA
7 People found this is helpful
Advertisement