Having problems while following the video(section 108)

[Watcher2747]

Video: https://youtu.be/Z_ikDlimN6A?t=60117

I looked around the discussion section and it seems like this might be related to my issue:
#113

I receive this error:
RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.

When trying to run this code:

def train_step(model: torch.nn.Module,
               data_loader: torch.utils.data.DataLoader,
               loss_fn: torch.nn.Module,
               optimizer: torch.optim.Optimizer,
               accuracy_fn,
               device: torch.device = 'cpu'):
  '''
  Performs a training step with model trying to learn on data_loader
  '''
  train_loss, train_acc = 0, 0

  # Put model into training mode
  model.train()

  # Add a loop to loop through the training batches
  for batch, (X, y) in enumerate(data_loader):
    # Put data on target device
    X, y = X.to(device), y.to(device)

    # 1. Forward pass
    y_pred = model(X)

    # 2. Accumulate the loss and acc(per batch)
    train_loss += loss_fn(y_pred, y) 
    train_acc += accuracy_fn(y_true=y,
                             y_pred=y_pred.argmax(dim=1))

    # 3. Optimizer zero grad
    optimizer.zero_grad()

    # 4. Loss backward
    loss.backward()

    # 5. Optimizer step 
    optimizer.step()
  
  # Divide total train loss and acc by length of train dataloader
  train_loss /= len(data_loader)
  train_acc /= len(data_loader)

  print(f'Train loss: {train_loss:.5f} | Train acc: {train_acc:.2f}%\n')

For reference, this is the full error:

[Watcher2747]

Pffft.

Found the issue, a pretty silly one, I somehow forgot to add this line of code:
loss = loss_fn(y_pred, y)

So when I was trying to do loss.backward() the value was not getting reset and just used the loss variable that was defined somewhere earlier in the code(outside of this function)

So the fix would be this:

 # 2. Accumulate the loss and acc(per batch)
    loss = loss_fn(y_pred, y)  # <---------------
    train_loss += loss
    train_acc += accuracy_fn(y_true=y,
                             y_pred=y_pred.argmax(dim=1))

    # 3. Optimizer zero grad
    optimizer.zero_grad()

    # 4. Loss backward
    loss.backward()

[mrdbourke]

Nice work! Glad you figured it out, I've missed that same line plenty of times haha