Numerical Instability: FP16 Error & Loss Collapse on SST2/OPT-1.3B Sweep

[lyh030925]

I am trying to reproduce the SST2 classification sweep for OPT-1.3B using the provided configuration (sgd_cls_ft.yml).
I am encountering a critical numerical stability issue:
With FP16 enabled: The run fails with the error: ValueError: Attempting to unscale FP16 gradients.
With FP16 disabled (FP32): The loss immediately drops to zero (loss: 0.0) in the first fraction of an epoch.
Both issues indicate severe gradient explosion.
My setup is PyTorch 2.1.0 on Python 3.10, could you teach me how to deal with this problem?