Skip to content
Merged
24 changes: 24 additions & 0 deletions docs/source-pytorch/tuning/profiler.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,30 @@
Find bottlenecks in your code
#############################

.. warning::

**Do not wrap** ``Trainer.fit()``, ``Trainer.validate()``, or other Trainer methods
inside a manual ``torch.profiler.profile`` context manager.
This will cause unexpected crashes and cryptic errors due to incompatibility between
PyTorch Profiler's context management and Lightning's internal training loop.
Instead, always use the ``profiler`` argument in the ``Trainer`` constructor.

Example (correct usage):

.. code-block:: python
import pytorch_lightning as pl
trainer = pl.Trainer(
profiler="pytorch", # <- This enables built-in profiling safely!
...
)
trainer.fit(model, train_dataloaders=...)
**References:**
- https://github.com/pytorch/pytorch/issues/88472
- https://github.com/Lightning-AI/lightning/issues/16958

.. raw:: html

<div class="display-card-container">
Expand Down
8 changes: 8 additions & 0 deletions src/lightning/pytorch/trainer/trainer.py
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,14 @@ def __init__(
profiler: To profile individual steps during training and assist in identifying bottlenecks.
Default: ``None``.

.. note::
Do **not** use a manual ``torch.profiler.profile`` context manager around
``Trainer.fit()``, ``Trainer.validate()``, etc.
This will lead to internal errors and cryptic crashes due to incompatibility between
PyTorch Profiler and Lightning's training loop.
Always use this ``profiler`` argument to enable profiling in Lightning.


detect_anomaly: Enable anomaly detection for the autograd engine.
Default: ``False``.

Expand Down
Loading