Add testing for PyTorch 2.4 (Trainer) #20010
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #20010 +/- ##
=======================================
Coverage 90% 91%
=======================================
Files 266 266
Lines 22942 22952 +10
=======================================
+ Hits 20721 20780 +59
+ Misses 2221 2172 -49 |
e12ea4c
to
90d61a3
Compare
95c4b36
to
65b86c5
Compare
808129e
to
2b4f413
Compare
73f9bea
to
ef7955e
Compare
⚡ Required checks status: All passing 🟢Groups summary🟢 pytorch_lightning: Tests workflow
These checks are required after the changes to 🟢 pytorch_lightning: Azure GPU
These checks are required after the changes to 🟢 pytorch_lightning: Benchmarks
These checks are required after the changes to 🟢 fabric: Docs
These checks are required after the changes to 🟢 pytorch_lightning: Docs
These checks are required after the changes to 🟢 pytorch_lightning: Docker
These checks are required after the changes to 🟢 lightning_fabric: CPU workflowThese checks are required after the changes to 🟢 lightning_fabric: Azure GPU
These checks are required after the changes to 🟢 mypy
These checks are required after the changes to 🟢 installThese checks are required after the changes to Thank you for your contribution! 💜
|
What does this PR do?
Adds testing PyTorch 2.4 to the CI matrix.
Previous PR for reference: #19289
The main updates are:
PyTorch wants to flip the default of
torch.load(..., weights_only=)to True in the future. In 2.4, they start raising a FutureWarning if you're not explicitly setting this argument to either True or False. This is the case in our code base. For checkpoint loading, we need to setweights_only=Falsefor now because so far we allowed users to include arbitrary objects in checkpoints, and right now the user has no control over this flag internally in the Trainer. In the future, we might want to expose this: Exposeweights_onlyoption for loading checkpoints #20058FSDP is deprecating the state-dict APIs and raises a warning:
Since it is a non-trivial effort to switch to the new ones and also provide backward-compatibility of older PyTorch versions, I will defer this work to future PRs (Use new state-dict APIs in FSDPStrategy #20060) and for now suppress this warnings (for the user and also in our tests).
The PyTorchProfiler is deprecating the use of the
use_cudaargument. I updated the logic in our Profiler wrapper to avoid passing this argument, but kept it backward-compatible with older PyTorch versions.cc @Borda @carmocca @justusschock @awaelchli