[Feature] Compiled and cudagraph for policies

[vmoens]

Stack from ghstack (oldest at bottom):

• -> [Feature] Compiled and cudagraph for policies #2478

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2478

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 13 New Failures, 4 Unrelated Failures

As of commit 2dcc23e with merge base efa5745 ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / CPU Pytest benchmark (gh)
  /home/runner/work/_temp/3071ba60-0be5-438b-8688-d866f497ff7b.sh: line 5: 2163 Bus error (core dumped) $RUN_BENCHMARK /tmp/tmp.kAudBNhvHa
• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  /__w/_temp/2623a2dc-6fa5-43af-ad25-8af424f43def.sh: line 5: 10150 Bus error (core dumped) $RUN_BENCHMARK /tmp/tmp.ReiU7b06EJ
• Examples Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  ##[error]fatal: couldn't find remote ref refs/pull/2478/merge
• Generate documentation / build-docs (3.10, 12.1) / linux-job (gh)
  ##[error]No files were found with the provided path: /home/ec2-user/actions-runner/_work/_temp/artifacts/. No artifacts will be uploaded.
• Habitat Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  ##[error]fatal: couldn't find remote ref refs/pull/2478/merge
• Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job (gh)
  ##[error]fatal: couldn't find remote ref refs/pull/2478/merge
• Libs Tests on Linux / unittests-sklearn (3.9, 12.1) / linux-job (gh)
  ##[error]fatal: couldn't find remote ref refs/pull/2478/merge
• RLHF Tests on Linux / unittests (3.9, 12.1) / linux-job (gh)
  ##[error]fatal: couldn't find remote ref refs/pull/2478/merge
• Unit-tests on Linux / tests-gpu (3.11, 12.1) / linux-job (gh)
  ##[error]fatal: couldn't find remote ref refs/pull/2478/merge
• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh)
  ##[error]fatal: couldn't find remote ref refs/pull/2478/merge
• Unit-tests on Linux / tests-optdeps (3.11, 12.1) / linux-job (gh)
  ##[error]fatal: couldn't find remote ref refs/pull/2478/merge
• Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job (gh)
  ##[error]fatal: couldn't find remote ref refs/pull/2478/merge
• Unit-tests on Windows / unittests-cpu / windows-job (gh)
  ##[error]fatal: couldn't find remote ref refs/pull/2478/merge

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Build Windows Wheels / pytorch/rl / upload / wheel-py3_9-cpu (gh) (trunk failure)
  ##[error]Unable to find any artifacts for the associated workflow
• Build Windows Wheels / pytorch/rl / upload / wheel-py3_9-cuda11_8 (gh) (trunk failure)
  ##[error]Unable to find any artifacts for the associated workflow
• Build Windows Wheels / pytorch/rl / upload / wheel-py3_9-cuda12_1 (gh) (trunk failure)
  ##[error]Unable to find any artifacts for the associated workflow
• Build Windows Wheels / pytorch/rl / upload / wheel-py3_9-cuda12_4 (gh) (trunk failure)
  ##[error]Unable to find any artifacts for the associated workflow

This comment was automatically generated by Dr. CI and updates every 15 minutes.