WIP Add 3D channels last tensor iterator support

[dmenig]

Part of a multi-PR work to improve #59168

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/118377

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 36 New Failures, 3 Cancelled Jobs, 4 Unrelated Failures

As of commit 1e03b32 with merge base 68a1f78 ():

NEW FAILURES - The following jobs have failed:

• Lint / lintrunner-clang / linux-job (gh)
  Process completed with exit code 1.
• Lint / lintrunner-noclang / linux-job (gh)
  Process completed with exit code 1.
• pull / linux-focal-cpu-py3.10-gcc9-bazel-test / build-and-test (default, 1, 1, amz2023.linux.4xlarge) (gh)
  Process completed with exit code 1.
• pull / linux-focal-cuda11.8-py3.10-gcc9 / build (gh)
  Process completed with exit code 1.
• pull / linux-focal-cuda12.1-py3.10-gcc9 / build (gh)
  Process completed with exit code 1.
• pull / linux-focal-cuda12.1-py3.10-gcc9-bazel-test / build-and-test (default, 1, 1, amz2023.linux.4xlarge.nvidia.gpu) (gh)
  Process completed with exit code 1.
• pull / linux-focal-cuda12.1-py3.10-gcc9-experimental-split-build / build (gh)
  Process completed with exit code 1.
• pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / build (gh)
  Process completed with exit code 1.
• pull / linux-focal-cuda12.4-py3.10-gcc9-bazel-test / build-and-test (default, 1, 1, amz2023.linux.4xlarge.nvidia.gpu) (gh)
  Process completed with exit code 1.
• pull / linux-focal-py3.11-clang10 / test (crossref, 1, 2, amz2023.linux.2xlarge) (gh)
  test_fake_tensor.py::FakeTensorTest::test__adaptive_avg_pool3d_backward
• pull / linux-focal-py3.11-clang10 / test (crossref, 2, 2, amz2023.linux.2xlarge) (gh)
  test_ops.py::TestCompositeComplianceCPU::test_cow_input_nn_functional_adaptive_avg_pool3d_cpu_float32
• pull / linux-focal-py3.11-clang10 / test (default, 1, 4, amz2023.linux.2xlarge) (gh)
  test_fake_tensor.py::FakeTensorTest::test__adaptive_avg_pool3d_backward
• pull / linux-focal-py3.11-clang10 / test (default, 2, 4, amz2023.linux.2xlarge) (gh)
  test_ops.py::TestCompositeComplianceCPU::test_cow_input_nn_functional_adaptive_avg_pool3d_cpu_float32
• pull / linux-focal-py3.11-clang10 / test (default, 3, 4, amz2023.linux.2xlarge) (gh)
  test_ops.py::TestCompositeComplianceCPU::test_cow_input_nn_functional_interpolate_area_cpu_float32
• pull / linux-focal-py3.11-clang10 / test (dynamo, 1, 3, amz2023.linux.2xlarge) (gh)
  test_fake_tensor.py::FakeTensorTest::test__adaptive_avg_pool3d_backward
• pull / linux-focal-py3.12-clang10 / test (default, 1, 4, amz2023.linux.2xlarge) (gh)
  test_fake_tensor.py::FakeTensorTest::test__adaptive_avg_pool3d_backward
• pull / linux-focal-py3.12-clang10 / test (default, 2, 4, amz2023.linux.2xlarge) (gh)
  test_ops.py::TestCompositeComplianceCPU::test_cow_input_nn_functional_adaptive_avg_pool3d_cpu_float32
• pull / linux-focal-py3.12-clang10 / test (default, 3, 4, amz2023.linux.2xlarge) (gh)
  test_ops.py::TestCompositeComplianceCPU::test_cow_input_nn_functional_interpolate_area_cpu_float32
• pull / linux-focal-py3.12-clang10 / test (dynamo, 1, 3, amz2023.linux.2xlarge) (gh)
  test_fake_tensor.py::FakeTensorTest::test__adaptive_avg_pool3d_backward
• pull / linux-focal-py3.12-clang10-experimental-split-build / test (default, 1, 3, amz2023.linux.2xlarge) (gh)
  test_fake_tensor.py::FakeTensorTest::test__adaptive_avg_pool3d_backward
• pull / linux-focal-py3.12-clang10-experimental-split-build / test (default, 2, 3, amz2023.linux.2xlarge) (gh)
  test_ops.py::TestCompositeComplianceCPU::test_cow_input_nn_functional_adaptive_avg_pool3d_cpu_float32
• pull / linux-focal-py3.12-clang10-experimental-split-build / test (default, 3, 3, amz2023.linux.2xlarge) (gh)
  test_ops.py::TestCompositeComplianceCPU::test_cow_input_nn_functional_interpolate_area_cpu_float32
• pull / linux-focal-py3.12-clang10-experimental-split-build / test (dynamo, 1, 3, amz2023.linux.2xlarge) (gh)
  test_fake_tensor.py::FakeTensorTest::test__adaptive_avg_pool3d_backward
• pull / linux-focal-py3.8-clang10 / test (crossref, 1, 2, amz2023.linux.2xlarge) (gh)
  test_fake_tensor.py::FakeTensorTest::test__adaptive_avg_pool3d_backward
• pull / linux-focal-py3.8-clang10 / test (crossref, 2, 2, amz2023.linux.2xlarge) (gh)
  test_ops.py::TestCompositeComplianceCPU::test_cow_input_nn_functional_adaptive_avg_pool3d_cpu_float32
• pull / linux-focal-py3.8-clang10 / test (default, 1, 4, amz2023.linux.2xlarge) (gh)
  test_fake_tensor.py::FakeTensorTest::test__adaptive_avg_pool3d_backward
• pull / linux-focal-py3.8-clang10 / test (default, 2, 4, amz2023.linux.2xlarge) (gh)
  test_ops.py::TestCompositeComplianceCPU::test_cow_input_nn_functional_adaptive_avg_pool3d_cpu_float32
• pull / linux-focal-py3.8-clang10 / test (default, 3, 4, amz2023.linux.2xlarge) (gh)
  test_ops.py::TestCompositeComplianceCPU::test_cow_input_nn_functional_interpolate_area_cpu_float32
• pull / linux-focal-py3.8-clang10 / test (dynamo, 1, 3, amz2023.linux.2xlarge) (gh)
  test_fake_tensor.py::FakeTensorTest::test__adaptive_avg_pool3d_backward
• pull / linux-jammy-cuda11.8-cudnn9-py3.8-clang12 / build (gh)
  Process completed with exit code 1.
• pull / linux-jammy-py3.10-clang15-asan / test (default, 1, 6, amz2023.linux.4xlarge) (gh)
  test_fake_tensor.py::FakeTensorTest::test__adaptive_avg_pool3d_backward
• pull / linux-jammy-py3.10-clang15-asan / test (default, 3, 6, amz2023.linux.4xlarge) (gh)
  test_ops.py::TestCompositeComplianceCPU::test_cow_input_nn_functional_interpolate_area_cpu_float32
• pull / linux-jammy-py3.10-clang15-asan / test (default, 5, 6, amz2023.linux.4xlarge) (gh)
  test_ops.py::TestCompositeComplianceCPU::test_cow_input_nn_functional_adaptive_avg_pool3d_cpu_float32
• pull / linux-jammy-py3.8-gcc11 / test (default, 1, 4, amz2023.linux.2xlarge) (gh)
  test_fake_tensor.py::FakeTensorTest::test__adaptive_avg_pool3d_backward
• pull / linux-jammy-py3.8-gcc11 / test (default, 2, 4, amz2023.linux.2xlarge) (gh)
  test_ops.py::TestCompositeComplianceCPU::test_cow_input_nn_functional_adaptive_avg_pool3d_cpu_float32
• pull / linux-jammy-py3.8-gcc11 / test (default, 3, 4, amz2023.linux.2xlarge) (gh)
  test_ops.py::TestCompositeComplianceCPU::test_cow_input_nn_functional_interpolate_area_cpu_float32

CANCELLED JOBS - The following jobs were cancelled. Please retry:

• inductor / cuda12.1-py3.10-gcc9-sm80 / build (gh)
  ##[error]The operation was canceled.
• inductor / cuda12.1-py3.10-gcc9-sm86 / build (gh)
  ##[error]The operation was canceled.
• inductor-periodic / cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / build (gh)
  ##[error]The operation was canceled.

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• pull / linux-focal-py3.11-clang10 / test (dynamo, 2, 3, amz2023.linux.2xlarge) (gh) (disabled by #104012 but the issue was closed recently and a rebase is needed to make it pass)
  test_fx.py::TestFXAPIBackwardCompatibility::test_public_api_surface
• pull / linux-focal-py3.12-clang10 / test (dynamo, 2, 3, amz2023.linux.2xlarge) (gh) (disabled by #104012 but the issue was closed recently and a rebase is needed to make it pass)
  test_fx.py::TestFXAPIBackwardCompatibility::test_public_api_surface
• pull / linux-focal-py3.12-clang10-experimental-split-build / test (dynamo, 2, 3, amz2023.linux.2xlarge) (gh) (disabled by #104012 but the issue was closed recently and a rebase is needed to make it pass)
  test_fx.py::TestFXAPIBackwardCompatibility::test_public_api_surface
• pull / linux-focal-py3.8-clang10 / test (dynamo, 2, 3, amz2023.linux.2xlarge) (gh) (disabled by #104012 but the issue was closed recently and a rebase is needed to make it pass)
  test_fx.py::TestFXAPIBackwardCompatibility::test_public_api_surface

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.

[dmenig]

@peterbell10 could you please advise me ? In all honesty Idk what I'm doing. I understand that the strides of the grad output are making this test fail, but I can't udnerstand what I've done wrong.

[peterbell10]

This is running under fake tensor mode which uses the meta-registration from torch/_meta_registrations.py rather than the concrete implementation. So this test isn't hitting your code at all.

[dmenig]

Damn. I modified the aten library though. Do you have any idea why the registration doesn't take into account my aten modifications ?

[peterbell10]

The meta function doesn't call into aten, you need to update it directly:

pytorch/torch/_meta_registrations.py

Lines 2830 to 2850 in 499ead9

	@register_meta(aten._adaptive_avg_pool2d_backward.default)
	def meta__adaptive_avg_pool2d_backward(grad_out, self):
	ndim = grad_out.ndim
	for i in range(1, ndim):
	torch._check(
	grad_out.size(i) > 0,
	lambda: f"adaptive_avg_pool2d_backward(): Expected grad_output to have non-zero \
	size for non-batch dimensions, {grad_out.shape} with dimension {i} being empty",
	)
	torch._check(
	ndim == 3 or ndim == 4,
	lambda: f"adaptive_avg_pool2d_backward(): Expected 3D or 4D tensor, but got {self.shape}",
	)
	torch._check(
	self.dtype == grad_out.dtype,
	lambda: f"expected dtype {self.dtype} for `grad_output` but got dtype {grad_out.dtype}",
	)
	memory_format = torch.contiguous_format
	if is_channels_last(self):
	memory_format = torch.channels_last
	return self.new_empty(self.shape).to(memory_format=memory_format)