[Inductor] Fix the High Order Op layout issue (#128275)

[zou3519]

Fix the issue: #127995

• In current implementation of creating FallbackKernel, the device of the NoneLayout is set to None when example_output returns from cls.process_kernel is None.

  pytorch/torch/_inductor/ir.py

  Lines 5632 to 5649 in 921aa19

  	with context:
  	(
  	example_output,
  	tensor_args,
  	non_tensor_args,
  	unflatten_args,
  	unbacked_bindings,
  	) = cls.process_kernel(kernel, *args, **kwargs)
  	if example_output is None:
  	packed = cls(
  	NoneLayout(None),
  	kernel,
  	tensor_args,
  	non_tensor_args,
  	unflatten_args,
  	unbacked_bindings=unbacked_bindings,
  	)

• If a ExternalKernel schedulerNode has None device, the previous buffer will not flush before codegen this ExternalKernel schedulerNode which causes the wrong generated code.

  pytorch/torch/_inductor/scheduler.py

  Lines 2701 to 2709 in ef2b5ed

  	if not isinstance(node, NopKernelSchedulerNode) and (
  	device := node.get_device()
  	):
  	if (
  	device != self.current_device
  	or node.is_extern()
  	or node.is_template()
  	):
  	self.flush()

Test Plan

python -u -m pytest -s -v test/higher_order_ops/test_with_effects.py -k test_compile_inductor_external_op_return_none

Pull Request resolved: #128275
Approved by: https://github.com/eellison

Fixes #ISSUE_NUMBER

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire @chauhang

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/128834

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• inductor_torchbench_perf jobs are broken due to numpy 2.0 update

❌ 17 New Failures

As of commit e41397d with merge base b66e3f0 ():

NEW FAILURES - The following jobs have failed:

• inductor / cuda12.1-py3.10-gcc9-sm86 / test (aot_inductor_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
  AttributeError: np.float_was removed in the NumPy 2.0 release. Usenp.float64 instead.. Did you mean: 'float16'?
• inductor / cuda12.1-py3.10-gcc9-sm86 / test (aot_inductor_torchbench, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
  AttributeError: np.float_was removed in the NumPy 2.0 release. Usenp.float64 instead.. Did you mean: 'float16'?
• inductor / cuda12.1-py3.10-gcc9-sm86 / test (dynamic_inductor_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
  AttributeError: np.float_was removed in the NumPy 2.0 release. Usenp.float64 instead.. Did you mean: 'float16'?
• inductor / cuda12.1-py3.10-gcc9-sm86 / test (dynamic_inductor_torchbench, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
  AttributeError: np.float_was removed in the NumPy 2.0 release. Usenp.float64 instead.. Did you mean: 'float16'?
• inductor / cuda12.1-py3.10-gcc9-sm86 / test (inductor_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
  AttributeError: np.float_was removed in the NumPy 2.0 release. Usenp.float64 instead.. Did you mean: 'float16'?
• inductor / cuda12.1-py3.10-gcc9-sm86 / test (inductor_torchbench, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
  AttributeError: np.float_was removed in the NumPy 2.0 release. Usenp.float64 instead.. Did you mean: 'float16'?
• inductor / linux-jammy-cpu-py3.8-gcc11-inductor / test (cpu_inductor_torchbench_freezing, 1, 2, linux.12xlarge) (gh)
  DALLE2_pytorch
• inductor / linux-jammy-cpu-py3.8-gcc11-inductor / test (cpu_inductor_torchbench, 1, 2, linux.12xlarge) (gh)
  DALLE2_pytorch
• inductor / linux-jammy-cpu-py3.8-gcc11-inductor / test (dynamic_cpu_inductor_torchbench, 1, 2, linux.12xlarge) (gh)
  DALLE2_pytorch
• inductor-periodic / cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (aot_eager_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
  AttributeError: np.float_was removed in the NumPy 2.0 release. Usenp.float64 instead.. Did you mean: 'float16'?
• inductor-periodic / cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (aot_eager_torchbench, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
  AttributeError: np.float_was removed in the NumPy 2.0 release. Usenp.float64 instead.. Did you mean: 'float16'?
• inductor-periodic / cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamic_aot_eager_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
  AttributeError: np.float_was removed in the NumPy 2.0 release. Usenp.float64 instead.. Did you mean: 'float16'?
• inductor-periodic / cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamic_aot_eager_torchbench, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
  AttributeError: np.float_was removed in the NumPy 2.0 release. Usenp.float64 instead.. Did you mean: 'float16'?
• inductor-periodic / cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
  AttributeError: np.float_was removed in the NumPy 2.0 release. Usenp.float64 instead.. Did you mean: 'float16'?
• inductor-periodic / cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_torchbench, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
  AttributeError: np.float_was removed in the NumPy 2.0 release. Usenp.float64 instead.. Did you mean: 'float16'?
• pull / linux-focal-py3_8-clang9-xla / test (xla, 1, 1, linux.12xlarge) (gh)
  test_repeat_truncated
• pull / linux-focal-py3.8-clang10-onnx / test (default, 2, 2, linux.2xlarge) (gh)
  onnx/test_fx_op_consistency.py::TestOnnxModelOutputConsistency_opset_version_18_model_type_TorchModelType.TORCH_EXPORT_EXPORTEDPROGRAMCPU::test_output_match_nn_functional_huber_loss_cpu_float16

This comment was automatically generated by Dr. CI and updates every 15 minutes.