[sym_shapes][perf] Optimize bound_sympy avoiding sympy equals #124211

IvanKobzarev · 2024-04-16T20:36:16Z

Stack from ghstack (oldest at bottom):

-> [sym_shapes][perf] Optimize bound_sympy avoiding sympy equals #124211

cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10

Differential Revision: D56212479

[ghstack-poisoned]

pytorch-bot · 2024-04-16T20:36:18Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/124211

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 28 New Failures, 5 Unrelated Failures

As of commit 1ab00e5 with merge base 42e22bb ():

NEW FAILURES - The following jobs have failed:

Lint / lintrunner-noclang / linux-job (gh)
>>> Lint for torch/utils/_sympy/value_ranges.py:
pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 2, 5, linux.4xlarge.nvidia.gpu) (gh)
inductor/test_torchinductor_dynamic_shapes.py::DynamicShapesCpuTests::test_adaptive_avg_pool1d_argmax_dynamic_shapes_cpu
pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 3, 5, linux.4xlarge.nvidia.gpu) (gh)
inductor/test_inductor_freezing.py::FreezingCpuTests::test_conv_layout_convert_with_view_cpu
pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 4, 5, linux.4xlarge.nvidia.gpu) (gh)
inductor/test_cpu_repro.py::CPUReproTests::test_ModularIndexing_range_issue_103133
pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 5, 5, linux.4xlarge.nvidia.gpu) (gh)
dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_addr_alpha_beta_out_dynamic_shapes
pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 1, 5, linux.g5.4xlarge.nvidia.gpu) (gh)
inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast1_broadcast2
pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 2, 5, linux.g5.4xlarge.nvidia.gpu) (gh)
inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast1_dense
pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 3, 5, linux.g5.4xlarge.nvidia.gpu) (gh)
dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_addr_alpha_beta_out_dynamic_shapes
pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 4, 5, linux.g5.4xlarge.nvidia.gpu) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive___getitem___cuda_float32
pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 5, 5, linux.g5.4xlarge.nvidia.gpu) (gh)
inductor/test_inductor_freezing.py::FreezingCpuTests::test_conv_layout_convert_with_view_cpu
pull / linux-focal-py3.11-clang10 / test (default, 1, 3, linux.2xlarge) (gh)
inductor/test_torchinductor.py::SweepInputsCpuTest::test_cpu_broadcast1_strided
pull / linux-focal-py3.11-clang10 / test (default, 2, 3, linux.2xlarge) (gh)
dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_addr_alpha_beta_out_dynamic_shapes
pull / linux-focal-py3.11-clang10 / test (default, 3, 3, linux.2xlarge) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive__chunk_cat_cpu_float32
pull / linux-focal-py3.12-clang10 / test (default, 1, 3, linux.2xlarge) (gh)
inductor/test_torchinductor.py::SweepInputsCpuTest::test_cpu_broadcast1_strided
pull / linux-focal-py3.12-clang10 / test (default, 2, 3, linux.2xlarge) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive___getitem___cpu_float64
pull / linux-focal-py3.12-clang10 / test (default, 3, 3, linux.2xlarge) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive__chunk_cat_cpu_float32
pull / linux-focal-py3.8-clang10 / test (default, 1, 3, linux.2xlarge) (gh)
inductor/test_torchinductor.py::SweepInputsCpuTest::test_cpu_broadcast1_strided
pull / linux-focal-py3.8-clang10 / test (default, 2, 3, linux.2xlarge) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive___getitem___cpu_float64
pull / linux-focal-py3.8-clang10 / test (default, 3, 3, linux.2xlarge) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive__chunk_cat_cpu_float32
pull / linux-jammy-py3.10-clang15-asan / test (default, 1, 6, linux.4xlarge) (gh)
inductor/test_torchinductor.py::SweepInputsCpuTest::test_cpu_broadcast1_strided
pull / linux-jammy-py3.10-clang15-asan / test (default, 2, 6, linux.4xlarge) (gh)
inductor/test_torchinductor.py::SweepInputsCpuTest::test_cpu_broadcast1_double
pull / linux-jammy-py3.10-clang15-asan / test (default, 3, 6, linux.4xlarge) (gh)
inductor/test_torchinductor.py::SweepInputsCpuTest::test_cpu_broadcast1_broadcast2
pull / linux-jammy-py3.10-clang15-asan / test (default, 4, 6, linux.4xlarge) (gh)
dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_addr_alpha_beta_out_dynamic_shapes
pull / linux-jammy-py3.10-clang15-asan / test (default, 5, 6, linux.4xlarge) (gh)
dynamo/test_repros.py::ReproTests::test_addr_alpha_beta_out
pull / linux-jammy-py3.10-clang15-asan / test (default, 6, 6, linux.4xlarge) (gh)
inductor/test_inductor_freezing.py::FreezingCpuTests::test_conv_layout_convert_with_view_cpu
pull / linux-jammy-py3.8-gcc11 / test (default, 1, 3, linux.2xlarge) (gh)
inductor/test_torchinductor.py::SweepInputsCpuTest::test_cpu_broadcast1_strided
pull / linux-jammy-py3.8-gcc11 / test (default, 2, 3, linux.2xlarge) (gh)
dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_addr_alpha_beta_out_dynamic_shapes
pull / linux-jammy-py3.8-gcc11 / test (default, 3, 3, linux.2xlarge) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive__chunk_cat_cpu_float16

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

pull / linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 1, 3, linux.8xlarge.nvidia.gpu) (gh)
distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_call_method_forward
pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 1, 5, linux.4xlarge.nvidia.gpu) (gh)
inductor/test_torchinductor_dynamic_shapes.py::DynamicShapesCpuTests::test_add_inplace_permuted_dynamic_shapes_cpu
pull / linux-focal-py3.11-clang10 / test (crossref, 1, 2, linux.2xlarge) (gh)
inductor/test_mmdecomp.py::TestDecompCPU::test_batched_mm_float32_bs_10_cpu_float32
pull / linux-focal-py3.8-clang10 / test (crossref, 1, 2, linux.2xlarge) (gh)
inductor/test_mmdecomp.py::TestDecompCPU::test_batched_mm_float32_bs_10_cpu_float32
pull / linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, linux.2xlarge) (gh)
distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_call_method_forward

This comment was automatically generated by Dr. CI and updates every 15 minutes.

ghstack-source-id: 149f11b60ca49c364bd3ca120e32cd30921efcc1 Pull Request resolved: #124211

ezyang · 2024-04-16T21:04:37Z

torch/utils/_sympy/value_ranges.py

+ for r in ranges.keys():
+ if r.name == name:
+ found = True
+ break


It's so nuts that this is faster. cc @lezcano

lol cc @asmeurer @isuruf

Also, if not any(...): add

Is there a possibility to have different symbols with different properties (like is_Integer) to have the same name?

No, a symbol is identified by its name in dynamo/inductor.

As Isuru noted, Symbols are compared by name + assumptions. That comparison should be fast, but I suppose it's possible there is some performance issue somewhere. It would be useful to have more precise timing information here.

ezyang · 2024-04-16T21:08:20Z

Oh, this string thing! Crazy stuff! Will need to be debugged, sounds like a preexisting problem

IvanKobzarev · 2024-04-16T21:29:18Z

@IvanKobzarev has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

ezyang · 2024-04-17T14:43:50Z

@IvanKobzarev I did a simple microbenchmark to try to reproduce this problem but I didn't see as big an effect:

import sympy
import itertools

N = 50000

x = set(sympy.Symbol(f"s{i}", integer=True) for i in range(N*2))
y = set(sympy.Symbol(f"s{i}", integer=True) for i in range(N))

print(len(x - y))

Is it possible to get printouts of the inputs into this expression on the slow run?

IvanKobzarev · 2024-04-17T14:52:06Z

Ok. Will try to repro it on the original run with printouts.

ezyang · 2024-04-18T03:15:51Z

This isn't blocking for this PR, but it's not obvious to me that we should be going out and rewriting all of our sympy symbol sets

lezcano · 2024-04-18T05:03:30Z

torch/utils/_sympy/value_ranges.py

+ ranges_names = [r.name for r in ranges.keys()]
+ for fs in expr.free_symbols:
+ if fs.name not in ranges_names:
+ unbounded_vars.add(fs)


Make range_names a set and add range_names - expr.free_symbols,

github-actions · 2024-06-22T22:33:38Z

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.

[sym_shapes][perf] Optimize bound_sympy avoiding sympy equals

1ab00e5

[ghstack-poisoned]

pytorch-bot bot added the module: cpu CPU specific problem (e.g., perf, algorithm) label Apr 16, 2024

IvanKobzarev added a commit that referenced this pull request Apr 16, 2024

[sym_shapes][perf] Optimize bound_sympy avoiding sympy equals

04f5d79

ghstack-source-id: 149f11b60ca49c364bd3ca120e32cd30921efcc1 Pull Request resolved: #124211

IvanKobzarev mentioned this pull request Apr 16, 2024

maybe_evaluate_static performance problem with unbacked SymInts #123964

Open

ezyang reviewed Apr 16, 2024

View reviewed changes

ezyang requested review from lezcano and avikchaudhuri April 18, 2024 03:25

lezcano reviewed Apr 18, 2024

View reviewed changes

github-actions bot added the Stale label Jun 22, 2024

github-actions bot closed this Jul 22, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[sym_shapes][perf] Optimize bound_sympy avoiding sympy equals #124211

[sym_shapes][perf] Optimize bound_sympy avoiding sympy equals #124211

IvanKobzarev commented Apr 16, 2024 •

edited

Loading

pytorch-bot bot commented Apr 16, 2024 •

edited

Loading

ezyang Apr 16, 2024

lezcano Apr 18, 2024

isuruf Apr 18, 2024

lezcano Apr 18, 2024

asmeurer Apr 23, 2024

ezyang commented Apr 16, 2024

IvanKobzarev commented Apr 16, 2024

ezyang commented Apr 17, 2024

IvanKobzarev commented Apr 17, 2024

ezyang commented Apr 18, 2024

lezcano Apr 18, 2024 •

edited

Loading

github-actions bot commented Jun 22, 2024

[sym_shapes][perf] Optimize bound_sympy avoiding sympy equals #124211

[sym_shapes][perf] Optimize bound_sympy avoiding sympy equals #124211

Conversation

IvanKobzarev commented Apr 16, 2024 • edited Loading

pytorch-bot bot commented Apr 16, 2024 • edited Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/124211

❌ 28 New Failures, 5 Unrelated Failures

ezyang Apr 16, 2024

Choose a reason for hiding this comment

lezcano Apr 18, 2024

Choose a reason for hiding this comment

isuruf Apr 18, 2024

Choose a reason for hiding this comment

lezcano Apr 18, 2024

Choose a reason for hiding this comment

asmeurer Apr 23, 2024

Choose a reason for hiding this comment

ezyang commented Apr 16, 2024

IvanKobzarev commented Apr 16, 2024

ezyang commented Apr 17, 2024

IvanKobzarev commented Apr 17, 2024

ezyang commented Apr 18, 2024

lezcano Apr 18, 2024 • edited Loading

Choose a reason for hiding this comment

github-actions bot commented Jun 22, 2024

IvanKobzarev commented Apr 16, 2024 •

edited

Loading

pytorch-bot bot commented Apr 16, 2024 •

edited

Loading

lezcano Apr 18, 2024 •

edited

Loading