Silently disable default saved tensor hooks during tracing (#123196)

Summary:
FIXES #113263. Same idea as in pytorch/pytorch#113417, but we need a more intrusive C API to silently nop default saved tensor hooks, in order to support user-code that use torch.autograd.disable_saved_tensors_hooks (see test_unpack_hooks_can_be_disabled). We mock the output of get_hooks while leaving push/pop untouched.

For compiled autograd, we're firing pack hooks once and unpack hooks twice right now, I'll look into this separately from this issue.

X-link: pytorch/pytorch#123196
Approved by: https://github.com/soulitzer

Reviewed By: izaitsevfb

Differential Revision: D58628125

Pulled By: xmfan

fbshipit-source-id: 399cba07d50610ba52da55951f7b2200eb9c217f

userbenchmark/dynamo/dynamobench/_dynamo/utils.py

@@ -2773,3 +2773,13 @@ def strip_color_from_string(text):
  # This regular expression matches ANSI escape codes
  ansi_escape = re.compile(r"\x1B[@-_][0-?]*[ -/]*[@-~]")
  return ansi_escape.sub("", text)
+
+
+@contextlib.contextmanager
+def _disable_saved_tensors_hooks_during_tracing():
+ # See NOTE: [Deferring tensor pack/unpack hooks until runtime]
+ try:
+ prior = torch._C._autograd._saved_tensors_hooks_set_tracing(True)
+ yield
+ finally:
+ torch._C._autograd._saved_tensors_hooks_set_tracing(prior)