{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":92541759,"defaultBranch":"main","name":"benchmark","ownerLogin":"pytorch","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2017-05-26T19:21:12.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/21003710?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1719008601.0","currentOid":""},"activityList":{"items":[{"before":"8ed33eab139c80dee01142910d4548f9f46c8213","after":"cd84ec9dcf8c436b8712b1fe03f4abd63baf33c4","ref":"refs/heads/xz9/improve-ci","pushedAt":"2024-06-21T22:52:49.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"xuzhao9","name":"Xu Zhao","path":"/xuzhao9","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/502017?s=80&v=4"},"commit":{"message":"Another fix","shortMessageHtmlLink":"Another fix"}},{"before":"e289e9f68b6d24e7ab6d9c16f9c09ee82a19a717","after":"8ed33eab139c80dee01142910d4548f9f46c8213","ref":"refs/heads/xz9/improve-ci","pushedAt":"2024-06-21T22:52:06.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"xuzhao9","name":"Xu Zhao","path":"/xuzhao9","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/502017?s=80&v=4"},"commit":{"message":"Fix concurrent jobs","shortMessageHtmlLink":"Fix concurrent jobs"}},{"before":"e8c04d55de81aca5624a71d78e82773f59a04290","after":"e289e9f68b6d24e7ab6d9c16f9c09ee82a19a717","ref":"refs/heads/xz9/improve-ci","pushedAt":"2024-06-21T22:43:20.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"xuzhao9","name":"Xu Zhao","path":"/xuzhao9","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/502017?s=80&v=4"},"commit":{"message":"Add debugging","shortMessageHtmlLink":"Add debugging"}},{"before":"eaccc33e39353b25f9779e962bc3463845fd115b","after":"e8c04d55de81aca5624a71d78e82773f59a04290","ref":"refs/heads/xz9/improve-ci","pushedAt":"2024-06-21T22:40:12.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"xuzhao9","name":"Xu Zhao","path":"/xuzhao9","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/502017?s=80&v=4"},"commit":{"message":"Test the conda env","shortMessageHtmlLink":"Test the conda env"}},{"before":"e06fbaf95363714dbd7edaf84f3154ac9e36416e","after":"eaccc33e39353b25f9779e962bc3463845fd115b","ref":"refs/heads/xz9/improve-ci","pushedAt":"2024-06-21T22:32:36.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"xuzhao9","name":"Xu Zhao","path":"/xuzhao9","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/502017?s=80&v=4"},"commit":{"message":"Add test yml","shortMessageHtmlLink":"Add test yml"}},{"before":"1705527aa4d057435c52a6b932b193b6ec85c3d0","after":"e06fbaf95363714dbd7edaf84f3154ac9e36416e","ref":"refs/heads/xz9/improve-ci","pushedAt":"2024-06-21T22:28:49.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"xuzhao9","name":"Xu Zhao","path":"/xuzhao9","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/502017?s=80&v=4"},"commit":{"message":"Bugfix","shortMessageHtmlLink":"Bugfix"}},{"before":"4078588d41ae6f7b652bb0c8948f0dff14a641f7","after":"1705527aa4d057435c52a6b932b193b6ec85c3d0","ref":"refs/heads/xz9/improve-ci","pushedAt":"2024-06-21T22:26:13.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"xuzhao9","name":"Xu Zhao","path":"/xuzhao9","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/502017?s=80&v=4"},"commit":{"message":"Another test","shortMessageHtmlLink":"Another test"}},{"before":null,"after":"4078588d41ae6f7b652bb0c8948f0dff14a641f7","ref":"refs/heads/xz9/improve-ci","pushedAt":"2024-06-21T22:23:21.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"xuzhao9","name":"Xu Zhao","path":"/xuzhao9","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/502017?s=80&v=4"},"commit":{"message":"Improve the ci runners","shortMessageHtmlLink":"Improve the ci runners"}},{"before":"967e7468aef53cb8d2ad75009bbcd694c89a913f","after":null,"ref":"refs/heads/xz9/fix-ci","pushedAt":"2024-06-21T20:55:20.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"xuzhao9","name":"Xu Zhao","path":"/xuzhao9","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/502017?s=80&v=4"}},{"before":"1425f68ec53f5fb341e34bcc49ac62ddef3692c1","after":"d910b8aefdb3a247f441557ba413d6a352456904","ref":"refs/heads/main","pushedAt":"2024-06-21T16:22:47.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Fix the soft_actor_critic model (#2326)\n\nSummary:\nUnfortunately, https://github.com/pytorch/benchmark/pull/2318 has a bug that breaks the `soft_actor_critic` model.\n\nPull Request resolved: https://github.com/pytorch/benchmark/pull/2326\n\nReviewed By: aaronenyeshi\n\nDifferential Revision: D58871386\n\nPulled By: xuzhao9\n\nfbshipit-source-id: 5f8b5fbe00722ccb647b08a8089fd52a7719208c","shortMessageHtmlLink":"Fix the soft_actor_critic model (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2365515854\" data-permission-text=\"Title is private\" data-url=\"https://github.com/pytorch/benchmark/issues/2326\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/pytorch/benchmark/pull/2326/hovercard\" href=\"https://github.com/pytorch/benchmark/pull/2326\">#2326</a>)"}},{"before":"53faa0a2c87e77e6545722ce8cf4b2cba569c3a7","after":"1425f68ec53f5fb341e34bcc49ac62ddef3692c1","ref":"refs/heads/main","pushedAt":"2024-06-21T08:23:17.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Add variable seqlen and sparsity parameters to jagged_sum benchmark (#2324)\n\nSummary:\nPull Request resolved: https://github.com/pytorch/benchmark/pull/2324\n\nModify existing `jagged_sum` operator benchmark to optionally accept any of the following parameters: `B` (dimension 0 of nested tensor), `M` (dimension 2 of nested tensor), `seqlen` (maximum sequence length on ragged dimension), or `sparsity` (average sparsity on ragged dimension). This diff fixes the provided command line parameters and varies all other parameters above, enabling testing of all combinations of multiple parameters in parallel.\n\nThe following errors persist with sufficiently large inputs:\n- `RuntimeError: numel needs to be smaller than int32_t max; otherwise, please use packed_accessor64` (when running command `buck2 run mode/{opt,inplace} //pytorch/benchmark:triton -- --op jagged_sum --B 1024 --M 1024 --sparsity 0.3`)\n- `torch.OutOfMemoryError: CUDA out of memory.`\n\nReviewed By: davidberard98\n\nDifferential Revision: D58772201\n\nfbshipit-source-id: 365bc1735652736a562d350c7938c1a565a630fe","shortMessageHtmlLink":"Add variable seqlen and sparsity parameters to jagged_sum benchmark (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2365440280\" data-permission-text=\"Title is private\" data-url=\"https://github.com/pytorch/benchmark/issues/2324\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/pytorch/benchmark/pull/2324/hovercard\" href=\"https://github.com/pytorch/benchmark/pull/2324\">#…</a>"}},{"before":"76489bd16396937254a74d1f5712a2a2a7f6ac7c","after":"967e7468aef53cb8d2ad75009bbcd694c89a913f","ref":"refs/heads/xz9/fix-ci","pushedAt":"2024-06-21T03:39:03.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"xuzhao9","name":"Xu Zhao","path":"/xuzhao9","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/502017?s=80&v=4"},"commit":{"message":"Fix sac","shortMessageHtmlLink":"Fix sac"}},{"before":null,"after":"76489bd16396937254a74d1f5712a2a2a7f6ac7c","ref":"refs/heads/xz9/fix-ci","pushedAt":"2024-06-21T01:17:32.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"xuzhao9","name":"Xu Zhao","path":"/xuzhao9","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/502017?s=80&v=4"},"commit":{"message":"Fix the CI","shortMessageHtmlLink":"Fix the CI"}},{"before":"1918fa02c54a883f88f84ac9e33efc29fb89c376","after":null,"ref":"refs/heads/xz9/fix-numpy","pushedAt":"2024-06-21T00:13:13.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"xuzhao9","name":"Xu Zhao","path":"/xuzhao9","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/502017?s=80&v=4"}},{"before":"a529b5aa3822df617104b0d84068cfc619a88af0","after":"53faa0a2c87e77e6545722ce8cf4b2cba569c3a7","ref":"refs/heads/main","pushedAt":"2024-06-20T23:31:01.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Add package contraints to torchbench (#2318)\n\nSummary:\nThis PR introduces package constraints because we want to guarantee that certain packages are the same version before and after the installation. This is important for the CI in pytorch/pytorch because overriding critical package versions will break the CI due to ABI incompatibility.\n\nIf we find a certain model breaks the constraints in the future, we could ask the model installer to apply the `constraint.txt` file with their requirements.txt file.\n\nPyTorch CI Test: https://github.com/pytorch/pytorch/pull/129114\n\nPull Request resolved: https://github.com/pytorch/benchmark/pull/2318\n\nReviewed By: huydhn\n\nDifferential Revision: D58801449\n\nPulled By: xuzhao9\n\nfbshipit-source-id: 13e9c258f04fecd2b14e5a236c4c07ce54873d9b","shortMessageHtmlLink":"Add package contraints to torchbench (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2363004024\" data-permission-text=\"Title is private\" data-url=\"https://github.com/pytorch/benchmark/issues/2318\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/pytorch/benchmark/pull/2318/hovercard\" href=\"https://github.com/pytorch/benchmark/pull/2318\">#2318</a>)"}},{"before":"a3af77282f8be6e7271afa4ce9adbd8065f3627c","after":"a529b5aa3822df617104b0d84068cfc619a88af0","ref":"refs/heads/main","pushedAt":"2024-06-20T23:17:25.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Add unit tests on CPU for TritonBench features (#2323)\n\nSummary:\nPull Request resolved: https://github.com/pytorch/benchmark/pull/2323\n\nAdd unit tests that run on the CPU to verify the behavior of the following:\n- `x_only = True` for metric registration in [`register_metric()`](https://www.internalfb.com/code/fbsource/[731f07681fbbb38750aee3b165137e39fa6cee50]/fbcode/pytorch/benchmark/torchbenchmark/util/triton_op.py?lines=337)\n- custom `label` argument for benchmark registration in [`register_benchmark()`](https://www.internalfb.com/code/fbsource/[731f07681fbbb38750aee3b165137e39fa6cee50]/fbcode/pytorch/benchmark/torchbenchmark/util/triton_op.py?lines=316)\n\nReviewed By: xuzhao9\n\nDifferential Revision: D58558868\n\nfbshipit-source-id: 50e35ec3359db02ce86f45b1d44c34dac9c1a03b","shortMessageHtmlLink":"Add unit tests on CPU for TritonBench features (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2365313306\" data-permission-text=\"Title is private\" data-url=\"https://github.com/pytorch/benchmark/issues/2323\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/pytorch/benchmark/pull/2323/hovercard\" href=\"https://github.com/pytorch/benchmark/pull/2323\">#2323</a>)"}},{"before":"caa76d82ea83ebefea8b1fd022da589d927434e4","after":"a3af77282f8be6e7271afa4ce9adbd8065f3627c","ref":"refs/heads/main","pushedAt":"2024-06-20T22:52:44.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Add simple fused Triton kernel for jagged_sum operator (#2322)\n\nSummary:\nPull Request resolved: https://github.com/pytorch/benchmark/pull/2322\n\nAdd Triton kernel implementation to `jagged_sum` operator in TritonBench. This Triton kernel performs a sum along the ragged dimension of a nested tensor of logical dimensions `(B, *, M)`, where `*` is the ragged dimension.  It loads in blocks of the `values` tensor along its last dimension `M`, reduces each block of variable length along its first dimension `*`, and stores each of `B` reductions in an output tensor of shape `(B, M)`.\n\nThis Triton kernel is benchmarked against two PyTorch implementations, one which does not pad blocks of variable length and one which does pad.\n\nReviewed By: davidberard98\n\nDifferential Revision: D58549297\n\nfbshipit-source-id: 247f81ed65e010de490114963aac3326e91acb8c","shortMessageHtmlLink":"Add simple fused Triton kernel for jagged_sum operator (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2365301415\" data-permission-text=\"Title is private\" data-url=\"https://github.com/pytorch/benchmark/issues/2322\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/pytorch/benchmark/pull/2322/hovercard\" href=\"https://github.com/pytorch/benchmark/pull/2322\">#2322</a>)"}},{"before":"4c2721b15567a325416e448b786fe10394f11eb4","after":"1918fa02c54a883f88f84ac9e33efc29fb89c376","ref":"refs/heads/xz9/fix-numpy","pushedAt":"2024-06-20T22:21:33.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"xuzhao9","name":"Xu Zhao","path":"/xuzhao9","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/502017?s=80&v=4"},"commit":{"message":"Change the model","shortMessageHtmlLink":"Change the model"}},{"before":"ec6cd8abb4c195e534467a329ad86b96ef7c90b9","after":"4c2721b15567a325416e448b786fe10394f11eb4","ref":"refs/heads/xz9/fix-numpy","pushedAt":"2024-06-20T22:20:09.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"xuzhao9","name":"Xu Zhao","path":"/xuzhao9","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/502017?s=80&v=4"},"commit":{"message":"Update message","shortMessageHtmlLink":"Update message"}},{"before":"14e33ff7e79592bd254b7bb58941df4627e36fd7","after":"ec6cd8abb4c195e534467a329ad86b96ef7c90b9","ref":"refs/heads/xz9/fix-numpy","pushedAt":"2024-06-20T20:22:49.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"xuzhao9","name":"Xu Zhao","path":"/xuzhao9","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/502017?s=80&v=4"},"commit":{"message":"Rename sac utils","shortMessageHtmlLink":"Rename sac utils"}},{"before":"46df033156540ed0d80b8a4c8f3eb164a7de60a7","after":"14e33ff7e79592bd254b7bb58941df4627e36fd7","ref":"refs/heads/xz9/fix-numpy","pushedAt":"2024-06-20T19:20:09.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"xuzhao9","name":"Xu Zhao","path":"/xuzhao9","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/502017?s=80&v=4"},"commit":{"message":"Append extra args to pip_install_requirements","shortMessageHtmlLink":"Append extra args to pip_install_requirements"}},{"before":"ebbbd505c876b28ff2f7d218136f14e73ef3c492","after":null,"ref":"refs/heads/xz9/fix-yolov3","pushedAt":"2024-06-20T18:16:32.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"xuzhao9","name":"Xu Zhao","path":"/xuzhao9","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/502017?s=80&v=4"}},{"before":"525491003b484bfca33892052acf5733df0ce3d2","after":"caa76d82ea83ebefea8b1fd022da589d927434e4","ref":"refs/heads/main","pushedAt":"2024-06-20T18:14:59.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Fix yolov3 train accuracy test (#2321)\n\nSummary:\nSetup the correct example inputs for the yolov3 train accuracy test\nFixes https://github.com/pytorch/benchmark/issues/2248\n\nPull Request resolved: https://github.com/pytorch/benchmark/pull/2321\n\nReviewed By: aaronenyeshi\n\nDifferential Revision: D58823036\n\nPulled By: xuzhao9\n\nfbshipit-source-id: d63c069bdf0da6ba496f274510e64d76e3c10f76","shortMessageHtmlLink":"Fix yolov3 train accuracy test (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2363462998\" data-permission-text=\"Title is private\" data-url=\"https://github.com/pytorch/benchmark/issues/2321\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/pytorch/benchmark/pull/2321/hovercard\" href=\"https://github.com/pytorch/benchmark/pull/2321\">#2321</a>)"}},{"before":"794d12bd5c43e230b36554d8d484757ed7942583","after":null,"ref":"refs/heads/xz9/enable-detectron2","pushedAt":"2024-06-20T18:12:52.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"xuzhao9","name":"Xu Zhao","path":"/xuzhao9","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/502017?s=80&v=4"}},{"before":"a6390beee82876080d761ce415ed19bc7deb2511","after":null,"ref":"refs/heads/xz9/test-bench","pushedAt":"2024-06-20T18:12:43.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"xuzhao9","name":"Xu Zhao","path":"/xuzhao9","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/502017?s=80&v=4"}},{"before":"8c8fc4b5815520a3a678a45255b40b53ec8faca5","after":null,"ref":"refs/heads/xz9/add-skip-install","pushedAt":"2024-06-20T18:12:40.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"xuzhao9","name":"Xu Zhao","path":"/xuzhao9","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/502017?s=80&v=4"}},{"before":"8451c1f8fae21d78d21f5f02595fe15ce7c28fb0","after":"525491003b484bfca33892052acf5733df0ce3d2","ref":"refs/heads/main","pushedAt":"2024-06-20T18:01:38.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Add --skip to install.py (#2314)\n\nSummary:\nUse `--skip` to skip the install of certain models.\n\nFixes https://github.com/pytorch/benchmark/issues/2308\n\nPull Request resolved: https://github.com/pytorch/benchmark/pull/2314\n\nTest Plan:\n```\n$ python install.py hf_Bert hf_Bart yolov3 --skip hf_Bert yolov3\nchecking packages torch, torchvision, torchaudio are installed...OK\nrunning setup for /Users/xzhao9/git/benchmark/torchbenchmark/models/hf_Bart...OK\n```\n\nReviewed By: eellison\n\nDifferential Revision: D58767785\n\nPulled By: xuzhao9\n\nfbshipit-source-id: 5fcf5d539dda322e86bc6c2ae86b37e288ec71ad","shortMessageHtmlLink":"Add --skip to install.py (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2361034303\" data-permission-text=\"Title is private\" data-url=\"https://github.com/pytorch/benchmark/issues/2314\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/pytorch/benchmark/pull/2314/hovercard\" href=\"https://github.com/pytorch/benchmark/pull/2314\">#2314</a>)"}},{"before":"8de4ad38d6457c2f55d25bda4311833112706063","after":"8451c1f8fae21d78d21f5f02595fe15ce7c28fb0","ref":"refs/heads/main","pushedAt":"2024-06-20T17:52:44.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Fix test_bench script (#2320)\n\nSummary:\nFixes https://github.com/pytorch/benchmark/issues/2315\n\nPull Request resolved: https://github.com/pytorch/benchmark/pull/2320\n\nTest Plan:\n```\n$ pytest test_bench.py -k \"test_eval[BERT_pytorch-cpu]\" --ignore_machine_config\n========================================================================================== test session starts ===========================================================================================\nplatform linux -- Python 3.11.5, pytest-7.4.3, pluggy-1.0.0\nbenchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)\nrootdir: /home/xz/git/benchmark\nplugins: benchmark-4.0.0, hypothesis-6.98.15\ncollected 411 items / 410 deselected / 1 selected\n\ntest_bench.py .                                                                                                                                                                                    [100%]\n\n------------------------------------------------- benchmark 'hub': 1 tests -------------------------------------------------\nName (time in ms)                    Min       Max      Mean  StdDev    Median     IQR  Outliers     OPS  Rounds  Iterations\n----------------------------------------------------------------------------------------------------------------------------\ntest_eval[BERT_pytorch-cpu]     114.2104  117.3853  115.4276  1.0485  115.3054  1.4325       4;0  8.6634       9           1\n----------------------------------------------------------------------------------------------------------------------------\n\nLegend:\n  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.\n  OPS: Operations Per Second, computed as 1 / Mean\n=================================================================================== 1 passed, 410 deselected in 5.68s ====================================================================================\n```\n\nReviewed By: aaronenyeshi\n\nDifferential Revision: D58823072\n\nPulled By: xuzhao9\n\nfbshipit-source-id: 172be1d922b2a51ec2df08b822102dc0a20818ac","shortMessageHtmlLink":"Fix test_bench script (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2363420265\" data-permission-text=\"Title is private\" data-url=\"https://github.com/pytorch/benchmark/issues/2320\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/pytorch/benchmark/pull/2320/hovercard\" href=\"https://github.com/pytorch/benchmark/pull/2320\">#2320</a>)"}},{"before":"3bd542d5ac744f705e805c2af25ce6f6d42b1b56","after":"8de4ad38d6457c2f55d25bda4311833112706063","ref":"refs/heads/main","pushedAt":"2024-06-20T15:08:22.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"torch_multimodal_clip model get failed due to network issue (#2289)\n\nSummary:\nDescription:\nwhile running model torch_multimodal_clip getting this issue\nFileNotFoundError: [Errno 2] No such file or directory: 'torchbenchmark/models/torch_multimodal_clip/.data/pizza.jpg'\nmain reason is while downloading the pizza.jpg , getting Failed to download file with status code 403 issue so this change will resolve this issue\n\nPull Request resolved: https://github.com/pytorch/benchmark/pull/2289\n\nReviewed By: aaronenyeshi\n\nDifferential Revision: D58813450\n\nPulled By: xuzhao9\n\nfbshipit-source-id: aa4cbee8fa4633182898fe7c1037b471505ce2ec","shortMessageHtmlLink":"torch_multimodal_clip model get failed due to network issue (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2343165006\" data-permission-text=\"Title is private\" data-url=\"https://github.com/pytorch/benchmark/issues/2289\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/pytorch/benchmark/pull/2289/hovercard\" href=\"https://github.com/pytorch/benchmark/pull/2289\">#2289</a>)"}},{"before":"b2b4158d3278d39e5575143e1d6954a9dc01e396","after":"3bd542d5ac744f705e805c2af25ce6f6d42b1b56","ref":"refs/heads/main","pushedAt":"2024-06-20T14:40:41.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Re-enable detectron2 model accuracy checks. (#2317)\n\nSummary:\nDetectron2 models can pass the accuracy check when the deterministic mode is disabled.\n\nRevert https://github.com/pytorch/benchmark/commit/abed1e7d6033ea9b75e4fc821d6c74d900fe9cc5.\nFixes https://github.com/pytorch/benchmark/issues/2290\n\nPull Request resolved: https://github.com/pytorch/benchmark/pull/2317\n\nReviewed By: aaronenyeshi\n\nDifferential Revision: D58794036\n\nPulled By: xuzhao9\n\nfbshipit-source-id: b0c38bd63c63b71e33f9692ad2169f8267e93ab1","shortMessageHtmlLink":"Re-enable detectron2 model accuracy checks. (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2362910789\" data-permission-text=\"Title is private\" data-url=\"https://github.com/pytorch/benchmark/issues/2317\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/pytorch/benchmark/pull/2317/hovercard\" href=\"https://github.com/pytorch/benchmark/pull/2317\">#2317</a>)"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEbAS1qwA","startCursor":null,"endCursor":null}},"title":"Activity · pytorch/benchmark"}