From 7b6be9a99af30766bab2ec620e8db2538f3c59a4 Mon Sep 17 00:00:00 2001 From: Range King Date: Fri, 21 Apr 2023 14:28:29 +0800 Subject: [PATCH] Fix `pip install` path in `mmlu.ipynb` (#121) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit # Thank you for contributing an eval! ♥️ 🚨 Please make sure your PR follows these guidelines, __failure to follow the guidelines below will result in the PR being closed automatically__. Note that even if the criteria are met, that does not guarantee the PR will be merged nor GPT-4 access granted. 🚨 __PLEASE READ THIS__: In order for a PR to be merged, it must fail on GPT-4. We are aware that right now, users do not have access, so you will not be able to tell if the eval fails or not. Please run your eval with GPT-3.5-Turbo, but keep in mind as we run the eval, if GPT-4 gets higher than 90% on the eval, we will likely reject since GPT-4 is already capable of completing the task. We plan to roll out a way for users submitting evals to see the eval performance on GPT-4 soon. Stay tuned! Until then, you will not be able to see the eval performance on GPT-4. We encourage partial PR's with ~5-10 example that we can then run the evals on and share the results with you so you know how your eval does with GPT-4 before writing all 100 examples. ## Eval details 📑 The current pip command in `mmlu.ipynb` will raise an error as blow. ```shell ERROR: file:///home/ubuntu/repo/evals/examples does not appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' found. ``` This PR modifies the command to be consistent with `lafand-mt.ipynb` --- examples/mmlu.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/mmlu.ipynb b/examples/mmlu.ipynb index 40e791e370..285094eeb3 100644 --- a/examples/mmlu.ipynb +++ b/examples/mmlu.ipynb @@ -21,7 +21,7 @@ "outputs": [], "source": [ "# Install, and download MMLU if you haven't already\n", - "%pip install -e .\n", + "%pip install -e ../.\n", "\n", "!curl -O https://people.eecs.berkeley.edu/~hendrycks/data.tar\n", "!tar -xf data.tar\n",