Sourcery Starbot ⭐ refactored mkirch/evals #2

sourcery-ai-bot · 2023-12-24T09:07:28Z

Thanks for starring sourcery-ai/sourcery ✨ 🌟 ✨

Here's your pull request refactoring your most popular Python repo.

If you want Sourcery to refactor all your Python repos and incoming pull requests install our bot.

Review changes via command line

To manually merge these changes, make sure you're on the main branch, then run:

git fetch https://github.com/sourcery-ai-bot/evals main
git merge --ff-only FETCH_HEAD
git reset HEAD^

sourcery-ai-bot · 2023-12-24T09:07:29Z

evals/api.py

- assert not kwargs.get("logprobs") is None
+ assert kwargs.get("logprobs") is not None


Function sample_freeform refactored with the following changes:

Simplify logical expression using De Morgan identities (de-morgan)

sourcery-ai-bot · 2023-12-24T09:07:29Z

evals/base.py

- raise ValueError(f"Must specify a model")
+ raise ValueError("Must specify a model")


Function ModelSpec.__post_init__ refactored with the following changes:

Replace f-string with no interpolated values with string (remove-redundant-fstring)

sourcery-ai-bot · 2023-12-24T09:07:29Z

evals/data.py

- if scheme == "" or scheme == "file":
+ if scheme in ["", "file"]:


Function open_by_file_pattern refactored with the following changes:

Replace multiple comparisons of same variable with in operator (merge-comparisons)

sourcery-ai-bot · 2023-12-24T09:07:29Z

evals/data.py

- md5 = hashlib.md5((name + ":" + str((args, kwargs))).encode("utf-8")).hexdigest()
+ md5 = hashlib.md5(f"{name}:{(args, kwargs)}".encode("utf-8")).hexdigest()


Function filecache refactored with the following changes:

Use f-string instead of string concatenation [×2] (use-fstring-for-concatenation)

Remove unnecessary calls to str() from formatted values in f-strings (remove-str-from-fstring)

sourcery-ai-bot · 2023-12-24T09:07:30Z

evals/data.py

- return [row for row in reader]
+ return list(reader)


Function get_csv refactored with the following changes:

Replace identity comprehension with call to collection constructor (identity-comprehension)

sourcery-ai-bot · 2023-12-24T09:07:31Z

evals/registry.py

- if not name in self._evals:
+ if name not in self._evals:


Function Registry.get_base_eval refactored with the following changes:

Simplify logical expression using De Morgan identities (de-morgan)

sourcery-ai-bot · 2023-12-24T09:07:31Z

evals/cli/oaieval.py

- # first, look for a prefix match
- for model_prefix, n_ctx in DICT_OF_N_CTX_BY_MODEL_NAME_PREFIX.items():
- if model_name.startswith(model_prefix):
- return n_ctx
- # otherwise, look for an exact match and return None if not found
- return DICT_OF_N_CTX_BY_MODEL_NAME.get(model_name, None)
+ return next(
+ (
+ n_ctx
+ for model_prefix, n_ctx in DICT_OF_N_CTX_BY_MODEL_NAME_PREFIX.items()
+ if model_name.startswith(model_prefix)
+ ),
+ DICT_OF_N_CTX_BY_MODEL_NAME.get(model_name, None),
+ )


Function n_ctx_from_model_name refactored with the following changes:

Use the built-in function next instead of a for-loop (use-next)

This removes the following comments ( why? ):

# first, look for a prefix match # otherwise, look for an exact match and return None if not found

sourcery-ai-bot · 2023-12-24T09:07:31Z

evals/cli/oaieval.py

- result = ModelSpec(name=name, model=name, is_chat=(name in self.CHAT_MODELS))
- return result
-
+ return ModelSpec(name=name, model=name, is_chat=(name in self.CHAT_MODELS))
 if name in self.api_model_ids:
- result = ModelSpec(
+ return ModelSpec(
 name=name,
 model=name,
 is_chat=(name in self.CHAT_MODELS),
 n_ctx=n_ctx_from_model_name(name),
 )
- return result
-


Function ModelResolver.resolve refactored with the following changes:

Inline variable that is immediately returned [×2] (inline-immediately-returned-variable)

sourcery-ai-bot · 2023-12-24T09:07:31Z

evals/cli/oaievalset.py

- print(" " + command_str)
+ print(f" {command_str}")


Function main refactored with the following changes:

Use f-string instead of string concatenation (use-fstring-for-concatenation)

sourcery-ai-bot · 2023-12-24T09:07:31Z

evals/elsuite/utils.py

- if idx == -1:
- return None
- return text[idx + len(answer_prompt) :]
+ return None if idx == -1 else text[idx + len(answer_prompt) :]


Function get_answer refactored with the following changes:

Lift code into else after jump in control flow (reintroduce-else)

Replace if statement with if expression (assign-if-exp)

sourcery-ai-bot · 2023-12-24T09:07:33Z

evals/elsuite/utils.py

- if s1 == "" or s2 == "":
- return s1 == s2
-
- return s1 in s2 or s2 in s1
+ return s1 == s2 if s1 == "" or s2 == "" else s1 in s2 or s2 in s1


Function fuzzy_match refactored with the following changes:

Lift code into else after jump in control flow (reintroduce-else)

Replace if statement with if expression (assign-if-exp)

sourcery-ai-bot · 2023-12-24T09:07:33Z

evals/elsuite/utils.py

- return {k: v for k, v in dict(matches).items()}
+ return dict(dict(matches))


Function get_yesno_from_text refactored with the following changes:

Replace identity comprehension with call to collection constructor (identity-comprehension)

sourcery-ai-bot · 2023-12-24T09:07:33Z

evals/elsuite/utils.py

- char = max(last_y, last_n)[1]
- return char
+ return max(last_y, last_n)[1]


Function get_letter_from_data refactored with the following changes:

Inline variable that is immediately returned (inline-immediately-returned-variable)

sourcery-ai-bot · 2023-12-24T09:07:33Z

evals/elsuite/utils.py

- return max([_f1_score(prediction, answer) for answer in answers])
+ return max(_f1_score(prediction, answer) for answer in answers)


Function f1_score refactored with the following changes:

Replace unneeded comprehension with generator (comprehension-to-generator)

sourcery-ai-bot · 2023-12-24T09:07:33Z

evals/elsuite/basic/includes.py

- [evals.elsuite.utils.get_answer(sampled, ref) for ref in sample["ideal"]]
+ evals.elsuite.utils.get_answer(sampled, ref) for ref in sample["ideal"]


Function Includes.eval_sample refactored with the following changes:

Replace unneeded comprehension with generator (comprehension-to-generator)

sourcery-ai-bot · 2023-12-24T09:07:34Z

evals/elsuite/modelgraded/classify.py

- missing_samples = len(all_sample_metrics) - len(chosen)
- if missing_samples:
+ if missing_samples := len(all_sample_metrics) - len(chosen):
 counts["__missing_samples__"] = missing_samples
 record_metrics.update({f"counts/{metric}/{k}": v for k, v in counts.items()})
 if self.metaeval:
- metascores = [m[metric + "_metascore"] for m in all_sample_metrics if metric in m]
+ metascores = [
+ m[f"{metric}_metascore"]
+ for m in all_sample_metrics
+ if metric in m
+ ]


Function ModelBasedClassify.run refactored with the following changes:

Use named expression to simplify assignment and conditional (use-named-expression)

Use f-string instead of string concatenation (use-fstring-for-concatenation)

sourcery-ai-bot · 2023-12-24T09:07:34Z

evals/prompt/base.py

- prefix = chat_to_prefixes.get(role, role.capitalize() + ": ")
+ prefix = chat_to_prefixes.get(role, f"{role.capitalize()}: ")


Function chat_prompt_to_text_prompt refactored with the following changes:

Use f-string instead of string concatenation (use-fstring-for-concatenation)

sourcery-ai-bot · 2023-12-24T09:07:34Z

evals/utils/snowflake.py

- for arg in args:
- if arg is not None:
- return arg
- return None
+ return next((arg for arg in args if arg is not None), None)


Function _first_not_none refactored with the following changes:

Use the built-in function next instead of a for-loop (use-next)

sourcery-ai-bot · 2023-12-24T09:07:34Z

evals/utils/snowflake.py

- cs = self.ctx.cursor(*args, **kwargs)
- return cs
+ return self.ctx.cursor(*args, **kwargs)


Function SnowflakeConnection.cursor refactored with the following changes:

Inline variable that is immediately returned (inline-immediately-returned-variable)

sourcery-ai-bot · 2023-12-24T09:07:34Z

scripts/pattern_identification_generator.py

- num_symbols = int(len(SYMBOLS) / 2)
+ num_symbols = len(SYMBOLS) // 2


Function generate_example refactored with the following changes:

Simplify division expressions (simplify-division)

'Refactored by Sourcery'

f1a14de

sourcery-ai-bot commented Dec 24, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sourcery Starbot ⭐ refactored mkirch/evals #2

Sourcery Starbot ⭐ refactored mkirch/evals #2

sourcery-ai-bot commented Dec 24, 2023

sourcery-ai-bot Dec 24, 2023

sourcery-ai-bot Dec 24, 2023

sourcery-ai-bot Dec 24, 2023

sourcery-ai-bot Dec 24, 2023

sourcery-ai-bot Dec 24, 2023

sourcery-ai-bot Dec 24, 2023

sourcery-ai-bot Dec 24, 2023

sourcery-ai-bot Dec 24, 2023

sourcery-ai-bot Dec 24, 2023

sourcery-ai-bot Dec 24, 2023

sourcery-ai-bot Dec 24, 2023

sourcery-ai-bot Dec 24, 2023

sourcery-ai-bot Dec 24, 2023

sourcery-ai-bot Dec 24, 2023

sourcery-ai-bot Dec 24, 2023

sourcery-ai-bot Dec 24, 2023

sourcery-ai-bot Dec 24, 2023

sourcery-ai-bot Dec 24, 2023

sourcery-ai-bot Dec 24, 2023

sourcery-ai-bot Dec 24, 2023

		assert not kwargs.get("logprobs") is None
		assert kwargs.get("logprobs") is not None

		raise ValueError(f"Must specify a model")
		raise ValueError("Must specify a model")

		if scheme == "" or scheme == "file":
		if scheme in ["", "file"]:

		md5 = hashlib.md5((name + ":" + str((args, kwargs))).encode("utf-8")).hexdigest()
		md5 = hashlib.md5(f"{name}:{(args, kwargs)}".encode("utf-8")).hexdigest()

		return {k: v for k, v in dict(matches).items()}
		return dict(dict(matches))

		return max([_f1_score(prediction, answer) for answer in answers])
		return max(_f1_score(prediction, answer) for answer in answers)

		[evals.elsuite.utils.get_answer(sampled, ref) for ref in sample["ideal"]]
		evals.elsuite.utils.get_answer(sampled, ref) for ref in sample["ideal"]

		prefix = chat_to_prefixes.get(role, role.capitalize() + ": ")
		prefix = chat_to_prefixes.get(role, f"{role.capitalize()}: ")

		num_symbols = int(len(SYMBOLS) / 2)
		num_symbols = len(SYMBOLS) // 2

Sourcery Starbot ⭐ refactored mkirch/evals #2

Are you sure you want to change the base?

Sourcery Starbot ⭐ refactored mkirch/evals #2

Conversation

sourcery-ai-bot commented Dec 24, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment