Editing the Mind of Giants: An In-Depth Exploration of Pitfalls of Knowledge Editing in Large Language Models

In the single edit scenario, we further classify the methods into two settings:

• •

  One-Hop: This setting focuses on evaluating the impact of a single edit on direct, one-hop reasoning tasks.

• •

  Multi-Hop: This setting assesses the impact of a single edit on more complex, multi-hop reasoning tasks.

For one-hop evaluations, we adopt the methods proposed by (Yao et al., 2023a). These include:

• •

  Subject Replace: This metric tests the model’s generalization ability by replacing the subject in the question with an alias or synonym, assessing if the edited attribute is generalized to other descriptions of the same subject.

• •

  Reversed Relation: This metric evaluates the model’s capability to handle reversed relations by filtering for suitable relations such as one-to-one and asking the reverse question to check if the target entity is also updated.

• •

  One-Hop Test: This metric assesses the edited language model’s performance on downstream tasks that require one-hop reasoning.

In the multi-hop setting, we assess the model’s performance on multi-hop questions using the evaluation methods proposed by (Zhong et al., 2023), which include:

• •

  Edit-wise Success Rate (EW): This metric measures how many facts can be successfully recalled from the edited language model.

  $$\mathrm{EW}=\mathbbm{1}\{f^{*}(s)=o^{*}\}$$

  (5)

  where $f^{*}$ is the model after editing.

• •

  Instance-wise Accuracy (IW): This metric tests how many multi-hop instances the model can recall all the individual single-hop facts. This metric is crucial for multi-hop performance, as the model must encode each fact to answer the multi-hop question.

  $$\mathrm{IW}=\mathbbm{1}\{\bigwedge_{(s,r,o^{*})\in C^{*}}[f^{*}(s)=o^{*}]\}$$

  (6)

  where $C^{*}=\langle(s_{1},r_{1},o_{1}),\dots,(s_{n},r_{n},o_{n})\rangle$ is the chain of facts of a multi-hop question. In this chain, the object of the $i^{\mathrm{th}}$ fact is the subject of the next fact. (i.e., $o_{i}=s_{i+1}$)

• •

  Multi-hop Accuracy (MH): This metric assesses the accuracy of the original and edited language models on multi-hop questions. In the MQuAKE dataset (Zhong et al., 2023), there are three generated multi-hop questions for each instance. If any of the three questions is correctly answered by the model, we consider it accurate.

  $$\mathrm{MH}=\mathbbm{1}\{\bigvee_{q\in Q}f^{*}(q)=a^{*}\}$$

  (7)

  where $Q$ is a set of similar multi-hop questions with the same answer $a^{*}$.

In the multiple edits scenario, we test the model’s performance after applying multiple edits. For this, we use the setting and evaluation methods from (Li et al., 2024b). The settings consist of:

• •

  Reverse Conflict: This setting introduces conflicts by editing facts with reverse relations. For example:
  edit 1: ($s_{1},r_{1},o_{1}$→$o_{2}$)
  Hamlet was written by Shakespeare → Agatha Christie.
  edit 2: ($o_{2},r_{2},s_{1}$→$s_{2}$)
  The notable work of Agatha Christie is Hamlet → Odyssey
  the updated knowledge then could be represented as:

  $$\left\{\begin{array}[]{l}k_{0}=(s_{1},r_{1},o_{2})\\ k_{n}=(s_{2},r_{1},o_{2})\end{array}\right.$$

• •

  Composite Conflict: This explores more complex situations where the edits are associated with a fact that is not influenced by the editing (tied fact). For example:
  edit 1: ($s_{1},r_{1},o_{1}$→$o_{2}$)
  Hamlet was written in English → French
  edit 2: ($s_{2},r_{2},o_{2}$→$o_{3}$)
  Shakespeare wrote in French → German
  tied fact: ($s_{1},r,s_{2}$)
  The notable work of Shakespeare is Hamlet
  where $r\land r_{1}\to r_{2}$ is a logical rule. The updated knowledge then could be represented as:

  $$\left\{\begin{array}[]{l}k_{f}=(s_{1},r,s_{2})\\ k_{0}=(s_{1},r_{1},o_{2})\\ k_{n}=(s_{1},r_{1},o_{3})\end{array}\right.$$

The evaluation methods include:

• •

  Conflict Score (CS): Measures how well a knowledge editing method handles knowledge conflicts by calculating the ratio that the new fact is more probable than the old fact after knowledge editing.

  $$\mathrm{CS=}\mathbbm{1}\{p_{f_{\theta}^{\prime}}(k_{n})>p_{f_{\theta}^{\prime}}(k_{o})\}$$

  (8)

• •

  Conflict Magnitude (CM): Estimates the decrease in probability of the old fact after editing.

  $$\mathrm{CM=}\frac{p_{f_{\theta^{m}}}(k_{o})-p_{f_{\theta^{\prime}}}(k_{o})}{p_{f_{\theta^{m}}}(k_{o})}$$

  (9)

  $\theta^{m}$ is the intermediate model parameters after edit 1.

In the single edit scenario for locality, we adopt the methods proposed by (Yao et al., 2023b), including:

• •

  Other Attribution The modified CounterFact dataset is applied to test whether the non-target attributes of the edited subjects remained the same. For example, if we reset Lionel Messi as a basketball player, his nationality should stay the same.

• •

  Distract Neighbor Pretend model editing by modifying the neighborhood prompt in the CounterFact. For example, if the original prompt is "Windows 11 is a product of __", the modified prompt would be "Windows 11 is a product of Google. Office 365, developed by __". It testifies whether the model prediction would be "distracted" by the revised prompt.

• •

  Other Task The edited model is tested on the multiple-choice QA task Physical Interaction QA(PIQA, Bisk et al. (2020)) and the performance is evaluated by accuracy.

We also test the model’s locality in the multiple edits scenario adopting the methods and evaluations from (Li et al., 2024b). The settings consist of:

• •

  Round Edit: This edits the knowledge triplet back-and-forth, for example:
  edit 1: ($s,r,o_{1}$→$o*$)
  edit 2: ($s,r,o*$→$o_{1}$)

the evaluation metrics include:

• •

  Distortion (D) (Li et al., 2024b):

  $$D=JS\left(p_{f_{\theta}}(\text{Obj}\mid(s,r)),p_{f_{\theta^{\prime}}}(\text{Obj}\mid(s,r))\right)$$

  (10)

  estimates the JS divergence of the objects distribution before and after edit.

• •

  Ignore Rate (IR) (Li et al., 2024b):

  $\displaystyle\begin{split}\mathrm{IR=}\frac{1}{\left|\mathrm{Obj}\right|-1}\sum_{\begin{subarray}{c}o\in\mathrm{Obj}\setminus\{o1\}\end{subarray}}&\mathbbm{1}\{p_{f_{\theta}}(o\mid(s,r))>\\ &p_{f_{\theta}^{\prime}}(o\mid(s,r))\}\end{split}$

  (11)

• •

  Failure Rate (FR) (Li et al., 2024b):

  $$\mathrm{FR=}\mathbbm{1}\{\mathrm{IR}>0.5\}$$

  (12)

• •

  Tied Fact Damage (TDF) (Li et al., 2024b):

  $$\mathrm{TFD=}\frac{p_{f_{\theta^{m}}}(k_{f})-p_{f_{\theta^{\prime}}}(k_{f})}{p_{f_{\theta^{m}}}(k_{f})}$$

  (13)

  $k_{f}$ denotes the tied facts and $\theta^{m}$ is the intermediate model parameters after edit 1.

• •

  Neighborhood KL Divergence (Hoelscher-Obermaier et al., 2023):

  $$\mathrm{NKL}\stackrel{{\scriptstyle\mathrm{def}}}{{=}}\sum_{w\in W}\log\left(\frac{P(w)}{P^{*}(w)}\right)$$

  (14)

• •

  Neighborhood Score (NS) (Meng et al., 2022): collect a set of "neighbor" subjects and evaluation the success fraction for $\mathbb{P}\left[o^{c}\right]>\mathbb{P}\left[o^{*}\right]$, while the $o^{c}$ denotes the correct facts and $o^{*}$ denotes the false facts.

• •

  Neighborhood Magnitude (NM) (Meng et al., 2022): the differences of $\mathbb{P}\left[o^{c}\right]$ and $\mathbb{P}\left[o^{*}\right]$ for the "neighborhood" subjects.