Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how to show the heatmap on Figure 10 in the paper? #1

Open
DavideHe opened this issue Mar 13, 2024 · 0 comments
Open

how to show the heatmap on Figure 10 in the paper? #1

DavideHe opened this issue Mar 13, 2024 · 0 comments
Assignees

Comments

@DavideHe
Copy link

I see the heatmap in this paper Figure 10 and RMT Figure 6
I have some question:

  1. many LLMs are casual. attention heatmap is bidirectional attention in RMT's Figure 6. But bidirectional attention generally is symmetric matrix . In the RMT's Figure 6, it show the casual feature.

  2. what information you use to show that heatmap ? how to use these information . If these are some code or implementation principle,thank you very much for providing these .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants