Commits

Commits on Jun 2, 2024

make sure linear attention can be turned off
lucidrains
committedJun 2, 2024

Commits on Jan 17, 2024

cleanup
lucidrains
committedJan 17, 2024
start using einx.get_at
lucidrains
committedJan 17, 2024

Commits on Jan 14, 2024

allow for linear attention to attend to nothing
lucidrains
committedJan 14, 2024
Merge pull request #12 from lucidrains/fix-tests
lucidrains
committedJan 14, 2024
try again
lucidrains
committedJan 14, 2024
try to fix tests
lucidrains
committedJan 14, 2024

Commits on Jan 11, 2024

upgrade to a better linear attention
lucidrains
committedJan 11, 2024

Commits on Oct 27, 2023

use another flavor of linear attention
lucidrains
committedOct 27, 2023

Commits on Oct 7, 2023

more plumbing
lucidrains
committedOct 7, 2023
readme
lucidrains
committedOct 7, 2023
fix pypi
lucidrains
committedOct 7, 2023
add gating to both feature as well as coordinate updates
lucidrains
committedOct 7, 2023

Commits on Aug 25, 2023

fix readme
lucidrains
committedAug 25, 2023

Commits on Aug 9, 2023

used by a new paper!
lucidrains
committedAug 9, 2023

Commits on Aug 1, 2023

allow the network to tailor the magnitudes of the cross product vectors, remove tanh from the gates that control the magnitude and small init
lucidrains
committedAug 1, 2023

Commits on Jul 31, 2023

stop supporting 3.7
lucidrains
committedJul 31, 2023
start adopting l2 distance attention
lucidrains
committedJul 31, 2023

Commits on Mar 17, 2023

add global linear attention
lucidrains
committedMar 17, 2023

Commits on Jan 21, 2023

test citation feature
lucidrains
committedJan 21, 2023

Commits on Jan 4, 2023

tweak ff
lucidrains
committedJan 4, 2023

Commits on Dec 30, 2022

Commits on Nov 24, 2022

switch to cosine sim attention, seems to be more stable
lucidrains
committedNov 24, 2022

Commits on Mar 4, 2022

numerical stability
lucidrains
committedMar 4, 2022
make sure dropout is available
lucidrains
committedMar 4, 2022

Commits on Jan 5, 2022

readme
lucidrains
committedJan 5, 2022

Commits on Nov 27, 2021

0.5.1
lucidrains
committedNov 27, 2021
correct checkpointing
lucidrains
committedNov 27, 2021

Commits on Aug 27, 2021

make theta in rotary sinusoidal embedding tuneable, and also allow for relative distance scale and max cutoff to be configurable
lucidrains
committedAug 27, 2021

Commits on Aug 25, 2021

add checkpointing, to allow for greatly increased number of neighbors attended to
lucidrains
committedAug 25, 2021
readme and cite
lucidrains
committedAug 25, 2021
remove global linear attention, but add talking heads
lucidrains
committedAug 25, 2021

Commits on Jun 10, 2021

cleanup
lucidrains
committedJun 10, 2021