You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
what‘s the self.if_bidirectional,I konw that we define bimaba_type=v1/v2 in mamba_simple.py to present we use bidirectional ssm. So what's the meaning of self.if_bidirectional in models_mamba.py,I don't know it,please help me.
thanks !
if not self.if_bidirectional:
..............
else:
# get two layers in a single for-loop
for i in range(len(self.layers) // 2): #
if self.if_rope:
hidden_states = self.rope(hidden_states)
if residual is not None and self.if_rope_residual:
residual = self.rope(residual)
hidden_states_f, residual_f = self.layers[i * 2](
hidden_states, residual, inference_params=inference_params
)
hidden_states_b, residual_b = self.layers[i * 2 + 1](
hidden_states.flip([1]), None if residual == None else residual.flip([1]), inference_params=inference_params
)
hidden_states = hidden_states_f + hidden_states_b.flip([1])
residual = residual_f + residual_b.flip([1])
The text was updated successfully, but these errors were encountered:
what‘s the self.if_bidirectional,I konw that we define bimaba_type=v1/v2 in mamba_simple.py to present we use bidirectional ssm. So what's the meaning of self.if_bidirectional in models_mamba.py,I don't know it,please help me.
thanks !
The text was updated successfully, but these errors were encountered: