Package io.github.kirstenali.deepj.layers.transformer.attention

package io.github.kirstenali.deepj.layers.transformer.attention

Related Packages

Package	Description
io.github.kirstenali.deepj.layers.transformer
io.github.kirstenali.deepj.layers.transformer.blocks
io.github.kirstenali.deepj.layers.transformer.norm

Classes

Class	Description
MultiHeadLatentAttention	Compact Multi-Head Latent Attention (MLA) inspired by DeepSeek-V2/V3.
MultiHeadSelfAttention	Multi-head causal self-attention for a single sequence (no batch dimension).
RoPEMultiHeadSelfAttention	Multi-head self-attention with Rotary Positional Embedding (RoPE).