Package io.github.kirstenali.deepj.transformer.embeddings

package io.github.kirstenali.deepj.transformer.embeddings

Related Packages

Package	Description
io.github.kirstenali.deepj.transformer

Classes

Class	Description
Embedding	Token embedding: ids -> vectors.
PositionalEmbedding	Learnable positional embeddings added to token embeddings.
RotaryEmbedding	Rotary Positional Embedding (RoPE) — used in Llama, Mistral, Qwen, DeepSeek, and GPT-NeoX.