Package io.github.kirstenali.deepj.models.deepseek

Class DeepSeekModel

java.lang.Object

io.github.kirstenali.deepj.models.DecoderOnlyModel

io.github.kirstenali.deepj.models.deepseek.DeepSeekModel

All Implemented Interfaces:

CausalLM, Persistable, Trainable

public final class DeepSeekModel
extends DecoderOnlyModel

DeepSeek-style decoder-only transformer.

Architecture differences from LlamaModel:

• Attention uses a compact MLA-style factorisation: Q and K/V pass through low-rank bottlenecks (qRank / kvRank). An incremental decoder could use the KV latent as a compressed cache; this model currently recomputes the supplied context and does not maintain an inference cache.
• All other components (RMSNorm, RoPE, SwiGLU, token embedding, final norm) are identical to the Llama architecture.

Forward/backward/parameters are provided by DecoderOnlyModel.

Field Summary

Fields inherited from class io.github.kirstenali.deepj.models.DecoderOnlyModel

lmHead, normF, stack, tokEmb

Constructor Summary

Constructors

Constructor	Description
DeepSeekModel(DeepSeekConfig cfg, long seed)

Method Summary

Modifier and Type	Method	Description
DeepSeekConfig	config()
float	gradClipNorm()	Global gradient-clipping threshold (e.g.

Methods inherited from class io.github.kirstenali.deepj.models.DecoderOnlyModel

applyInitScale, backward, backwardEmbeddings, embed, embeddingParameters, forward, parameters

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface io.github.kirstenali.deepj.persistence.Persistable

load, save

Methods inherited from interface io.github.kirstenali.deepj.training.Trainable

zeroGrad

Constructor Details

DeepSeekModel

public DeepSeekModel(DeepSeekConfig cfg,
 long seed)

Method Details

gradClipNorm

public float gradClipNorm()

Description copied from interface: CausalLM

Global gradient-clipping threshold (e.g. 1.0).

config

public DeepSeekConfig config()