Package io.github.kirstenali.deepj.examples

Class TrainSmallDeepSeek

java.lang.Object

io.github.kirstenali.deepj.examples.TrainSmallDeepSeek

public final class TrainSmallDeepSeek
extends Object

Example: tiny DeepSeek training on a small text file using byte-level tokens.

Architecture differences from the Llama example:

• Multi-Head Latent Attention (MLA) compresses Q through a low-rank Q bottleneck (qRank) and K/V through a KV bottleneck (kvRank).
• All other components (RMSNorm, RoPE, SwiGLU, token embedding) are identical to the Llama architecture.

Intended as a smoke test / reference, not for serious training.

Constructor Summary

Constructors

Constructor	Description
TrainSmallDeepSeek()

Method Summary

Modifier and Type	Method	Description
static void	main(String[] args)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

TrainSmallDeepSeek

public TrainSmallDeepSeek()

Method Details

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception