Package io.github.kirstenali.deepj.examples

Class TrainSmallLlama

java.lang.Object

io.github.kirstenali.deepj.examples.TrainSmallLlama

public final class TrainSmallLlama
extends Object

Example: tiny Llama training on a small text file using byte-level tokens.

Architecture differences from the GPT example:

• No learned positional embedding — RoPE is applied inside each attention block.
• RMSNorm instead of LayerNorm.
• SwiGLU feed-forward instead of GELU-FFN.

Intended as a smoke test / reference, not for serious training.

Constructor Summary

Constructors

Constructor	Description
TrainSmallLlama()

Method Summary

Modifier and Type	Method	Description
static void	main(String[] args)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

TrainSmallLlama

public TrainSmallLlama()

Method Details

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception