Package io.github.kirstenali.deepj.nn.transformer.loop

Class DeepJLoop

java.lang.Object

io.github.kirstenali.deepj.nn.transformer.DecoderOnlyModel

io.github.kirstenali.deepj.nn.transformer.loop.DeepJLoop

All Implemented Interfaces:

CausalLM, Persistable, Trainable

public final class DeepJLoop
extends DecoderOnlyModel

A looped causal decoder reusing grouped-query attention and SwiGLU blocks across passes.

Field Summary

Fields inherited from class io.github.kirstenali.deepj.nn.transformer.DecoderOnlyModel

lmHead, normF, stack, tokEmb

Constructor Summary

Constructors

Constructor	Description
DeepJLoop(DeepJLoopConfig config, long seed)
DeepJLoop(DeepJLoopConfig config, long seed, ParameterFactory parameters)
DeepJLoop(DeepJLoopConfig config, long seed, DType dtype)

Method Summary

Modifier and Type	Method	Description
DeepJLoopConfig	config()
float	gradClipNorm()
List<Parameter>	parameters()
protected boolean	supportsCachedGeneration()
protected boolean	supportsRollingGeneration()

Methods inherited from class io.github.kirstenali.deepj.nn.transformer.DecoderOnlyModel

applyInitScale, embed, embed, embedBatch, embeddingParameters, forward, forwardBatch, load, metadata, openGeneration, openGeneration, promptFormat, save, setMetadata, setPromptFormat, supportsBatchedForward

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface io.github.kirstenali.deepj.training.Trainable

zeroGrad

Constructor Details

DeepJLoop

public DeepJLoop(DeepJLoopConfig config,
 long seed)

DeepJLoop

public DeepJLoop(DeepJLoopConfig config,
 long seed,
 DType dtype)

DeepJLoop

public DeepJLoop(DeepJLoopConfig config,
 long seed,
 ParameterFactory parameters)

Method Details

config

public DeepJLoopConfig config()

Specified by:

config in class DecoderOnlyModel

gradClipNorm

public float gradClipNorm()

supportsCachedGeneration

protected boolean supportsCachedGeneration()

Overrides:

supportsCachedGeneration in class DecoderOnlyModel

supportsRollingGeneration

protected boolean supportsRollingGeneration()

Overrides:

supportsRollingGeneration in class DecoderOnlyModel

parameters

public List<Parameter> parameters()

Specified by:

parameters in interface Persistable

Specified by:

parameters in interface Trainable

Overrides:

parameters in class DecoderOnlyModel