Package io.github.kirstenali.deepj.nn.transformer.orbit

Class DeepJOrbit

java.lang.Object

io.github.kirstenali.deepj.nn.transformer.DecoderOnlyModel

io.github.kirstenali.deepj.nn.transformer.orbit.DeepJOrbit

All Implemented Interfaces:

CausalLM, Persistable, Trainable

public final class DeepJOrbit
extends DecoderOnlyModel

Field Summary

Fields inherited from class io.github.kirstenali.deepj.nn.transformer.DecoderOnlyModel

lmHead, normF, stack, tokEmb

Constructor Summary

Constructors

Constructor	Description
DeepJOrbit(DeepJOrbitConfig cfg, long seed)
DeepJOrbit(DeepJOrbitConfig cfg, long seed, ParameterFactory parameters)
DeepJOrbit(DeepJOrbitConfig cfg, long seed, DType dtype)

Method Summary

Modifier and Type	Method	Description
DeepJOrbitConfig	config()
float	gradClipNorm()
protected boolean	supportsCachedGeneration()
protected boolean	supportsRollingGeneration()

Methods inherited from class io.github.kirstenali.deepj.nn.transformer.DecoderOnlyModel

applyInitScale, embed, embed, embedBatch, embeddingParameters, forward, forwardBatch, load, metadata, openGeneration, openGeneration, parameters, promptFormat, save, setMetadata, setPromptFormat, supportsBatchedForward

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface io.github.kirstenali.deepj.training.Trainable

zeroGrad

Constructor Details

DeepJOrbit

public DeepJOrbit(DeepJOrbitConfig cfg,
 long seed)

DeepJOrbit

public DeepJOrbit(DeepJOrbitConfig cfg,
 long seed,
 DType dtype)

DeepJOrbit

public DeepJOrbit(DeepJOrbitConfig cfg,
 long seed,
 ParameterFactory parameters)

Method Details

supportsCachedGeneration

protected boolean supportsCachedGeneration()

Overrides:

supportsCachedGeneration in class DecoderOnlyModel

supportsRollingGeneration

protected boolean supportsRollingGeneration()

Overrides:

supportsRollingGeneration in class DecoderOnlyModel

gradClipNorm

public float gradClipNorm()

config

public DeepJOrbitConfig config()

Specified by:

config in class DecoderOnlyModel