Package io.github.kirstenali.deepj.nn.transformer.nova

Class DeepJNova

java.lang.Object

io.github.kirstenali.deepj.nn.transformer.DecoderOnlyModel

io.github.kirstenali.deepj.nn.transformer.nova.DeepJNova

All Implemented Interfaces:

CausalLM, Persistable, Trainable

public final class DeepJNova
extends DecoderOnlyModel

A decoder with grouped-query attention, optional QK normalization, and tied embeddings.

Field Summary

Fields inherited from class io.github.kirstenali.deepj.nn.transformer.DecoderOnlyModel

lmHead, normF, stack, tokEmb

Constructor Summary

Constructors

Constructor	Description
DeepJNova(DeepJNovaConfig config, long seed)
DeepJNova(DeepJNovaConfig config, long seed, ParameterFactory parameters)
DeepJNova(DeepJNovaConfig config, long seed, DType dtype)

Method Summary

Modifier and Type	Method	Description
DeepJNovaConfig	config()
float	gradClipNorm()
List<Parameter>	parameters()
protected boolean	supportsCachedGeneration()
protected boolean	supportsRollingGeneration()

Methods inherited from class io.github.kirstenali.deepj.nn.transformer.DecoderOnlyModel

applyInitScale, embed, embed, embedBatch, embeddingParameters, forward, forwardBatch, load, metadata, openGeneration, openGeneration, promptFormat, save, setMetadata, setPromptFormat, supportsBatchedForward

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface io.github.kirstenali.deepj.training.Trainable

zeroGrad

Constructor Details

DeepJNova

public DeepJNova(DeepJNovaConfig config,
 long seed)

DeepJNova

public DeepJNova(DeepJNovaConfig config,
 long seed,
 DType dtype)

DeepJNova

public DeepJNova(DeepJNovaConfig config,
 long seed,
 ParameterFactory parameters)

Method Details

config

public DeepJNovaConfig config()

Specified by:

config in class DecoderOnlyModel

gradClipNorm

public float gradClipNorm()

supportsCachedGeneration

protected boolean supportsCachedGeneration()

Overrides:

supportsCachedGeneration in class DecoderOnlyModel

supportsRollingGeneration

protected boolean supportsRollingGeneration()

Overrides:

supportsRollingGeneration in class DecoderOnlyModel

parameters

public List<Parameter> parameters()

Specified by:

parameters in interface Persistable

Specified by:

parameters in interface Trainable

Overrides:

parameters in class DecoderOnlyModel