Package io.github.kirstenali.deepj.nn.transformer.origin

Class DeepJOrigin

java.lang.Object

io.github.kirstenali.deepj.nn.transformer.DecoderOnlyModel

io.github.kirstenali.deepj.nn.transformer.origin.DeepJOrigin

All Implemented Interfaces:

CausalLM, Persistable, Trainable

public final class DeepJOrigin
extends DecoderOnlyModel

Field Summary

Fields inherited from class io.github.kirstenali.deepj.nn.transformer.DecoderOnlyModel

lmHead, normF, stack, tokEmb

Constructor Summary

Constructors

Constructor	Description
DeepJOrigin(DeepJOriginConfig cfg, long seed)
DeepJOrigin(DeepJOriginConfig cfg, long seed, ParameterFactory parameters)
DeepJOrigin(DeepJOriginConfig cfg, long seed, DType dtype)

Method Summary

Modifier and Type	Method	Description
DeepJOriginConfig	config()
protected Tensor	embed(int[] inputIds)
protected Tensor	embed(int[] inputIds, int offset)
protected Tensor	embedBatch(int[][] inputIds)
protected List<Parameter>	embeddingParameters()
float	gradClipNorm()
protected boolean	supportsCachedGeneration()

Methods inherited from class io.github.kirstenali.deepj.nn.transformer.DecoderOnlyModel

applyInitScale, forward, forwardBatch, load, metadata, openGeneration, openGeneration, parameters, promptFormat, save, setMetadata, setPromptFormat, supportsBatchedForward, supportsRollingGeneration

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface io.github.kirstenali.deepj.training.Trainable

zeroGrad

Constructor Details

DeepJOrigin

public DeepJOrigin(DeepJOriginConfig cfg,
 long seed)

DeepJOrigin

public DeepJOrigin(DeepJOriginConfig cfg,
 long seed,
 DType dtype)

DeepJOrigin

public DeepJOrigin(DeepJOriginConfig cfg,
 long seed,
 ParameterFactory parameters)

Method Details

embed

protected Tensor embed(int[] inputIds)

Overrides:

embed in class DecoderOnlyModel

embed

protected Tensor embed(int[] inputIds,
 int offset)

Overrides:

embed in class DecoderOnlyModel

embedBatch

protected Tensor embedBatch(int[][] inputIds)

Overrides:

embedBatch in class DecoderOnlyModel

embeddingParameters

protected List<Parameter> embeddingParameters()

Overrides:

embeddingParameters in class DecoderOnlyModel

gradClipNorm

public float gradClipNorm()

supportsCachedGeneration

protected boolean supportsCachedGeneration()

Overrides:

supportsCachedGeneration in class DecoderOnlyModel

config

public DeepJOriginConfig config()

Specified by:

config in class DecoderOnlyModel