Package io.github.kirstenali.deepj.transformer

Class LoopedTransformer

java.lang.Object

io.github.kirstenali.deepj.transformer.LoopedTransformer

All Implemented Interfaces:

Layer, Trainable

public final class LoopedTransformer
extends Object
implements Layer

Repeats shared grouped-query attention and SwiGLU blocks; each pass saves its own activations.

Constructor Summary

Constructors

Constructor	Description
LoopedTransformer(DeepJLoopConfig config, Random random)
LoopedTransformer(DeepJLoopConfig config, Random random, ParameterFactory parameters)

Method Summary

Modifier and Type	Method	Description
Tensor	forward(Tensor input)
Tensor	forward(Tensor input, InferenceContext inference)
List<Parameter>	parameters()
boolean	supportsIncrementalInference()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface io.github.kirstenali.deepj.training.Trainable

zeroGrad

Constructor Details

LoopedTransformer

public LoopedTransformer(DeepJLoopConfig config,
 Random random)

LoopedTransformer

public LoopedTransformer(DeepJLoopConfig config,
 Random random,
 ParameterFactory parameters)

Method Details

forward

public Tensor forward(Tensor input)

Specified by:

forward in interface Layer

supportsIncrementalInference

public boolean supportsIncrementalInference()

Specified by:

supportsIncrementalInference in interface Layer

forward

public Tensor forward(Tensor input,
 InferenceContext inference)

Specified by:

forward in interface Layer

parameters

public List<Parameter> parameters()

Specified by:

parameters in interface Trainable