Package io.github.kirstenali.deepj.layers.transformer.blocks

Class DeepJOriginTransformerBlock

java.lang.Object

io.github.kirstenali.deepj.layers.transformer.blocks.DeepJOriginTransformerBlock

All Implemented Interfaces:

Layer, Trainable

public class DeepJOriginTransformerBlock
extends Object

Constructor Summary

Constructors

Constructor	Description
DeepJOriginTransformerBlock(int dModel, int nHeads, int dFF, Supplier<ActivationFunction> ffnActivationFactory, Random rnd)
DeepJOriginTransformerBlock(int dModel, int nHeads, int dFF, Supplier<ActivationFunction> ffnActivationFactory, Random rnd, ParameterFactory parameters)
DeepJOriginTransformerBlock(int dModel, int nHeads, int dFF, Random rnd)

Method Summary

Modifier and Type	Method	Description
Tensor	forward(Tensor input)
Tensor	forward(Tensor input, InferenceContext context)
List<Parameter>	parameters()
boolean	supportsIncrementalInference()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface io.github.kirstenali.deepj.training.Trainable

zeroGrad

Constructor Details

DeepJOriginTransformerBlock

public DeepJOriginTransformerBlock(int dModel,
 int nHeads,
 int dFF,
 Random rnd)

DeepJOriginTransformerBlock

public DeepJOriginTransformerBlock(int dModel,
 int nHeads,
 int dFF,
 Supplier<ActivationFunction> ffnActivationFactory,
 Random rnd)

DeepJOriginTransformerBlock

public DeepJOriginTransformerBlock(int dModel,
 int nHeads,
 int dFF,
 Supplier<ActivationFunction> ffnActivationFactory,
 Random rnd,
 ParameterFactory parameters)

Method Details

forward

public Tensor forward(Tensor input)

Specified by:

forward in interface Layer

forward

public Tensor forward(Tensor input,
 InferenceContext context)

Specified by:

forward in interface Layer

supportsIncrementalInference

public boolean supportsIncrementalInference()

Specified by:

supportsIncrementalInference in interface Layer

parameters

public List<Parameter> parameters()

Specified by:

parameters in interface Trainable