Package io.github.kirstenali.deepj.layers.transformer

Class SwiGLULayer

java.lang.Object

io.github.kirstenali.deepj.layers.transformer.SwiGLULayer

All Implemented Interfaces:

Layer, Trainable

public final class SwiGLULayer
extends Object
implements Layer

Constructor Summary

Constructors

Constructor	Description
SwiGLULayer(int dModel, int dFF, Random rnd)
SwiGLULayer(int dModel, int dFF, Random rnd, boolean bias)
SwiGLULayer(int dModel, int dFF, Random rnd, boolean bias, ParameterFactory parameters)

Method Summary

Modifier and Type	Method	Description
Tensor	forward(Tensor x)
List<Parameter>	parameters()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface io.github.kirstenali.deepj.layers.Layer

forward, supportsIncrementalInference

Methods inherited from interface io.github.kirstenali.deepj.training.Trainable

zeroGrad

Constructor Details

SwiGLULayer

public SwiGLULayer(int dModel,
 int dFF,
 Random rnd)

SwiGLULayer

public SwiGLULayer(int dModel,
 int dFF,
 Random rnd,
 boolean bias)

SwiGLULayer

public SwiGLULayer(int dModel,
 int dFF,
 Random rnd,
 boolean bias,
 ParameterFactory parameters)

Method Details

forward

public Tensor forward(Tensor x)

Specified by:

forward in interface Layer

parameters

public List<Parameter> parameters()

Specified by:

parameters in interface Trainable