Package io.github.kirstenali.deepj.layers.transformer.norm

Class LayerNorm1D

java.lang.Object

io.github.kirstenali.deepj.layers.transformer.norm.LayerNorm1D

All Implemented Interfaces:

Layer, NormLayer, Trainable

public final class LayerNorm1D
extends Object
implements NormLayer

LayerNorm over feature dimension (cols) with trainable gamma/beta exposed as Parameters.

Constructor Summary

Constructors

Constructor	Description
LayerNorm1D(int dim)

Method Summary

Modifier and Type	Method	Description
Tensor	backward(Tensor gradOut)
Tensor	forward(Tensor x)
List<Parameter>	parameters()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface io.github.kirstenali.deepj.training.Trainable

zeroGrad

Constructor Details

LayerNorm1D

public LayerNorm1D(int dim)

Method Details

forward

public Tensor forward(Tensor x)

Specified by:

forward in interface Layer

backward

public Tensor backward(Tensor gradOut)

Specified by:

backward in interface Layer

parameters

public List<Parameter> parameters()

Specified by:

parameters in interface Trainable