Class TransformerBlock

java.lang.Object
io.github.kirstenali.deepj.layers.transformer.TransformerBlock
All Implemented Interfaces:
Layer, Trainable

public final class TransformerBlock extends Object implements Layer
Pre-LN Transformer block: x = x + Attn(LN(x)) x = x + MLP(LN(x))