Package io.github.kirstenali.deepj.nn

Class TransformerEncoder

java.lang.Object

io.github.kirstenali.deepj.nn.Module

io.github.kirstenali.deepj.nn.TransformerEncoder

All Implemented Interfaces:

Layer, Trainable

public final class TransformerEncoder
extends Module

Stacked encoder for [batch, time, width] embeddings, with padding masks and masked mean pooling.

Constructor Summary

Constructors

Constructor	Description
TransformerEncoder(TransformerEncoderConfig config, Random random)
TransformerEncoder(TransformerEncoderConfig config, Random random, ParameterFactory parameters)

Method Summary

Modifier and Type	Method	Description
protected List<Layer>	children()
TransformerEncoderConfig	config()
Tensor	forward(Tensor input)
Tensor	forward(Tensor input, boolean[][] validTokens)
Tensor	meanPool(Tensor input, boolean[][] validTokens)

Methods inherited from class io.github.kirstenali.deepj.nn.Module

buffers, eval, isTraining, parameters, setTraining, train

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface io.github.kirstenali.deepj.layers.Layer

forward, supportsIncrementalInference

Methods inherited from interface io.github.kirstenali.deepj.training.Trainable

zeroGrad

Constructor Details

TransformerEncoder

public TransformerEncoder(TransformerEncoderConfig config,
 Random random)

TransformerEncoder

public TransformerEncoder(TransformerEncoderConfig config,
 Random random,
 ParameterFactory parameters)

Method Details

config

public TransformerEncoderConfig config()

children

protected List<Layer> children()

Overrides:

children in class Module

forward

public Tensor forward(Tensor input)

forward

public Tensor forward(Tensor input,
 boolean[][] validTokens)

meanPool

public Tensor meanPool(Tensor input,
 boolean[][] validTokens)