Package io.github.kirstenali.deepj.models

Interface TransformerConfig

All Known Implementing Classes:
DeepSeekConfig, GPTConfig, LlamaConfig
public interface TransformerConfig
Common configuration fields shared by all decoder-only transformer models.

Implemented by GPTConfig and LlamaConfig.

  • Method Summary

    Modifier and Type
    Method
    Description
    int
    dFF()
     
    int
    dModel()
     
    float
    gradClipNorm()
     
    int
    maxSeqLen()
     
    int
    nHeads()
     
    int
    nLayers()
     
    static void
    validateCommon(int vocabSize, int maxSeqLen, int dModel, int nHeads, int nLayers, int dFF, float gradClipNorm)
    Validates the fields common to all transformer configs.
    static void
    validateRotaryHeadDimension(int dModel, int nHeads)
    Validates the even per-head width required by rotary embeddings.
    int
    vocabSize()
     

  • Method Details

    • vocabSize

      int vocabSize()

    • maxSeqLen

      int maxSeqLen()

    • dModel

      int dModel()

    • nHeads

      int nHeads()

    • nLayers

      int nLayers()

    • dFF

      int dFF()

    • gradClipNorm

      float gradClipNorm()

    • validateCommon

      static void validateCommon(int vocabSize, int maxSeqLen, int dModel, int nHeads, int nLayers, int dFF, float gradClipNorm)
      Validates the fields common to all transformer configs. Call from each record's compact constructor.

    • validateRotaryHeadDimension

      static void validateRotaryHeadDimension(int dModel, int nHeads)
      Validates the even per-head width required by rotary embeddings.