Package io.github.kirstenali.deepj.models.llama

Record Class LlamaConfig

java.lang.Object
java.lang.Record
io.github.kirstenali.deepj.models.llama.LlamaConfig
All Implemented Interfaces:
TransformerConfig
public record LlamaConfig(int vocabSize, int maxSeqLen, int dModel, int nHeads, int nLayers, int dFF, float initScale, float gradClipNorm) extends Record implements TransformerConfig

  • Constructor Summary

    Constructors
    Constructor
    Description
    LlamaConfig(int vocabSize, int maxSeqLen, int dModel, int nHeads, int nLayers, int dFF)
    Convenience constructor with stable initialization and gradient-clipping defaults.
    LlamaConfig(int vocabSize, int maxSeqLen, int dModel, int nHeads, int nLayers, int dFF, float gradClipNorm)
    Convenience constructor retaining the historical gradient-clip argument.
    LlamaConfig(int vocabSize, int maxSeqLen, int dModel, int nHeads, int nLayers, int dFF, float initScale, float gradClipNorm)
    Creates an instance of a LlamaConfig record class.

  • Method Summary

    Modifier and Type
    Method
    Description
    static int
    defaultDFF(int dModel)
    Returns a common Llama-style dFF for the given dModel: round(8/3 * dModel) rounded up to the nearest multiple of 64.
    int
    dFF()
    Returns the value of the dFF record component.
    int
    dModel()
    Returns the value of the dModel record component.
    final boolean
    equals(Object o)
    Indicates whether some other object is "equal to" this one.
    float
    gradClipNorm()
    Returns the value of the gradClipNorm record component.
    final int
    hashCode()
    Returns a hash code value for this object.
    float
    initScale()
    Returns the value of the initScale record component.
    int
    maxSeqLen()
    Returns the value of the maxSeqLen record component.
    int
    nHeads()
    Returns the value of the nHeads record component.
    int
    nLayers()
    Returns the value of the nLayers record component.
    final String
    toString()
    Returns a string representation of this record class.
    int
    vocabSize()
    Returns the value of the vocabSize record component.

    Methods inherited from class java.lang.Object

    clone, finalize, getClass, notify, notifyAll, wait, wait, wait

  • Constructor Details

    • LlamaConfig

      public LlamaConfig(int vocabSize, int maxSeqLen, int dModel, int nHeads, int nLayers, int dFF)
      Convenience constructor with stable initialization and gradient-clipping defaults.

    • LlamaConfig

      public LlamaConfig(int vocabSize, int maxSeqLen, int dModel, int nHeads, int nLayers, int dFF, float gradClipNorm)
      Convenience constructor retaining the historical gradient-clip argument.

    • LlamaConfig

      public LlamaConfig(int vocabSize, int maxSeqLen, int dModel, int nHeads, int nLayers, int dFF, float initScale, float gradClipNorm)
      Creates an instance of a LlamaConfig record class.
      Parameters:
      vocabSize - the value for the vocabSize record component
      maxSeqLen - the value for the maxSeqLen record component
      dModel - the value for the dModel record component
      nHeads - the value for the nHeads record component
      nLayers - the value for the nLayers record component
      dFF - the value for the dFF record component
      initScale - the value for the initScale record component
      gradClipNorm - the value for the gradClipNorm record component

  • Method Details

    • defaultDFF

      public static int defaultDFF(int dModel)
      Returns a common Llama-style dFF for the given dModel: round(8/3 * dModel) rounded up to the nearest multiple of 64.

    • toString

      public final String toString()
      Returns a string representation of this record class. The representation contains the name of the class, followed by the name and value of each of the record components.
      Specified by:
      toString in class Record
      Returns:
      a string representation of this object

    • hashCode

      public final int hashCode()
      Returns a hash code value for this object. The value is derived from the hash code of each of the record components.
      Specified by:
      hashCode in class Record
      Returns:
      a hash code value for this object

    • equals

      public final boolean equals(Object o)
      Indicates whether some other object is "equal to" this one. The objects are equal if the other object is of the same class and if all the record components are equal. All components in this record class are compared with '=='.
      Specified by:
      equals in class Record
      Parameters:
      o - the object with which to compare
      Returns:
      true if this object is the same as the o argument; false otherwise.

    • vocabSize

      public int vocabSize()
      Returns the value of the vocabSize record component.
      Specified by:
      vocabSize in interface TransformerConfig
      Returns:
      the value of the vocabSize record component

    • maxSeqLen

      public int maxSeqLen()
      Returns the value of the maxSeqLen record component.
      Specified by:
      maxSeqLen in interface TransformerConfig
      Returns:
      the value of the maxSeqLen record component

    • dModel

      public int dModel()
      Returns the value of the dModel record component.
      Specified by:
      dModel in interface TransformerConfig
      Returns:
      the value of the dModel record component

    • nHeads

      public int nHeads()
      Returns the value of the nHeads record component.
      Specified by:
      nHeads in interface TransformerConfig
      Returns:
      the value of the nHeads record component

    • nLayers

      public int nLayers()
      Returns the value of the nLayers record component.
      Specified by:
      nLayers in interface TransformerConfig
      Returns:
      the value of the nLayers record component

    • dFF

      public int dFF()
      Returns the value of the dFF record component.
      Specified by:
      dFF in interface TransformerConfig
      Returns:
      the value of the dFF record component

    • initScale

      public float initScale()
      Returns the value of the initScale record component.
      Returns:
      the value of the initScale record component

    • gradClipNorm

      public float gradClipNorm()
      Returns the value of the gradClipNorm record component.
      Specified by:
      gradClipNorm in interface TransformerConfig
      Returns:
      the value of the gradClipNorm record component