DeepJ

0.6.0-alpha · Java 20+

Tensors and compact Transformers, built for Java.

Learn, train, and inspect GPT-, Llama-, and DeepSeek-style decoder models without leaving the JVM. Run anywhere on CPU or opt into Metal on Apple Silicon.

Install DeepJ Browse the API
  • MIT licensed
  • Built-in training gradients
  • 405 tests

Install

Available from Maven Central

Add DeepJ to a Java 20 or newer project. The published release includes binary, source, and Javadoc artifacts.

View the artifact on Maven Central
Mavenpom.xml
<dependency>
    <groupId>io.github.kirstenali</groupId>
    <artifactId>deepj</artifactId>
    <version>0.6.0-alpha</version>
</dependency>

Core library

Small enough to understand. Complete enough to train.

DeepJ keeps the path from tensor math to text generation visible, with focused APIs and no hidden Python runtime.

01

Tensors + gradients

Two-dimensional float32 tensors, built-in model gradient calculations, and tests that check gradient accuracy.

02

Three model families

Compact GPT-, Llama-, and DeepSeek-style decoder architectures behind a shared causal-LM API.

03

Tokenization

UTF-8 byte tokenization plus trainable, persistent byte-level BPE with special-token support.

04

Training utilities

AdamW, cosine schedules, gradient clipping, evaluation, checkpoints, and bounded-memory datasets.

05

Portable by default

A pure Java CPU backend runs everywhere; the optional Metal backend accelerates Apple Silicon.

06

Shareable checkpoints

Versioned DeepJ model bundles include weights, tokenizer, configuration, and a model card.

Quick start

Load a model and generate text

The checkpoint and configuration must describe the same architecture. Swap in a trained BPE tokenizer when loading a BPE-trained model.

See the repository guide
JavaGenerate.java
Tokenizer tokenizer = new ByteTokenizer();
DeepSeekConfig config = new DeepSeekConfig(
        tokenizer.vocabSize(), 128, 64, 4, 2, 256, 32, 16);

DeepSeekModel model = new DeepSeekModel(config, 42L);
model.load(Path.of("model.dj"));

String text = TextGenerator.generate(
        model, tokenizer, config, "Once upon a time", 80, 0.8f, 20, 42L);

Architectures

One learning-focused library, three approaches

GPT-style

The classic baseline

Learned positions, causal multi-head attention, LayerNorm, and GELU feed-forward layers.

Llama-style

Modern decoder blocks

Rotary position embeddings, causal multi-head attention, RMSNorm, and SwiGLU layers.

DeepSeek-style

Low-rank attention

Low-rank query and key/value projections with RoPE, RMSNorm, and SwiGLU.

These are compact educational architectures, not drop-in reproductions of the official GPT, Llama, or DeepSeek releases.

Published model

TinyStories DeepSeek demo

A 1.3M-parameter DeepJ model trained from scratch, published with its BPE tokenizer, configuration, model card, and native DeepJ weights.

Parameters
1.3M
Validation loss
2.182
Perplexity
8.866
Open on Hugging Face

Know the edges

Alpha software, clearly scoped

  • The public API and checkpoint format may change before a stable release.
  • Tensors are currently two-dimensional and use float32 values.
  • Each model layer has its own gradient code. DeepJ does not calculate gradients automatically for every tensor operation.
  • Generation recomputes context; incremental KV caching is not implemented yet.
  • Metal is Apple Silicon-only, and Hugging Face bundles use DeepJ—not PyTorch—formats.
  • DeepJ is not validated for safety-critical or production inference workloads.