alethic
open weights · open data · open results

Models, datasets and the research behind them.

Models

flagship · public weights

alethic-300m

A 300M-parameter model trained for code, English and Uzbek, with a 4,096-token context and a Unigram tokenizer. Comes as a matched pair: SGCA and a GQA baseline.

4,096 ctxUnigram tokenizertrained on one H100

alethic-151m

Pretraining and SinGatedAttention exploration from scratch on a single Colab T4. No matched baseline, which is why the 300M pair exists.

Model card →

ruzz-116m

The 116M text-generation model that came before the Alethic line.

Model card →

sinegated-lm-1m

Parameter-matched small LM experiment with a sinusoidal attention gate, trained on Tiny Shakespeare.

Model card →
dataset · gated access

alethic-newborn

The supervised fine-tuning data behind the Alethic assistant: instruction answers, thinking blocks, tool use, context maps, and long project sessions. English and Uzbek.

Request access
Accept the terms on Hugging Face to download. MIT / gated dataset license.
13,810
examples
2
languages: English, Uzbek

594 of the examples are long project sessions, written from the author's own projects and analysed with Gemini. Their ids start with D.

Research log

Main line: does separating attention retrieval from learned residual control help? Tested against conventional baselines at matched parameters and compute.

AHGN

Adaptive Harmonic Graph Networks

A 2026 proposal: learnable sinusoidal components alongside linear pathways, multiplicative harmonic terms, and adaptive complexity.

proposal
1 · small

SinGatedLM at ~64K and ~1M parameters

Parameter-matched comparison of a sinusoidal attention gate on Tiny Shakespeare.

2 · scale up

116M, then Alethic-151M

Pretraining from scratch on Colab. Useful signal, but no matched baseline, so no claim.

3 · controlled

Alethic-300M matched pair at 1B tokens

SGCA against GQA with a narrow-GQA ablation. At 1B tokens the GQA baseline came out ahead, and the code-focused model continues from the GQA checkpoint.

4 · in progress

10M SGCA vs GQA, then SFT

Overtrained 10M pair on the same 1B tokens, then supervised fine-tuning of Alethic-300M for real answers in English, Uzbek and code.

Empirical Benchmark · Alethic-300M (1B Tokens)

Validation Loss Relative to GQA Baseline

Held-out evaluation across 1,017,192 tokens from six sources (seed 1234, 1× H100 80 GB).

GQA (302.9M · 0.8068)GQA-narrow (277.7M · 0.8110)SGCA (302.9M · 0.8472)
-0.15-0.10-0.050.00+0.050M200M400M600M800M1.0BCrossover ~145M tokens
Checkpoint
1.0B (15,259)
GQA
0.8068 BPB
GQA-narrow
0.8110 BPB
SGCA
0.8472 BPB
SGCA − GQA
+0.0404
Mechanism Probe · SGCA vs GQA

Residual Control Surface

C = g · (α · sin A)
-π/2+π/2GQA (A)SGCA (g·α·sin A)
Input A
1.10
SGCA C
0.891
Slope ∂C/∂A
0.454
Regime
Transition

Illustration of the SGCA residual update vs. GQA identity. Near A = 0, sin(A) ≈ A; beyond ±π/2, the gradient cos(A) vanishes and flips sign, bounding residual magnitude to |g·α|.

Every run is published with its configs, tokenizer and data notes, checkpoints, logs and measured comparisons.

All artifacts on Hugging Face