2 Commits

Include renames

Author	SHA1	Message	Date
dela	49aede9cb2	Fit 0.5b training on 32GB: SDPA MLA, block checkpoint, chunked CE ... Whole-mixer checkpoint plus T×T MLA scores OOM'd a 31GB GPU on backward. Checkpoint each AttnRes block, run absorbed MLA through SDPA, and compute CE in vocab chunks so [B,T,V] logits are never materialized. --max-tokens is now the training budget; default --steps 2000 no longer caps a 1B-token run at 250 optimizer steps.	2026-08-25 20:09:27 +08:00
dela	584f7e9e73	Initial K3 snapshot: 0.5B KDA/MLA/MoE train path ... Standalone tree split from LLMRL/projects/kda. Includes Triton dt_bias backward fix, train_k3 --preset 0.5b, SFT, Docker runtime, and tests.	2026-08-25 14:43:17 +08:00