carrydela/K3
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
17 Commits 1 Branch 0 Tags
main
Commit Graph
2 Commits
Author SHA1 Message Date
dela a2c4217dae Document LatentMoE sigmoid routing, sparse dispatch, and K3 block figures 
Ledger C10–C12 match the permute-pad-bmm path and Switch aux/z-loss.
Section 8 adds overview and component TikZ; MoE capacity is C_moe so it
does not collide with KDA chunk size.
 2026-08-26 14:43:58 +08:00
dela 584f7e9e73 Initial K3 snapshot: 0.5B KDA/MLA/MoE train path 
Standalone tree split from LLMRL/projects/kda. Includes Triton dt_bias
backward fix, train_k3 --preset 0.5b, SFT, Docker runtime, and tests.
 2026-08-25 14:43:17 +08:00