carrydela/K3
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial K3 snapshot: 0.5B KDA/MLA/MoE train path

Browse Source 
Standalone tree split from LLMRL/projects/kda. Includes Triton dt_bias
backward fix, train_k3 --preset 0.5b, SFT, Docker runtime, and tests.
This commit is contained in:
dela
2026-08-25 14:43:17 +08:00
commit 584f7e9e73
140 changed files with 21592 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
data/sft/toy.jsonl
+32
View File
@@ -0,0 +1,32 @@
{"src": "你好。", "tgt": "Hello.", "target_lang": "en"}
{"src": "Hello.", "tgt": "你好。", "target_lang": "zh"}
{"src": "谢谢。", "tgt": "Thank you.", "target_lang": "en"}
{"src": "Thank you.", "tgt": "谢谢。", "target_lang": "zh"}
{"src": "我爱北京。", "tgt": "I love Beijing.", "target_lang": "en"}
{"src": "I love Beijing.", "tgt": "我爱北京。", "target_lang": "zh"}
{"src": "现在几点？", "tgt": "What time is it now?", "target_lang": "en"}
{"src": "What time is it now?", "tgt": "现在几点？", "target_lang": "zh"}
{"src": "水是透明的。", "tgt": "Water is transparent.", "target_lang": "en"}
{"src": "Water is transparent.", "tgt": "水是透明的。", "target_lang": "zh"}
{"src": "他是一名教师。", "tgt": "He is a teacher.", "target_lang": "en"}
{"src": "He is a teacher.", "tgt": "他是一名教师。", "target_lang": "zh"}
{"src": "明天会下雨。", "tgt": "It will rain tomorrow.", "target_lang": "en"}
{"src": "It will rain tomorrow.", "tgt": "明天会下雨。", "target_lang": "zh"}
{"src": "请坐。", "tgt": "Please sit down.", "target_lang": "en"}
{"src": "Please sit down.", "tgt": "请坐。", "target_lang": "zh"}
{"src": "这是一只狗。", "tgt": "This is a dog.", "target_lang": "en"}
{"src": "This is a dog.", "tgt": "这是一只狗。", "target_lang": "zh"}
{"src": "大门在左边。", "tgt": "The gate is on the left.", "target_lang": "en"}
{"src": "The gate is on the left.", "tgt": "大门在左边。", "target_lang": "zh"}
{"src": "我们走吧。", "tgt": "Let's go.", "target_lang": "en"}
{"src": "Let's go.", "tgt": "我们走吧。", "target_lang": "zh"}
{"src": "夜空中有星星。", "tgt": "There are stars in the night sky.", "target_lang": "en"}
{"src": "There are stars in the night sky.", "tgt": "夜空中有星星。", "target_lang": "zh"}
{"src": "面包放在厨房里。", "tgt": "The bread is in the kitchen.", "target_lang": "en"}
{"src": "The bread is in the kitchen.", "tgt": "面包放在厨房里。", "target_lang": "zh"}
{"src": "孩子在睡觉。", "tgt": "The child is sleeping.", "target_lang": "en"}
{"src": "The child is sleeping.", "tgt": "孩子在睡觉。", "target_lang": "zh"}
{"src": "这座山很高。", "tgt": "This mountain is very high.", "target_lang": "en"}
{"src": "This mountain is very high.", "tgt": "这座山很高。", "target_lang": "zh"}
{"src": "请关上门。", "tgt": "Please close the door.", "target_lang": "en"}
{"src": "Please close the door.", "tgt": "请关上门。", "target_lang": "zh"}