dela 94d0f2ff6a Do not let HF tokenizers truncate wiki articles at 4096 ...

Yi-6B sets model_max_length=4096. encode() would clip long Wikipedia
pages before we pack seq_len chunks. Raise the cap so only our
chunker limits context.

2026-08-26 14:31:07 +08:00

..

_fla

Initial K3 snapshot: 0.5B KDA/MLA/MoE train path

2026-08-25 14:43:17 +08:00

layers

Keep LatentMoE routed bmm in activation dtype under bf16 autocast

2026-08-25 20:22:51 +08:00

models

Keep attnres on resume, fix final chunk_index, default 0.5b to Yi-6B

2026-08-26 14:23:29 +08:00

ops

Initial K3 snapshot: 0.5B KDA/MLA/MoE train path

2026-08-25 14:43:17 +08:00

training

Do not let HF tokenizers truncate wiki articles at 4096

2026-08-26 14:31:07 +08:00

__init__.py

Initial K3 snapshot: 0.5B KDA/MLA/MoE train path

2026-08-25 14:43:17 +08:00