Keep attnres on resume, fix final chunk_index, default 0.5b to Yi-6B
CLI default attnres=off was overwriting block checkpoints on resume so later loads hit Unexpected key(s). Only apply flags the user passed. Track next_chunk so a budget-exit save does not skip the untrained yield. 0.5b now uses 01-ai/Yi-6B (64k); refuse resume when the ckpt tokenizer does not match.
This commit is contained in:
@@ -78,6 +78,7 @@ def test_preset_0_5b_schedule():
|
||||
assert cfg.hidden_size == 768
|
||||
assert cfg.num_heads * cfg.head_dim == cfg.hidden_size
|
||||
assert cfg.num_hidden_layers == 24
|
||||
assert cfg.vocab_size == 64000
|
||||
assert cfg.tie_word_embeddings
|
||||
assert cfg.chunk_size == 64
|
||||
assert cfg.gradient_checkpointing is True
|
||||
|
||||
Reference in New Issue
Block a user