Fix chrF fallback so English wiki garbage fails translation_success

Sacrebleu errors used to fall back to set-overlap unigrams, so any
English hyp scored ~70–80 against English refs and SFT reported
success 1.0. Use count-based char n-grams, keep BLEU failures from
clobbering chrF, and print a few hyps during eval.
This commit is contained in:
dela
2026-08-26 14:23:22 +08:00
parent 5a7d949b01
commit 5cc0555563
4 changed files with 48 additions and 14 deletions
+10
View File
@@ -28,6 +28,16 @@ def test_good_zh2en_passes():
assert translation_success(src, hyp, ref, target_lang="en") is True
def test_english_wiki_garbage_does_not_pass_zh2en():
from kda.training.success import _chrf
src = "今天天气很好。"
hyp = "The first one's the time."
ref = "The weather is very nice today."
assert _chrf(hyp, ref) < 40.0
assert translation_success(src, hyp, ref, target_lang="en") is False
def test_container_help_exits_2():
import importlib.util
from pathlib import Path