feat: add superderive child and open phase-2 routing
Attach the stepwise derivation toolkit as a submodule and default SUPERPAPER_PHASE to 2 so rewrite-figure can use superderive. Notes still keep align for short expansions without cancel or subst.
This commit is contained in:
@@ -0,0 +1,46 @@
|
||||
schema: superpaper.ledger/v1
|
||||
retired_ids: []
|
||||
paper:
|
||||
id: derive-delegate
|
||||
title: Scaled attention
|
||||
authors: ["Fixture"]
|
||||
notes_language: zh
|
||||
source: {kind: excerpt}
|
||||
coverage:
|
||||
mode: excerpt
|
||||
questions:
|
||||
- {id: Q1, text: "缩放从哪来?", source: "excerpt"}
|
||||
claims:
|
||||
- id: C1
|
||||
text: "1/√d_k 是改写,不是新算子"
|
||||
kind: theoretical
|
||||
status: core
|
||||
supports: [Q1]
|
||||
definitions: []
|
||||
assumptions: []
|
||||
lemmas: []
|
||||
symbols:
|
||||
- {name: Q, latex: "Q", meaning: "query", kind: value}
|
||||
- {name: K, latex: "K", meaning: "key", kind: value}
|
||||
- {name: dk, latex: "d_k", meaning: "key 维", kind: "shape parameter"}
|
||||
derivations:
|
||||
- id: DER1
|
||||
claim: C1
|
||||
title: "缩放"
|
||||
expand: true
|
||||
figure: F3
|
||||
steps:
|
||||
- {id: S1, from: "QK^{\\top}", to: "QK^{\\top}/\\sqrt{d_k}", rule: scale}
|
||||
figures:
|
||||
- id: F3
|
||||
claim: C1
|
||||
title: "缩放来自方差"
|
||||
grammar: derivation
|
||||
toolkit: superderive
|
||||
signals: [rewrite-figure]
|
||||
request: figures/F3/F3.request.md
|
||||
include: figures/F3/build/F3.pdf
|
||||
status: included
|
||||
evidence: []
|
||||
terms: []
|
||||
source_assets: []
|
||||
@@ -0,0 +1 @@
|
||||
$QK^\top \to QK^\top/\sqrt{d_k}$。缩放压方差,避免 softmax 饱和。
|
||||
@@ -0,0 +1,14 @@
|
||||
# Figure request F3
|
||||
toolkit: superderive
|
||||
language: cjk
|
||||
claim: 1/√d_k 是改写,不是新算子
|
||||
grammar: derivation
|
||||
work_rel_dir: figures/F3
|
||||
|
||||
## Flow
|
||||
stage: {name: D1, text: "缩放来自方差,不是装饰"}
|
||||
row: {name: R1}
|
||||
in_row:
|
||||
- {macro: sdstep, name: s1, role: rewrite,
|
||||
lhs: "QK^{\\top}", rhs: "QK^{\\top}/\\sqrt{d_k}"}
|
||||
- {macro: sdreason, name: s1, text: "§3.2.1,避免 softmax 饱和"}
|
||||
@@ -0,0 +1,20 @@
|
||||
% Golden example -- a scale rewrite that must be seen, not just aligned.
|
||||
% ../scripts/build.sh rewrite-cancel.tex
|
||||
\documentclass[border=10pt]{standalone}
|
||||
\usepackage[cjk]{superderive}
|
||||
|
||||
\begin{document}
|
||||
\begin{tikzpicture}
|
||||
\sdstage{D1}{缩放来自方差,不是装饰}
|
||||
\sdrow{R1}
|
||||
\sdstep[role=rewrite]{s1}{QK^{\top}}{QK^{\top}/\sqrt{d_k}}
|
||||
\sdreason{s1}{§3.2.1,避免 softmax 饱和}
|
||||
\sdrowend
|
||||
\sdbbox{all}
|
||||
\sdtopformula{F}{$\mathrm{Attention}(Q,K,V)=\mathrm{softmax}(QK^{\top}/\sqrt{d_k})V$}
|
||||
\sdmeaningbox{mb}{120mm}{all}
|
||||
{点积方差随 $d_k$ 线性涨}
|
||||
{缩放是改写,不是新算子}
|
||||
{没有缩放,softmax 进饱和区,梯度消失}
|
||||
\end{tikzpicture}
|
||||
\end{document}
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 109 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 15 KiB |
Binary file not shown.
Binary file not shown.
|
After Width: | Height: | Size: 105 KiB |
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 238 KiB |
@@ -0,0 +1,14 @@
|
||||
\documentclass[a4paper]{article}
|
||||
\input{notes-macros}
|
||||
\renewcommand{\notetitle}{缩放来自方差}
|
||||
\renewcommand{\notepaper}{Scaled attention}
|
||||
\begin{document}
|
||||
\tableofcontents
|
||||
\newpage
|
||||
\input{sections/sec-01.tex}
|
||||
\input{sections/sec-02.tex}
|
||||
\input{sections/sec-03.tex}
|
||||
\appendix
|
||||
\section{符号表}
|
||||
\input{sections/symbols.tex}
|
||||
\end{document}
|
||||
@@ -0,0 +1,2 @@
|
||||
\section{这篇论文在问什么}
|
||||
注意力里的 $1/\sqrt{d_k}$ 是装饰,还是一次必须看见的改写?
|
||||
@@ -0,0 +1,8 @@
|
||||
\section{主张与贡献}
|
||||
\splabel{C1}
|
||||
点积方差随 $d_k$ 涨。除掉 $\sqrt{d_k}$ 是改写,不是新算子。
|
||||
|
||||
\spfig{F3}{缩放来自方差。}{fixture;toolkit: \texttt{superderive};ledger id: F3。}
|
||||
|
||||
\subsection{本章小结}
|
||||
这一步必须被看见,所以走 superderive,而不是笔记里的 \texttt{align}。
|
||||
@@ -0,0 +1,2 @@
|
||||
\section{总结与延伸}
|
||||
四步以内、没有消项或代入要画出来时,仍然用 \texttt{align}。
|
||||
@@ -0,0 +1,14 @@
|
||||
# Outline: Scaled attention
|
||||
|
||||
## Lecture map
|
||||
|
||||
| file | lecture_title | paper_sections | ledger_ids |
|
||||
|---|---|---|---|
|
||||
| sec-01.tex | 这篇论文在问什么 | 1 | Q1 |
|
||||
| sec-02.tex | 主张与贡献 | 1 | C1, F3 |
|
||||
| sec-03.tex | 总结与延伸 | 1 | C1 |
|
||||
| sec-app-a.tex | 符号表 | — | |
|
||||
|
||||
## Locked
|
||||
- 首节标题必须是「这篇论文在问什么」
|
||||
- 末节(appendix 前)必须是「总结与延伸」
|
||||
Reference in New Issue
Block a user