supertensor: shape-aware tensor figure toolkit
Extracted from the tensor-formula-viz skill and rebuilt around the idea that the geometry rules should be enforced by construction rather than restated as prose an agent has to remember. - assets/supertensor.sty: faces, stacks, index faces, shared caption lanes, meaning box, signature. Macros take a declared axis and a declared role, so equal shapes get equal edges, a x a is square, a transpose swaps the face, and contracted axes share an edge length -- without any manual alignment. - scripts/preflight.sh: decide the TikZ/CJK path before drawing. - scripts/build.sh: compile and fail on silent corruption (missing CJK glyphs, overfull boxes, undeclared roles), then export pdf/svg/png/thumb. - scripts/test.sh: build every figure as a regression test for the package. - examples/: three golden figures (TP-FFN, causal MHA, MoE top-k gather) plus an anti-pattern gallery of figures that compile cleanly and still lie. - SKILL.md + references/: lean entry point, details loaded on demand.
This commit is contained in:
dela
@@ -0,0 +1,7 @@
|
|||||||
|
build/
|
||||||
|
*.aux
|
||||||
|
*.log
|
||||||
|
*.out
|
||||||
|
*.fls
|
||||||
|
*.fdb_latexmk
|
||||||
|
*.synctex.gz
|
||||||
@@ -0,0 +1,87 @@
|
|||||||
|
# supertensor
|
||||||
|
|
||||||
|
A shape-aware toolkit for drawing tensor formulas: a LaTeX/TikZ macro package, a build
|
||||||
|
pipeline that fails on silent corruption, worked examples, and an agent skill that ties
|
||||||
|
them together.
|
||||||
|
|
||||||
|
It exists because figures of this kind fail in a specific way — they compile, they look
|
||||||
|
clean, and they tell the reader something false. A face captioned `Kᵀ` that was never
|
||||||
|
transposed; shards that do not tile their parent; an index tensor drawn with a lightness
|
||||||
|
ramp. The package's job is to make the correct thing the easy thing.
|
||||||
|
|
||||||
|
```
|
||||||
|
SKILL.md the skill entry point (lean; loads references on demand)
|
||||||
|
references/ geometry, semantics, layout, style, api, checklist, antipatterns
|
||||||
|
assets/supertensor.sty the macro package
|
||||||
|
scripts/preflight.sh is the TikZ + CJK path available?
|
||||||
|
scripts/build.sh compile, audit the log, export pdf/svg/png/thumb
|
||||||
|
examples/ three golden examples + an anti-pattern gallery
|
||||||
|
```
|
||||||
|
|
||||||
|
## Quick start
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/preflight.sh # 0 = full path, 1 = degraded, 2 = no LaTeX
|
||||||
|
./scripts/build.sh examples/mha-causal.tex # -> examples/build/mha-causal.{pdf,svg,png}
|
||||||
|
```
|
||||||
|
|
||||||
|
A minimal figure:
|
||||||
|
|
||||||
|
```tex
|
||||||
|
\documentclass[border=10pt]{standalone}
|
||||||
|
\usepackage[cjk]{supertensor}
|
||||||
|
|
||||||
|
\stsetrole{act}{stTeal} % one hue per tensor role, held across every stage
|
||||||
|
\stsetrole{w}{stOrange}
|
||||||
|
\stdim{T}{6} % one symbolic axis -> one physical edge length
|
||||||
|
\stdim{d}{4}
|
||||||
|
|
||||||
|
\begin{document}\begin{tikzpicture}
|
||||||
|
\stface[role=act, bracket=true]{X}{(0,0)}{T}{d}
|
||||||
|
\node[st op, right=6mm of X] (m) {$\times$};
|
||||||
|
\stface[role=w]{W}{($(m)+(1.4,0)$)}{d}{d}
|
||||||
|
\node[inner sep=0pt, fit=(X)(W)] (row) {};
|
||||||
|
\stlane{row}
|
||||||
|
\stcaption{X}{$\mathbf X$}{$T\times d$}
|
||||||
|
\stcaption{W}{$\mathbf W$}{$d\times d$}
|
||||||
|
\stnolane
|
||||||
|
\end{tikzpicture}\end{document}
|
||||||
|
```
|
||||||
|
|
||||||
|
Because `T` and `d` come from the ledger, the contracted axis is automatically one edge
|
||||||
|
length in both operands, `d×d` is automatically square, and any other face of shape `T×d`
|
||||||
|
in the figure is automatically identical to `X`.
|
||||||
|
|
||||||
|
See `references/api.md` for the full macro list.
|
||||||
|
|
||||||
|
## Examples
|
||||||
|
|
||||||
|
| file | shows |
|
||||||
|
|---|---|
|
||||||
|
| `tp-ffn-allreduce.tex` | column-then-row sharding, exact tiling, one hue per TP rank, a collective as a real node |
|
||||||
|
| `mha-causal.tex` | leading axes as stack depth, a physically swapped `Kᵀ`, a mask in a different grammar from the scores it gates |
|
||||||
|
| `moe-topk-gather.tex` | scores → indices → Boolean support → gather, with all three cell grammars side by side |
|
||||||
|
| `antipatterns.tex` | four figures that compile cleanly and still teach something false |
|
||||||
|
|
||||||
|
## Using it as an agent skill
|
||||||
|
|
||||||
|
`SKILL.md` is the entry point; the `references/` files are loaded on demand. Point your
|
||||||
|
agent runtime at this directory (for Claude Code, symlink or copy it under
|
||||||
|
`~/.claude/skills/`). The skill assumes `scripts/` and `assets/` sit beside it.
|
||||||
|
|
||||||
|
## Why the build script fails on warnings
|
||||||
|
|
||||||
|
Two LaTeX warnings produce a figure that is quietly wrong rather than visibly broken:
|
||||||
|
`Missing character` (a CJK glyph silently dropped — the label just is not there) and
|
||||||
|
`Overfull \hbox` (text escaping its reserved lane and landing on a tensor). `build.sh`
|
||||||
|
greps for both and exits non-zero. A `Package supertensor Warning` — an undeclared role
|
||||||
|
falling back to gray — is treated the same way.
|
||||||
|
|
||||||
|
A clean build still proves nothing about collisions, hue budget or whether the math is
|
||||||
|
right. That is what `references/checklist.md` is for.
|
||||||
|
|
||||||
|
## Provenance
|
||||||
|
|
||||||
|
Extracted from the `tensor-formula-viz` skill in `wdkns-skills`, which remains in place
|
||||||
|
unchanged. The prose rules that could be enforced mechanically became macros; the rest
|
||||||
|
became `references/`.
|
||||||
@@ -0,0 +1,93 @@
|
|||||||
|
---
|
||||||
|
name: supertensor
|
||||||
|
description: Create or refine clean, shape-aware figures for tensor/matrix/vector formulas or tensor code — matrix-block diagrams, entry heatmaps, row/column shard stripes, stacked 3D/4D tensors, attention, tensor/expert parallelism, broadcasting, reductions, contractions, gather/scatter and routing. Use whenever tensor shapes or axis meanings must be visually aligned with the computation. Not for plotting numeric data (loss curves, benchmark bars, scatter plots), architecture block diagrams without shapes, or generic flowcharts.
|
||||||
|
---
|
||||||
|
|
||||||
|
# supertensor
|
||||||
|
|
||||||
|
Turn a formula or a tensor-code path into one dense, slide-ready figure with three zones:
|
||||||
|
|
||||||
|
1. **Top — formula.** The clean mathematical definition. No shape underbraces.
|
||||||
|
2. **Middle — computation.** Colored faces, shards, stacks, operators, collectives.
|
||||||
|
3. **Bottom — meaning.** Axes, object semantics, mechanism. One box, three rows.
|
||||||
|
|
||||||
|
The `assets/supertensor.sty` package enforces most of the geometry and style rules
|
||||||
|
by construction. **Draw with the package; do not hand-roll `\draw` rectangles.**
|
||||||
|
A figure built from raw TikZ has to re-earn every invariant by hand and usually fails one.
|
||||||
|
|
||||||
|
## Workflow
|
||||||
|
|
||||||
|
1. **Preflight.** `./scripts/preflight.sh`. Exit 0 = TikZ+CJK path. Exit 1 = degraded
|
||||||
|
(say so in the delivery). Exit 2 = no LaTeX; fall back to SVG/matplotlib and say
|
||||||
|
explicitly that the figure is not TikZ.
|
||||||
|
2. **Reduce** the input to one primary computation path. Drop equivalent objectives,
|
||||||
|
diagnostics, and secondary metrics unless asked for.
|
||||||
|
3. **Build two ledgers** before drawing anything:
|
||||||
|
- *shape & semantics* — per symbol: semantic kind, dtype/domain, global and local
|
||||||
|
shape, axis meanings, producer/consumer, contracted/broadcast/reduced axes.
|
||||||
|
See `references/semantics.md`.
|
||||||
|
- *geometry* — one `\stdim{axis}{cells}` per symbolic axis, one `\stsetrole{role}{color}`
|
||||||
|
per tensor role. Declaring these makes the invariants automatic.
|
||||||
|
See `references/geometry.md`.
|
||||||
|
4. **Pick the smallest grammar** that exposes the mechanism (see below), then reserve
|
||||||
|
stage lanes and draw. See `references/layout.md` and `references/api.md`.
|
||||||
|
5. **Build and audit.** `./scripts/build.sh fig.tex`. A clean build only proves TeX was
|
||||||
|
happy; then run the visual audit in `references/checklist.md` against the PNG at full
|
||||||
|
size and at thumbnail size. Redraw on any mandatory-invariant violation.
|
||||||
|
|
||||||
|
For code input, trace the concrete `matmul`, `einsum`, `reshape/view`, `transpose/permute`,
|
||||||
|
concat, broadcast, and collective calls. Keep code variable names where useful; state any
|
||||||
|
shape or convention you inferred.
|
||||||
|
|
||||||
|
## Choose the visual grammar
|
||||||
|
|
||||||
|
| Fact to expose | Grammar | Package |
|
||||||
|
|---|---|---|
|
||||||
|
| mask, sparsity, causal structure, elementwise roles | entry cells | `\stface[pattern=causal/lower/band/diag/data]` |
|
||||||
|
| sharding, device ownership, channel groups | adjacent faces tiling a parent | two `\stface` calls, one role each |
|
||||||
|
| leading axes (`B`, `h`) | depth | `\ststack{...}{sheets}` |
|
||||||
|
| discrete choices (indices, token ids, expert ids) | symbols in cells, no ramp | `\stindexface` |
|
||||||
|
| data movement, collectives, non-linear ops | arrows and nodes | `\starrow`, `\starrowlabel`, `st comm` |
|
||||||
|
|
||||||
|
Combine grammars only when each one adds information. Known zeros stay unfilled; masks,
|
||||||
|
diagonals, sparsity and partitions must encode their exact structure.
|
||||||
|
|
||||||
|
## Non-negotiables
|
||||||
|
|
||||||
|
These are the rules that make the figure *true* rather than merely pretty. Each has a
|
||||||
|
reference file with the full statement and the failure it prevents.
|
||||||
|
|
||||||
|
- **Geometry** (`references/geometry.md`) — one symbolic axis, one physical edge length,
|
||||||
|
everywhere. `a×a` is a square. A transpose swaps the face, not the label. Both
|
||||||
|
occurrences of a contracted `k` are the same length. Shards tile their parent exactly.
|
||||||
|
- **Semantics** (`references/semantics.md`) — one block, one semantic object. Scores,
|
||||||
|
indices and masks are three different grammars, never one heatmap. Close the chain
|
||||||
|
from continuous score to discrete index to gathered value.
|
||||||
|
- **Layout** (`references/layout.md`) — everything is a bounding box; tangency counts as
|
||||||
|
collision. Reserved lanes for stage heading / tensors / symbols / shapes. Connectors on
|
||||||
|
the background layer, text on the foreground layer.
|
||||||
|
- **Style** (`references/style.md`) — muted palette, one hue per role, ≤4 active hues per
|
||||||
|
row plus gray, three separated lightness levels, non-periodic texture, no decoration.
|
||||||
|
|
||||||
|
`references/antipatterns.md` shows what each violation looks like in a rendered figure —
|
||||||
|
read it once before your first figure.
|
||||||
|
|
||||||
|
## Output
|
||||||
|
|
||||||
|
Default to editable TikZ. `scripts/build.sh` emits PDF, SVG, white-background PNG,
|
||||||
|
transparent PNG and a thumbnail. Deliver: the PNG preview, a short mechanism explanation,
|
||||||
|
and the `.tex` source plus vector artifact.
|
||||||
|
|
||||||
|
- **Chinese figures:** `\usepackage[cjk]{supertensor}` (XeLaTeX + portable Fandol). Do not
|
||||||
|
select an OS-specific CJK font unless the user asks and accepts the portability cost.
|
||||||
|
- **English figures:** `\usepackage[en]{supertensor}` — same geometry, English rail labels.
|
||||||
|
- Keep math in LaTeX, not raw Unicode.
|
||||||
|
- The identification line is `\stsignature{<subject>}{<box>}`; it renders
|
||||||
|
`<subject>@五道口纳什`. Change the handle with `\stsetauthor{...}` only when asked.
|
||||||
|
|
||||||
|
## Iterating
|
||||||
|
|
||||||
|
When the user asks for a change, do not restart the figure. Edit the ledger or the one
|
||||||
|
call that owns the offending object, rebuild, and re-audit. If a fix requires shrinking
|
||||||
|
type, closing the gutter, or covering another object, the layout is wrong — move the stage
|
||||||
|
to another row instead. Ask before dropping a stage or an object the user named.
|
||||||
Binary file not shown.
|
After Width: | Height: | Size: 31 KiB |
@@ -0,0 +1,339 @@
|
|||||||
|
%% supertensor.sty -- shape-aware tensor figure toolkit
|
||||||
|
%% Geometry invariants are enforced by construction: every face is sized from a
|
||||||
|
%% declared axis length, so equal shapes get equal edges, a x a is a square, and
|
||||||
|
%% a transpose physically swaps width and height.
|
||||||
|
%%
|
||||||
|
%% Options: cjk load ctex with the portable fandol fontset (XeLaTeX)
|
||||||
|
%% en English rail labels in the meaning box (default: zh)
|
||||||
|
\NeedsTeXFormat{LaTeX2e}
|
||||||
|
\ProvidesPackage{supertensor}[2026/08/04 v0.1 shape-aware tensor figure toolkit]
|
||||||
|
|
||||||
|
\newif\ifst@cjk\st@cjkfalse
|
||||||
|
\newif\ifst@en\st@enfalse
|
||||||
|
\DeclareOption{cjk}{\st@cjktrue}
|
||||||
|
\DeclareOption{en}{\st@entrue}
|
||||||
|
\DeclareOption{zh}{\st@enfalse}
|
||||||
|
\ProcessOptions\relax
|
||||||
|
|
||||||
|
\RequirePackage{amsmath}
|
||||||
|
\RequirePackage{amssymb}
|
||||||
|
\RequirePackage{xcolor}
|
||||||
|
\RequirePackage{array}
|
||||||
|
\RequirePackage{etoolbox}
|
||||||
|
\RequirePackage{xstring}
|
||||||
|
\RequirePackage{tikz}
|
||||||
|
\usetikzlibrary{calc,positioning,arrows.meta,backgrounds,fit,shapes.geometric,%
|
||||||
|
decorations.pathreplacing,decorations.markings}
|
||||||
|
|
||||||
|
\ifst@cjk
|
||||||
|
\RequirePackage[UTF8,fontset=fandol]{ctex}
|
||||||
|
\fi
|
||||||
|
|
||||||
|
% ---------------------------------------------------------------- layers ----
|
||||||
|
% Connectors on the background, tensors on main, all text on the foreground.
|
||||||
|
\pgfdeclarelayer{stbg}
|
||||||
|
\pgfdeclarelayer{stfg}
|
||||||
|
\pgfsetlayers{stbg,main,stfg}
|
||||||
|
|
||||||
|
% ---------------------------------------------------------------- palette ---
|
||||||
|
% Muted, mid-chroma paper colors. Do not add saturated primaries.
|
||||||
|
\definecolor{stTeal}{HTML}{4F8FA5}
|
||||||
|
\definecolor{stOrange}{HTML}{EE995B}
|
||||||
|
\definecolor{stCoral}{HTML}{C95B5B}
|
||||||
|
\definecolor{stViolet}{HTML}{8A74B5}
|
||||||
|
\definecolor{stGray}{HTML}{85898F}
|
||||||
|
\definecolor{stInk}{HTML}{1A1A1A}
|
||||||
|
|
||||||
|
% Role registry: draw macros take a ROLE, never a color, so one tensor role
|
||||||
|
% keeps one hue across every stage of the figure.
|
||||||
|
% \stsetrole{X}{stTeal} -> role "X" is teal everywhere
|
||||||
|
\newcommand{\stsetrole}[2]{\expandafter\gdef\csname st@role@#1\endcsname{#2}}
|
||||||
|
% Expandable on purpose: usable inside \edef. Undeclared roles fall back to
|
||||||
|
% neutral gray and are reported at the end of the run.
|
||||||
|
\newcommand{\strole}[1]{%
|
||||||
|
\ifcsname st@role@#1\endcsname\csname st@role@#1\endcsname\else stGray\fi}
|
||||||
|
\newcommand{\stcheckrole}[1]{%
|
||||||
|
\ifcsname st@role@#1\endcsname\else
|
||||||
|
\PackageWarning{supertensor}{Undeclared role `#1' -- drawn in neutral gray.
|
||||||
|
Declare it with \string\stsetrole\space so the hue budget stays visible}%
|
||||||
|
\fi}
|
||||||
|
\stsetrole{neutral}{stGray}
|
||||||
|
|
||||||
|
% Three separated lightness levels. Never render data cells below level 1.
|
||||||
|
\newcommand{\stlevelpct}[1]{\ifcase#1 0\or30\or55\or80\else55\fi}
|
||||||
|
|
||||||
|
% ------------------------------------------------------------ geometry ------
|
||||||
|
% The geometry ledger, made executable. Declare each symbolic axis once:
|
||||||
|
% \stdim{d}{8} \stdim{dh}{4}
|
||||||
|
% then every face built from `d' has the same physical edge, everywhere.
|
||||||
|
\newlength{\stunit}\setlength{\stunit}{4.6mm}
|
||||||
|
\newlength{\sttilegap}\setlength{\sttilegap}{0.5mm}
|
||||||
|
\newcommand{\stdim}[2]{\expandafter\gdef\csname st@dim@#1\endcsname{#2}}
|
||||||
|
\newcommand{\stresolve}[1]{\ifcsname st@dim@#1\endcsname\csname st@dim@#1\endcsname\else#1\fi}
|
||||||
|
|
||||||
|
% --------------------------------------------------------- type hierarchy ---
|
||||||
|
\newcommand{\stformula}[1]{{\large #1}}
|
||||||
|
\newcommand{\ststagelabel}[1]{{\small\bfseries\color{black!55}#1}}
|
||||||
|
\newcommand{\stoperator}[1]{{\Large #1}}
|
||||||
|
\newcommand{\stsymfont}[1]{{\small #1}}
|
||||||
|
\newcommand{\stshapefont}[1]{{\scriptsize\color{black!55}#1}}
|
||||||
|
\newcommand{\stprose}[1]{{\small #1}}
|
||||||
|
|
||||||
|
\tikzset{
|
||||||
|
st sym/.style = {font=\small, text=stInk, inner sep=1pt},
|
||||||
|
st shape/.style = {font=\scriptsize, text=black!55, inner sep=1pt},
|
||||||
|
st stage/.style = {font=\small\bfseries, text=black!55, inner sep=2pt},
|
||||||
|
st op/.style = {font=\Large, text=stInk, inner sep=2pt},
|
||||||
|
st note/.style = {font=\scriptsize, text=black!55, inner sep=2pt},
|
||||||
|
st arrow/.style = {-{Stealth[length=2.2mm,width=1.6mm]}, draw=black!45, line width=0.5pt},
|
||||||
|
st comm/.style = {draw=black!45, fill=black!4, rounded corners=1.2pt,
|
||||||
|
font=\scriptsize, inner sep=3pt},
|
||||||
|
st brace/.style = {decorate, decoration={brace,amplitude=3pt}, draw=black!45,
|
||||||
|
line width=0.4pt},
|
||||||
|
}
|
||||||
|
|
||||||
|
% ------------------------------------------------------------- faces --------
|
||||||
|
\newif\ifst@bracket
|
||||||
|
\newif\ifst@border
|
||||||
|
\newif\ifst@tiles
|
||||||
|
\pgfkeys{
|
||||||
|
/st/face/.cd,
|
||||||
|
role/.store in=\st@role,
|
||||||
|
pattern/.store in=\st@pattern,
|
||||||
|
data/.store in=\st@data,
|
||||||
|
level/.store in=\st@level,
|
||||||
|
bracket/.is if=st@bracket,
|
||||||
|
border/.is if=st@border,
|
||||||
|
tiles/.is if=st@tiles,
|
||||||
|
role=neutral, pattern=dense, data={}, level=2,
|
||||||
|
bracket=false, border=true, tiles=true,
|
||||||
|
}
|
||||||
|
|
||||||
|
% \st@hash{i}{j}{n} -> \st@hv in 0..n-1.
|
||||||
|
% Nested mods on purpose. Any polynomial in (i,j) reduced mod 3 is periodic
|
||||||
|
% with period 3 in BOTH directions, so a polynomial hash makes rows 1,2,4,5 of
|
||||||
|
% a face literally identical -- the eye reads that stripe as structure in the
|
||||||
|
% data. Composing coprime moduli (17, 19) pushes the period past any face we
|
||||||
|
% would ever draw.
|
||||||
|
\newcommand{\st@hash}[3]{%
|
||||||
|
\pgfmathtruncatemacro{\st@hv}%
|
||||||
|
{mod(mod(11*#1+7*#2,17)*mod(5*#1+13*#2,19)+3*#1+#2,#3)}}
|
||||||
|
|
||||||
|
% \st@cellfill{i}{j} -> \st@lv in 0..3 (0 = known zero, left unfilled)
|
||||||
|
\newcommand{\st@cellfill}[2]{%
|
||||||
|
\edef\st@p{\st@pattern}%
|
||||||
|
\IfStrEq{\st@p}{solid}{\def\st@lv{\st@level}}{%
|
||||||
|
\IfStrEq{\st@p}{diag}{\pgfmathtruncatemacro{\st@lv}{ifthenelse(#1==#2,3,0)}}{%
|
||||||
|
\IfStrEq{\st@p}{band}{\pgfmathtruncatemacro{\st@lv}{ifthenelse(abs(#1-#2)<=1,3,0)}}{%
|
||||||
|
\IfStrEq{\st@p}{empty}{\def\st@lv{0}}{%
|
||||||
|
\IfStrEq{\st@p}{lower}{%
|
||||||
|
\ifnum#1<#2 \def\st@lv{0}\else\st@hash{#1}{#2}{2}%
|
||||||
|
\pgfmathtruncatemacro{\st@lv}{\st@hv+2}\fi}{%
|
||||||
|
\IfStrEq{\st@p}{upper}{%
|
||||||
|
\ifnum#1>#2 \def\st@lv{0}\else\st@hash{#1}{#2}{2}%
|
||||||
|
\pgfmathtruncatemacro{\st@lv}{\st@hv+2}\fi}{%
|
||||||
|
\IfStrEq{\st@p}{causal}{%
|
||||||
|
\ifnum#1<#2 \def\st@lv{0}\else\st@hash{#1}{#2}{3}%
|
||||||
|
\pgfmathtruncatemacro{\st@lv}{\st@hv+1}\fi}{%
|
||||||
|
% default, including `dense'
|
||||||
|
\st@hash{#1}{#2}{3}\pgfmathtruncatemacro{\st@lv}{\st@hv+1}}}}}}}}}
|
||||||
|
|
||||||
|
% Shared setup: read keys, resolve the axis names through the geometry ledger.
|
||||||
|
\newcommand{\st@setup}[3]{%
|
||||||
|
\pgfkeys{/st/face/.cd,#1}%
|
||||||
|
\edef\st@rows{\stresolve{#2}}%
|
||||||
|
\edef\st@cols{\stresolve{#3}}%
|
||||||
|
\stcheckrole{\st@role}%
|
||||||
|
\edef\st@col{\strole{\st@role}}%
|
||||||
|
\pgfmathsetlengthmacro{\st@w}{\st@cols*\stunit}%
|
||||||
|
\pgfmathsetlengthmacro{\st@h}{\st@rows*\stunit}}
|
||||||
|
|
||||||
|
% \st@facecore{name}{coord} -- draws one face from the already-resolved state.
|
||||||
|
% \ststack calls this directly rather than re-entering \stface: passing
|
||||||
|
% `role=\st@role' back through pgfkeys would define \st@role in terms of
|
||||||
|
% itself and hang the run.
|
||||||
|
\newcommand{\st@facecore}[2]{%
|
||||||
|
\node[inner sep=0pt, outer sep=0pt, minimum width=\st@w, minimum height=\st@h]
|
||||||
|
(#1) at #2 {};
|
||||||
|
\st@facebody{#1}}
|
||||||
|
|
||||||
|
% \stface[keys]{name}{center coord}{rows}{cols}
|
||||||
|
% rows/cols accept a declared axis name or a raw integer. Height <- rows,
|
||||||
|
% width <- cols, always: a matrix face a x b never renders sideways.
|
||||||
|
\newcommand{\stface}[5][]{%
|
||||||
|
\begingroup
|
||||||
|
\st@setup{#1}{#4}{#5}%
|
||||||
|
\st@facecore{#2}{#3}%
|
||||||
|
\endgroup}
|
||||||
|
|
||||||
|
% \stindexface[keys]{name}{center coord}{rows}{cols}{entries}
|
||||||
|
% An INDEX face: the cells carry discrete symbols, not magnitudes. Entries are
|
||||||
|
% comma-separated in row-major order (rows*cols of them); `.' leaves a cell
|
||||||
|
% blank. Deliberately a different grammar from \stface -- outlined cells, no
|
||||||
|
% lightness ramp -- because an index that is drawn like a score invites the
|
||||||
|
% reader to compare 7 > 2 as if the numbers meant size.
|
||||||
|
\newcommand{\stindexface}[6][]{%
|
||||||
|
\begingroup
|
||||||
|
\st@setup{#1}{#4}{#5}%
|
||||||
|
\node[inner sep=0pt, outer sep=0pt, minimum width=\st@w, minimum height=\st@h]
|
||||||
|
(#2) at #3 {};
|
||||||
|
\foreach \st@e [count=\st@z from 0] in {#6} {%
|
||||||
|
\pgfmathtruncatemacro{\st@ii}{div(\st@z,\st@cols)+1}%
|
||||||
|
\pgfmathtruncatemacro{\st@jj}{mod(\st@z,\st@cols)+1}%
|
||||||
|
\edef\st@ee{\st@e}%
|
||||||
|
\IfStrEq{\st@ee}{.}{}{%
|
||||||
|
\draw[draw=\st@col!70, line width=0.4pt, rounded corners=0.7pt,
|
||||||
|
fill=\st@col!12]
|
||||||
|
($(#2.north west)+(\st@jj*\stunit-\stunit+0.5\sttilegap,%
|
||||||
|
-\st@ii*\stunit+\stunit-0.5\sttilegap)$)
|
||||||
|
rectangle
|
||||||
|
($(#2.north west)+(\st@jj*\stunit-0.5\sttilegap,%
|
||||||
|
-\st@ii*\stunit+0.5\sttilegap)$);
|
||||||
|
\node[font=\tiny, text=\st@col!85!black, inner sep=0pt]
|
||||||
|
at ($(#2.north west)+(\st@jj*\stunit-0.5\stunit,%
|
||||||
|
-\st@ii*\stunit+0.5\stunit)$) {\st@ee};}}%
|
||||||
|
\ifst@border
|
||||||
|
\draw[draw=black!60, line width=0.5pt, rounded corners=1pt]
|
||||||
|
(#2.south west) rectangle (#2.north east);
|
||||||
|
\fi
|
||||||
|
\endgroup}
|
||||||
|
|
||||||
|
\newcommand{\st@facebody}[1]{%
|
||||||
|
\def\st@n{#1}%
|
||||||
|
\ifst@tiles
|
||||||
|
\IfStrEq{\st@pattern}{data}{%
|
||||||
|
\foreach \st@row [count=\st@ii] in \st@data {%
|
||||||
|
\foreach \st@jj in {1,...,\st@cols} {%
|
||||||
|
\StrChar{\st@row}{\st@jj}[\st@c]%
|
||||||
|
\ifnum\st@c>0
|
||||||
|
\st@tile{#1}{\st@ii}{\st@jj}{\st@c}%
|
||||||
|
\fi}}%
|
||||||
|
}{%
|
||||||
|
\foreach \st@ii in {1,...,\st@rows} {%
|
||||||
|
\foreach \st@jj in {1,...,\st@cols} {%
|
||||||
|
\st@cellfill{\st@ii}{\st@jj}%
|
||||||
|
\ifnum\st@lv>0
|
||||||
|
\st@tile{#1}{\st@ii}{\st@jj}{\st@lv}%
|
||||||
|
\fi}}%
|
||||||
|
}%
|
||||||
|
\else
|
||||||
|
\fill[\st@col!\stlevelpct{\st@level}, rounded corners=1pt]
|
||||||
|
(#1.south west) rectangle (#1.north east);
|
||||||
|
\fi
|
||||||
|
\ifst@border
|
||||||
|
\draw[draw=black!60, line width=0.5pt, rounded corners=1pt]
|
||||||
|
(#1.south west) rectangle (#1.north east);
|
||||||
|
\fi
|
||||||
|
\ifst@bracket
|
||||||
|
\draw[black!55, line width=0.5pt]
|
||||||
|
($(#1.north west)+(-1.1mm,0.6mm)$) -- ++(-1.1mm,0)
|
||||||
|
-- ($(#1.south west)+(-2.2mm,-0.6mm)$) -- ++(1.1mm,0);
|
||||||
|
\draw[black!55, line width=0.5pt]
|
||||||
|
($(#1.north east)+(1.1mm,0.6mm)$) -- ++(1.1mm,0)
|
||||||
|
-- ($(#1.south east)+(2.2mm,-0.6mm)$) -- ++(-1.1mm,0);
|
||||||
|
\fi}
|
||||||
|
|
||||||
|
% \st@tile{face}{row}{col}{level} -- one rounded tile with a white gutter.
|
||||||
|
\newcommand{\st@tile}[4]{%
|
||||||
|
\fill[\st@col!\stlevelpct{#4}, rounded corners=0.7pt]
|
||||||
|
($(#1.north west)+(#3*\stunit-\stunit+0.5\sttilegap,%
|
||||||
|
-#2*\stunit+\stunit-0.5\sttilegap)$)
|
||||||
|
rectangle
|
||||||
|
($(#1.north west)+(#3*\stunit-0.5\sttilegap,%
|
||||||
|
-#2*\stunit+0.5\sttilegap)$);}
|
||||||
|
|
||||||
|
% \ststack[keys]{name}{center}{rows}{cols}{sheets}
|
||||||
|
% Leading axes become depth. Back sheets are outline-only and are included in
|
||||||
|
% the bounding box, so neighbours can be spaced against the real extent.
|
||||||
|
\newcommand{\ststack}[6][]{%
|
||||||
|
\begingroup
|
||||||
|
\st@setup{#1}{#4}{#5}%
|
||||||
|
\pgfmathtruncatemacro{\st@back}{#6-1}%
|
||||||
|
\pgfmathsetlengthmacro{\st@dx}{1.3mm}%
|
||||||
|
\coordinate (#2-c) at ($#3+(-0.5*\st@back*\st@dx,-0.5*\st@back*\st@dx)$);
|
||||||
|
\node[inner sep=0pt, outer sep=0pt, minimum width=\st@w, minimum height=\st@h]
|
||||||
|
(#2-front) at (#2-c) {};
|
||||||
|
\ifnum\st@back>0
|
||||||
|
% Ascending loop, descending index: `{\macro,...,1}' cannot infer its
|
||||||
|
% direction from an unexpanded macro and runs away.
|
||||||
|
\foreach \st@kk in {1,...,\st@back} {%
|
||||||
|
\pgfmathtruncatemacro{\st@k}{\st@back+1-\st@kk}%
|
||||||
|
\draw[draw=black!35, line width=0.4pt, rounded corners=1pt, fill=white]
|
||||||
|
($(#2-front.south west)+(\st@k*\st@dx,\st@k*\st@dx)$)
|
||||||
|
rectangle
|
||||||
|
($(#2-front.north east)+(\st@k*\st@dx,\st@k*\st@dx)$);}
|
||||||
|
\fi
|
||||||
|
\st@facebody{#2-front}%
|
||||||
|
% Braces are load-bearing: a bare coordinate expression contains a comma and
|
||||||
|
% would be split into two pgfkeys keys.
|
||||||
|
\node[inner sep=0pt, outer sep=0pt,
|
||||||
|
fit={(#2-front) ($(#2-front.north east)+(\st@back*\st@dx,\st@back*\st@dx)$)}]
|
||||||
|
(#2) {};
|
||||||
|
\endgroup}
|
||||||
|
|
||||||
|
% ------------------------------------------------- symbol / shape captions ---
|
||||||
|
% Symbol immediately under the block, shape on the next line. Both are reserved
|
||||||
|
% lanes: nothing else may be placed between a face and its caption.
|
||||||
|
% \stlane{node} declares a shared caption baseline: every later \stcaption
|
||||||
|
% hangs from the bottom of that node instead of from its own block, so symbols
|
||||||
|
% and shapes occupy two flat lanes even when the faces have different heights.
|
||||||
|
% Typical use: draw the row, \node[fit=(A)(B)(C)] (row) {};, \stlane{row}.
|
||||||
|
\def\st@lane{}
|
||||||
|
\newcommand{\stlane}[1]{\def\st@lane{#1}}
|
||||||
|
\newcommand{\stnolane}{\def\st@lane{}}
|
||||||
|
\newcommand{\stcaption}[3]{%
|
||||||
|
\ifdefempty{\st@lane}%
|
||||||
|
{\node[st sym, below=1.6mm of #1] (#1-sym) {#2};}%
|
||||||
|
{\node[st sym, anchor=north]
|
||||||
|
at ($(#1.center |- \st@lane.south)+(0,-1.6mm)$) (#1-sym) {#2};}%
|
||||||
|
\node[st shape, below=0.6mm of #1-sym] (#1-shape) {#3};}
|
||||||
|
\newcommand{\stcaptiontop}[2]{%
|
||||||
|
\node[st sym, above=1.6mm of #1] (#1-top) {#2};}
|
||||||
|
|
||||||
|
% ------------------------------------------------------------ connectors ----
|
||||||
|
% Routed on the background layer so a connector can never cover a face.
|
||||||
|
\newcommand{\starrow}[3][]{%
|
||||||
|
\begin{pgfonlayer}{stbg}
|
||||||
|
\draw[st arrow,#1] (#2) -- (#3);
|
||||||
|
\end{pgfonlayer}}
|
||||||
|
\newcommand{\starrowlabel}[4][]{%
|
||||||
|
\begin{pgfonlayer}{stbg}
|
||||||
|
\draw[st arrow,#1] (#2) -- node[st note, above, fill=white, inner sep=1pt] {#4} (#3);
|
||||||
|
\end{pgfonlayer}}
|
||||||
|
|
||||||
|
% ------------------------------------------------------------ meaning box ---
|
||||||
|
\ifst@en
|
||||||
|
\def\st@lblaxes{Axes}\def\st@lblobj{Objects}\def\st@lblmech{Mechanism}
|
||||||
|
\else
|
||||||
|
\def\st@lblaxes{轴}\def\st@lblobj{对象}\def\st@lblmech{机制}
|
||||||
|
\fi
|
||||||
|
\newcommand{\stsetlabels}[3]{\def\st@lblaxes{#1}\def\st@lblobj{#2}\def\st@lblmech{#3}}
|
||||||
|
|
||||||
|
% \stmeaningbox{node name}{total width}{below-of anchor}{axes}{objects}{mechanism}
|
||||||
|
% One full-width, low-contrast box, one reading column, at most three rows.
|
||||||
|
% Leave a row empty to drop it.
|
||||||
|
\newcommand{\stmeaningbox}[6]{%
|
||||||
|
\node[below=4mm of #3, anchor=north,
|
||||||
|
draw=black!18, fill=black!3, rounded corners=1.5pt,
|
||||||
|
inner xsep=9pt, inner ysep=8pt, text width=#2] (#1) {%
|
||||||
|
\small
|
||||||
|
\setlength{\tabcolsep}{0pt}%
|
||||||
|
\renewcommand{\arraystretch}{1.25}%
|
||||||
|
\begin{tabular}{@{}p{\st@raillen}@{\hspace{7pt}}p{\dimexpr#2-\st@raillen-7pt\relax}@{}}
|
||||||
|
% \ifblank, not \IfStrEq: the row content is math and CJK, and must not
|
||||||
|
% be expanded just to test for emptiness.
|
||||||
|
\ifblank{#4}{}{\textbf{\st@lblaxes} & \raggedright\arraybackslash #4 \\}
|
||||||
|
\ifblank{#5}{}{\textbf{\st@lblobj} & \raggedright\arraybackslash #5 \\}
|
||||||
|
\ifblank{#6}{}{\textbf{\st@lblmech} & \raggedright\arraybackslash #6 \\}
|
||||||
|
\end{tabular}};}
|
||||||
|
\newlength{\st@raillen}\setlength{\st@raillen}{3.2em}
|
||||||
|
\newcommand{\stsetrail}[1]{\setlength{\st@raillen}{#1}}
|
||||||
|
|
||||||
|
% ------------------------------------------------------------- signature ----
|
||||||
|
% One centered identification line, outside the meaning box, low contrast.
|
||||||
|
\def\st@author{五道口纳什}
|
||||||
|
\newcommand{\stsetauthor}[1]{\def\st@author{#1}}
|
||||||
|
\newcommand{\stsignature}[2]{%
|
||||||
|
\node[below=2.2mm of #2, font=\scriptsize, text=black!45] (st-signature) {#1@\st@author};}
|
||||||
|
|
||||||
|
\endinput
|
||||||
@@ -0,0 +1,84 @@
|
|||||||
|
% Anti-pattern gallery -- four ways to draw a figure that compiles cleanly and
|
||||||
|
% still teaches the reader something false. Left of each pair is wrong.
|
||||||
|
% ../scripts/build.sh antipatterns.tex
|
||||||
|
\documentclass[border=10pt]{standalone}
|
||||||
|
\usepackage[cjk]{supertensor}
|
||||||
|
|
||||||
|
\stsetrole{k}{stOrange}
|
||||||
|
\stsetrole{r1}{stTeal}
|
||||||
|
\stsetrole{r2}{stCoral}
|
||||||
|
\stsetrole{idx}{stViolet}
|
||||||
|
\stsetrole{act}{stTeal}
|
||||||
|
|
||||||
|
\stdim{T}{5}
|
||||||
|
\stdim{dh}{3}
|
||||||
|
\stdim{d}{6}
|
||||||
|
\stdim{k}{2}
|
||||||
|
|
||||||
|
\newcommand{\bad}[1]{{\color{stCoral}$\times$}\;#1}
|
||||||
|
\newcommand{\good}[1]{{\color{stTeal}$\checkmark$}\;#1}
|
||||||
|
|
||||||
|
\begin{document}
|
||||||
|
\begin{tikzpicture}
|
||||||
|
|
||||||
|
% ------------------------------------------------------------- pair 1 + 2 ---
|
||||||
|
\node[st stage, anchor=north west] (S1) at (0,0) {(1) 转置只改了标签};
|
||||||
|
\coordinate (p1) at ($(S1.west)+(1.0,-1.7)$);
|
||||||
|
|
||||||
|
% WRONG: same face as K, relabelled. The reader cannot see the contracted axis.
|
||||||
|
\stface[role=k, bracket=true]{A1}{(p1)}{T}{dh}
|
||||||
|
\stface[role=k, bracket=true]{A2}{($(A1.east)+(2.2,0)$)}{dh}{T}
|
||||||
|
|
||||||
|
\node[st stage, anchor=north west] (S2) at ($(S1.west)+(7.6,0)$) {(2) 分片没有铺满母体};
|
||||||
|
\coordinate (p2) at ($(S2.west)+(1.0,-1.7)$);
|
||||||
|
|
||||||
|
% WRONG: a gap between the shards. The parent's width is now a lie.
|
||||||
|
\stface[role=r1]{B1a}{(p2)}{T}{dh}
|
||||||
|
\stface[role=r2]{B1b}{($(B1a.east)+(2.6*\stunit,0)$)}{T}{dh}
|
||||||
|
\stface[role=r1]{B2a}{($(B1b.east)+(2.4,0)$)}{T}{dh}
|
||||||
|
\stface[role=r2]{B2b}{($(B2a.east)+(1.5*\stunit,0)$)}{T}{dh}
|
||||||
|
|
||||||
|
\node[inner sep=0pt, fit=(A1)(A2)(B1a)(B2b)] (row1) {};
|
||||||
|
\stlane{row1}
|
||||||
|
\stcaption{A1}{\bad{$\mathbf K^{\top}$}}{面仍是 $T\times d_h$}
|
||||||
|
\stcaption{A2}{\good{$\mathbf K^{\top}$}}{面已换成 $d_h\times T$}
|
||||||
|
\stcaption{B1a}{\bad{$[\mathbf W^{(1)}\mid\mathbf W^{(2)}]$}}{中间凭空多出空隙}
|
||||||
|
\stcaption{B2a}{\good{$[\mathbf W^{(1)}\mid\mathbf W^{(2)}]$}}{两片正好铺满}
|
||||||
|
\stnolane
|
||||||
|
|
||||||
|
% ------------------------------------------------------------- pair 3 + 4 ---
|
||||||
|
\coordinate (y2) at ($(A1-shape.south)+(0,-10mm)$);
|
||||||
|
\node[st stage, anchor=north west] (S3) at (S1.west |- y2) {(3) 索引画成了热力图};
|
||||||
|
\coordinate (p3) at ($(S3.west)+(1.0,-1.6)$);
|
||||||
|
|
||||||
|
% WRONG: a lightness ramp on an index invites `expert 3 > expert 0'.
|
||||||
|
\stface[role=idx]{C1}{(p3)}{T}{k}
|
||||||
|
\stindexface[role=idx]{C2}{($(C1.east)+(2.4,0)$)}{T}{k}{0,1, 1,2, 2,3, 3,0, 0,2}
|
||||||
|
|
||||||
|
\node[st stage, anchor=north west] (S4) at (S2.west |- y2) {(4) 全图一个色阶};
|
||||||
|
\coordinate (p4) at ($(S4.west)+(1.0,-1.6)$);
|
||||||
|
|
||||||
|
% WRONG: one pale level everywhere -- nothing is legible at thumbnail size.
|
||||||
|
\stface[role=act, pattern=solid, level=1]{D1}{(p4)}{T}{d}
|
||||||
|
\stface[role=act]{D2}{($(D1.east)+(2.6,0)$)}{T}{d}
|
||||||
|
|
||||||
|
\node[inner sep=0pt, fit=(C1)(C2)(D1)(D2)] (row2) {};
|
||||||
|
\stlane{row2}
|
||||||
|
\stcaption{C1}{\bad{$\mathcal I$}}{深浅暗示大小可比}
|
||||||
|
\stcaption{C2}{\good{$\mathcal I$}}{离散符号，无色阶}
|
||||||
|
\stcaption{D1}{\bad{$\mathbf X$}}{只有一档淡色}
|
||||||
|
\stcaption{D2}{\good{$\mathbf X$}}{三档亮度，非周期}
|
||||||
|
\stnolane
|
||||||
|
|
||||||
|
% ============================================================== meaning box ==
|
||||||
|
\node[inner sep=0pt, fit=(S1)(row1)(row2)(D2-shape)(B2a-shape)] (all) {};
|
||||||
|
\stmeaningbox{mb}{16.2cm}{all}
|
||||||
|
{}
|
||||||
|
{四张“错图”都能干净编译。编译器只检查 \TeX{} 的语法，不检查图讲的事情对不对，
|
||||||
|
所以交付前必须按 \texttt{references/checklist.md} 做一次人眼审图}
|
||||||
|
{(1) 转置要物理交换面宽高；(2) 分片必须精确铺满母体，省略要画省略号；
|
||||||
|
(3) 索引、掩码、数值是三套画法；(4) 亮度分档是对比度的来源，不是饱和度}
|
||||||
|
\stsignature{反面示例：编译通过但讲错的四种画法}{mb}
|
||||||
|
|
||||||
|
\end{tikzpicture}
|
||||||
|
\end{document}
|
||||||
@@ -0,0 +1,105 @@
|
|||||||
|
% Golden example 2 -- causal multi-head attention.
|
||||||
|
% Shows: leading axes as stack depth, a transpose that physically swaps the
|
||||||
|
% face, equal edge length on the contracted axis, and a Boolean mask drawn in
|
||||||
|
% a different grammar from the scores it gates.
|
||||||
|
% ../scripts/build.sh mha-causal.tex
|
||||||
|
\documentclass[border=10pt]{standalone}
|
||||||
|
\usepackage[cjk]{supertensor}
|
||||||
|
|
||||||
|
\stsetrole{q}{stTeal}
|
||||||
|
\stsetrole{k}{stOrange}
|
||||||
|
\stsetrole{v}{stViolet}
|
||||||
|
\stsetrole{s}{stCoral}
|
||||||
|
\stsetrole{w}{stGray}
|
||||||
|
|
||||||
|
\stdim{T}{6}
|
||||||
|
\stdim{dh}{3}
|
||||||
|
\stdim{d}{9} % d = h * d_h, h = 3
|
||||||
|
|
||||||
|
\begin{document}
|
||||||
|
\begin{tikzpicture}
|
||||||
|
|
||||||
|
% ================================================================= formula ==
|
||||||
|
\node (F) at (0,0) {\stformula{$\displaystyle
|
||||||
|
\mathbf A^{(i)}=\mathrm{softmax}\!\left(
|
||||||
|
\frac{\mathbf Q^{(i)}\mathbf K^{(i)\top}}{\sqrt{d_h}}+\mathbf M\right),\qquad
|
||||||
|
\mathbf O^{(i)}=\mathbf A^{(i)}\mathbf V^{(i)}$}};
|
||||||
|
|
||||||
|
% ============================================================ stage A row ===
|
||||||
|
\node[st stage, below=7mm of F] (SA) {每头打分：沿 $d_h$ 收缩};
|
||||||
|
\coordinate (a) at ($(SA)+(-3.9,-1.9)$);
|
||||||
|
|
||||||
|
\ststack[role=q, bracket=true]{Q}{(a)}{T}{dh}{3}
|
||||||
|
\node[st op, right=6mm of Q] (mA) {$\times$};
|
||||||
|
% K^T: the face is physically swapped, not relabelled. Its height equals Q's
|
||||||
|
% width -- that is the contracted axis d_h, drawn at one edge length.
|
||||||
|
\ststack[role=k, bracket=true]{KT}{($(mA)+(1.9,0)$)}{dh}{T}{3}
|
||||||
|
\node[st op, right=6mm of KT] (eA) {$=$};
|
||||||
|
\ststack[role=s]{S}{($(eA)+(2.0,0)$)}{T}{T}{3}
|
||||||
|
|
||||||
|
\node[inner sep=0pt, fit=(Q)(KT)(S)] (rowA) {};
|
||||||
|
\stlane{rowA}
|
||||||
|
\stcaption{Q}{$\mathbf Q^{(i)}$}{$h\times T\times d_h$}
|
||||||
|
\stcaption{KT}{$\mathbf K^{(i)\top}$}{$h\times d_h\times T$}
|
||||||
|
\stcaption{S}{$\mathbf S^{(i)}$}{$h\times T\times T$}
|
||||||
|
\stnolane
|
||||||
|
|
||||||
|
% ============================================================ stage B row ===
|
||||||
|
\node[st stage, below=9mm of Q-shape.south west, anchor=north west] (SB)
|
||||||
|
{因果掩码与加权求和};
|
||||||
|
\coordinate (b) at ($(SB)+(0.6,-2.0)$);
|
||||||
|
|
||||||
|
% The mask is a Boolean support, not a magnitude: one flat level, exact
|
||||||
|
% triangle, no stack -- it is shared by every head.
|
||||||
|
\stface[role=w, pattern=data,
|
||||||
|
data={300000,330000,333000,333300,333330,333333}]{M}{(b)}{T}{T}
|
||||||
|
\ststack[role=s, pattern=causal]{A}{($(M.east)+(3.75,0)$)}{T}{T}{3}
|
||||||
|
\starrowlabel{M.east}{A.west}{softmax}
|
||||||
|
\node[st op, right=6mm of A] (mB) {$\times$};
|
||||||
|
\ststack[role=v, bracket=true]{V}{($(mB)+(1.4,0)$)}{T}{dh}{3}
|
||||||
|
\node[st op, right=6mm of V] (eB) {$=$};
|
||||||
|
\ststack[role=v]{O}{($(eB)+(1.4,0)$)}{T}{dh}{3}
|
||||||
|
|
||||||
|
\node[inner sep=0pt, fit=(M)(A)(V)(O)] (rowB) {};
|
||||||
|
\stlane{rowB}
|
||||||
|
\stcaption{M}{$\mathbf M$}{$T\times T$}
|
||||||
|
\stcaption{A}{$\mathbf A^{(i)}$}{$h\times T\times T$}
|
||||||
|
\stcaption{V}{$\mathbf V^{(i)}$}{$h\times T\times d_h$}
|
||||||
|
\stcaption{O}{$\mathbf O^{(i)}$}{$h\times T\times d_h$}
|
||||||
|
\stnolane
|
||||||
|
|
||||||
|
% ============================================================ stage C row ===
|
||||||
|
\node[st stage, below=9mm of M-shape.south west, anchor=north west] (SC)
|
||||||
|
{沿 $d_h$ 拼接后投影};
|
||||||
|
\coordinate (c) at ($(SC)+(1.2,-2.0)$);
|
||||||
|
|
||||||
|
% Concatenation reverses the split: three h-shards of width d_h tile a face of
|
||||||
|
% width d exactly.
|
||||||
|
\stface[role=v]{C1}{(c)}{T}{dh}
|
||||||
|
\stface[role=v]{C2}{($(C1.east)+(1.5*\stunit,0)$)}{T}{dh}
|
||||||
|
\stface[role=v]{C3}{($(C2.east)+(1.5*\stunit,0)$)}{T}{dh}
|
||||||
|
\node[st op, right=6mm of C3] (mC) {$\times$};
|
||||||
|
\stface[role=w]{WO}{($(mC)+(2.5,0)$)}{d}{d}
|
||||||
|
\node[st op, right=6mm of WO] (eC) {$=$};
|
||||||
|
\stface[role=v, bracket=true]{Y}{($(eC)+(2.5,0)$)}{T}{d}
|
||||||
|
|
||||||
|
\node[inner sep=0pt, fit=(C1)(WO)(Y)] (rowC) {};
|
||||||
|
\stlane{rowC}
|
||||||
|
\stcaption{C2}{$[\,\mathbf O^{(1)}\mid\mathbf O^{(2)}\mid\mathbf O^{(3)}\,]$}{$T\times d$}
|
||||||
|
\stcaption{WO}{$\mathbf W_O$}{$d\times d$}
|
||||||
|
\stcaption{Y}{$\mathbf Y$}{$T\times d$}
|
||||||
|
\stnolane
|
||||||
|
|
||||||
|
% ============================================================== meaning box ==
|
||||||
|
\node[inner sep=0pt, fit=(F)(rowA)(rowB)(rowC)(Y-shape)(C2-shape)] (all) {};
|
||||||
|
\stmeaningbox{mb}{16.8cm}{all}
|
||||||
|
{$T$ 序列长度，$d_h$ 单头宽度，$h$ 头数（图中 $h=3$，即堆叠的三张面），
|
||||||
|
$d=h\,d_h$；批轴 $B$ 省略}
|
||||||
|
{$\mathbf S,\mathbf A$ 是分数与概率（行和为 $1$）；$\mathbf M\in\{0,-\infty\}^{T\times T}$
|
||||||
|
是布尔支撑而非数值，被所有头共享，故只画一张、不堆叠；紫色一族标记 $\mathbf V\rightarrow\mathbf O\rightarrow\mathbf Y$ 同一数据流}
|
||||||
|
{$(T\times d_h)(d_h\times T)\rightarrow(T\times T)$：$\mathbf K^{\top}$ 的面高即收缩维 $d_h$；
|
||||||
|
拼接是切分的逆运算，$3$ 个 $d_h$ 恰好铺满 $d$}
|
||||||
|
\stsignature{因果多头注意力（掩码 + 拼接投影）}{mb}
|
||||||
|
|
||||||
|
\end{tikzpicture}
|
||||||
|
\end{document}
|
||||||
@@ -0,0 +1,114 @@
|
|||||||
|
% Golden example 3 -- MoE top-k routing and gather.
|
||||||
|
% Shows the three cell semantics side by side in one figure: a SCORE face
|
||||||
|
% (graded lightness), an INDEX face (discrete symbols, no ramp), and a BOOLEAN
|
||||||
|
% support face (one flat level) -- plus a gather whose output heights are data
|
||||||
|
% dependent and must sum back to T*k.
|
||||||
|
% ../scripts/build.sh moe-topk-gather.tex
|
||||||
|
\documentclass[border=10pt]{standalone}
|
||||||
|
\usepackage[cjk]{supertensor}
|
||||||
|
|
||||||
|
\stsetrole{act}{stTeal} % X and the per-expert buffers: same data, regrouped
|
||||||
|
\stsetrole{wg}{stViolet} % learned router weight
|
||||||
|
\stsetrole{s}{stCoral} % scores
|
||||||
|
\stsetrole{idx}{stOrange} % indices / token ids
|
||||||
|
\stsetrole{m}{stGray} % boolean support
|
||||||
|
|
||||||
|
\stdim{T}{6}
|
||||||
|
\stdim{d}{4}
|
||||||
|
\stdim{E}{4}
|
||||||
|
\stdim{k}{2}
|
||||||
|
\stdim{ne}{3} % capacity per expert in this instance: n_e = 3
|
||||||
|
|
||||||
|
\begin{document}
|
||||||
|
\begin{tikzpicture}
|
||||||
|
|
||||||
|
% ================================================================= formula ==
|
||||||
|
\node (F) at (0,0) {\stformula{$\displaystyle
|
||||||
|
\mathbf G=\mathrm{softmax}(\mathbf{XW}_g),\quad
|
||||||
|
\mathcal I_t=\operatorname*{top-}k_{e}\,\mathbf G_{t,e},\quad
|
||||||
|
\mathbf D_{t,e}=\mathbf 1[e\in\mathcal I_t],\quad
|
||||||
|
\mathbf X^{(e)}=\mathrm{gather}(\mathbf X,\mathbf D_{:,e})$}};
|
||||||
|
|
||||||
|
% ============================================================ stage A row ===
|
||||||
|
\node[st stage, below=7mm of F] (SA) {打分：token 对专家};
|
||||||
|
\coordinate (a) at ($(SA)+(-3.6,-1.8)$);
|
||||||
|
|
||||||
|
\stface[role=act, bracket=true]{X}{(a)}{T}{d}
|
||||||
|
\node[st op, right=6mm of X] (mA) {$\times$};
|
||||||
|
\stface[role=wg]{Wg}{($(mA)+(1.6,0)$)}{d}{E}
|
||||||
|
\node[st op, right=6mm of Wg] (eA) {$=$};
|
||||||
|
\stface[role=s]{G}{($(eA)+(1.6,0)$)}{T}{E}
|
||||||
|
|
||||||
|
\node[inner sep=0pt, fit=(X)(Wg)(G)] (rowA) {};
|
||||||
|
\stlane{rowA}
|
||||||
|
\stcaption{X}{$\mathbf X$}{$T\times d$}
|
||||||
|
\stcaption{Wg}{$\mathbf W_g$}{$d\times E$}
|
||||||
|
\stcaption{G}{$\mathbf G$}{$T\times E$}
|
||||||
|
\stnolane
|
||||||
|
|
||||||
|
% ============================================================ stage B row ===
|
||||||
|
\node[st stage, below=9mm of X-shape.south west, anchor=north west] (SB)
|
||||||
|
{取前 $k$：连续分数 $\rightarrow$ 离散选择};
|
||||||
|
\coordinate (b) at ($(SB.west)+(0.6,-1.7)$);
|
||||||
|
|
||||||
|
% Indices are drawn as symbols, not as magnitudes: expert 3 is not "bigger"
|
||||||
|
% than expert 0, so the index face gets no lightness ramp.
|
||||||
|
\stindexface[role=idx]{I}{(b)}{T}{k}{0,1, 1,2, 2,3, 3,0, 0,2, 1,3}
|
||||||
|
% The same routing decision as a boolean support: one flat level, exactly k
|
||||||
|
% cells per row, and every unselected cell left unfilled.
|
||||||
|
\stface[role=m, pattern=data, level=3,
|
||||||
|
data={3300,0330,0033,3003,3030,0303}]{D}{($(I.east)+(3.1,0)$)}{T}{E}
|
||||||
|
\starrowlabel{I.east}{D.west}{one-hot}
|
||||||
|
|
||||||
|
\node[st comm, right=9mm of D] (gz) {Gather};
|
||||||
|
\starrow{D.east}{gz.west}
|
||||||
|
|
||||||
|
\node[inner sep=0pt, fit=(I)(D)(gz)] (rowB) {};
|
||||||
|
\stlane{rowB}
|
||||||
|
\stcaption{I}{$\mathcal I$}{$T\times k$}
|
||||||
|
\stcaption{D}{$\mathbf D$}{$T\times E$}
|
||||||
|
\stnolane
|
||||||
|
|
||||||
|
% ============================================================ stage C row ===
|
||||||
|
% Every stage heading starts on the same left rail; only the vertical position
|
||||||
|
% follows the previous row.
|
||||||
|
\coordinate (cy) at ($(I-shape.south)+(0,-9mm)$);
|
||||||
|
\node[st stage, anchor=north west] (SC) at (SB.west |- cy)
|
||||||
|
{按专家聚合：每个缓冲区的高度是数据决定的};
|
||||||
|
\coordinate (c) at ($(SC.west)+(0.5,-1.9)$);
|
||||||
|
|
||||||
|
% Each buffer keeps X's width d -- gather regroups rows, it never reshapes the
|
||||||
|
% feature axis. The heights are n_e, and they must sum to T*k.
|
||||||
|
\stindexface[role=idx, border=false]{t0}{(c)}{ne}{1}{1,4,5}
|
||||||
|
\stface[role=act]{B0}{($(t0.east)+(2*\stunit,0)$)}{ne}{d}
|
||||||
|
\stindexface[role=idx, border=false]{t1}{($(B0.east)+(1.1,0)$)}{ne}{1}{1,2,6}
|
||||||
|
\stface[role=act]{B1}{($(t1.east)+(2*\stunit,0)$)}{ne}{d}
|
||||||
|
\stindexface[role=idx, border=false]{t2}{($(B1.east)+(1.1,0)$)}{ne}{1}{2,3,5}
|
||||||
|
\stface[role=act]{B2}{($(t2.east)+(2*\stunit,0)$)}{ne}{d}
|
||||||
|
\stindexface[role=idx, border=false]{t3}{($(B2.east)+(1.1,0)$)}{ne}{1}{3,4,6}
|
||||||
|
\stface[role=act]{B3}{($(t3.east)+(2*\stunit,0)$)}{ne}{d}
|
||||||
|
|
||||||
|
\stcaptiontop{t0}{\stshapefont{token}}
|
||||||
|
|
||||||
|
\node[inner sep=0pt, fit=(t0)(B3)] (rowC) {};
|
||||||
|
\stlane{rowC}
|
||||||
|
\stcaption{B0}{$\mathbf X^{(1)}$}{$n_1\times d$}
|
||||||
|
\stcaption{B1}{$\mathbf X^{(2)}$}{$n_2\times d$}
|
||||||
|
\stcaption{B2}{$\mathbf X^{(3)}$}{$n_3\times d$}
|
||||||
|
\stcaption{B3}{$\mathbf X^{(4)}$}{$n_4\times d$}
|
||||||
|
\stnolane
|
||||||
|
|
||||||
|
% ============================================================== meaning box ==
|
||||||
|
\node[inner sep=0pt, fit=(F)(rowA)(rowB)(rowC)(B3-shape)(t0-top)] (all) {};
|
||||||
|
\stmeaningbox{mb}{16.6cm}{all}
|
||||||
|
{$T$ token 数，$d$ 模型宽度，$E$ 专家数（图中 $E=4$），$k$ 每 token 选中的专家数
|
||||||
|
（图中 $k=2$），$n_e$ 落到第 $e$ 个专家的 token 数}
|
||||||
|
{$\mathbf G$ 是分数，深浅可比大小；$\mathcal I$ 是索引，格内是符号不是数值，
|
||||||
|
故不用深浅；$\mathbf D$ 是布尔支撑，只有一档灰、每行恰好 $k$ 格；
|
||||||
|
$\mathbf X^{(e)}$ 与 $\mathbf X$ 同色，因为它是同一批数据换了分组}
|
||||||
|
{top-$k$ 把连续分数截成离散选择，这一步不可微；gather 只重排行、不动特征轴，
|
||||||
|
故每个缓冲区仍是 $d$ 宽；$\sum_e n_e=Tk$，图中 $4\times3=6\times2$}
|
||||||
|
\stsignature{MoE 路由：top-$k$ 选择与按专家 gather}{mb}
|
||||||
|
|
||||||
|
\end{tikzpicture}
|
||||||
|
\end{document}
|
||||||
@@ -0,0 +1,95 @@
|
|||||||
|
% Golden example 1 -- tensor parallel FFN, column-then-row sharding + AllReduce.
|
||||||
|
% Shows: partition geometry (shards tile the parent exactly), one hue per TP
|
||||||
|
% rank held across every stage, a collective node as a real operation.
|
||||||
|
% ../scripts/build.sh tp-ffn-allreduce.tex
|
||||||
|
\documentclass[border=10pt]{standalone}
|
||||||
|
\usepackage[cjk]{supertensor}
|
||||||
|
|
||||||
|
% --- role ledger: one hue per TP rank, held from W through H to P ----------
|
||||||
|
\stsetrole{act}{stViolet} % activations that every rank sees
|
||||||
|
\stsetrole{r1}{stTeal} % rank 1
|
||||||
|
\stsetrole{r2}{stOrange} % rank 2
|
||||||
|
|
||||||
|
% --- geometry ledger: one physical edge per symbolic axis ------------------
|
||||||
|
\stdim{bt}{6} % B*T rows
|
||||||
|
\stdim{d}{4} % model width
|
||||||
|
\stdim{dffl}{4} % d_ff / p (per-rank hidden width)
|
||||||
|
|
||||||
|
\begin{document}
|
||||||
|
\begin{tikzpicture}
|
||||||
|
|
||||||
|
% ================================================================= formula ==
|
||||||
|
\node (F) at (0,0) {\stformula{$\mathbf{XW}_1=[\,\mathbf{XW}_1^{(1)}\mid
|
||||||
|
\mathbf{XW}_1^{(2)}\,]=[\,\mathbf H^{(1)}\mid\mathbf H^{(2)}\,]$}};
|
||||||
|
\node[below=1.2mm of F] (F2) {\stformula{$\displaystyle
|
||||||
|
[\,\mathbf H^{(1)}\mid\mathbf H^{(2)}\,]
|
||||||
|
\begin{bmatrix}\mathbf W_2^{(1)}\\[-1pt]\mathbf W_2^{(2)}\end{bmatrix}
|
||||||
|
=\sum_{r}\mathbf H^{(r)}\mathbf W_2^{(r)}=\sum_r\mathbf P^{(r)}$}};
|
||||||
|
|
||||||
|
% ============================================================ stage A row ===
|
||||||
|
\node[st stage, below=7mm of F2] (SA) {列切 $\mathbf W_1$：无通信};
|
||||||
|
\coordinate (a) at ($(SA)+(-5.6,-1.8)$);
|
||||||
|
|
||||||
|
\stface[role=act, bracket=true]{X}{(a)}{bt}{d}
|
||||||
|
\node[st op, right=5mm of X] (mA) {$\times$};
|
||||||
|
% Two shards, tiled exactly: adjacent faces, no stretching, no gap.
|
||||||
|
\stface[role=r1]{W1a}{($(mA)+(1.5,0)$)}{d}{dffl}
|
||||||
|
\stface[role=r2]{W1b}{($(W1a.east)+(2*\stunit,0)$)}{d}{dffl}
|
||||||
|
\node[st op, right=5mm of W1b] (eA) {$=$};
|
||||||
|
\stface[role=r1]{Ha}{($(eA)+(1.5,0)$)}{bt}{dffl}
|
||||||
|
\stface[role=r2]{Hb}{($(Ha.east)+(2*\stunit,0)$)}{bt}{dffl}
|
||||||
|
|
||||||
|
\node[inner sep=0pt, fit=(X)(W1a)(Ha)(Hb)] (rowA) {};
|
||||||
|
\stlane{rowA}
|
||||||
|
\stcaption{X}{$\mathbf X$}{$BT\times d$}
|
||||||
|
\stcaption{W1a}{$\mathbf W_1^{(1)}$}{$d\times d_{\mathrm{ff}}/p$}
|
||||||
|
\stcaption{W1b}{$\mathbf W_1^{(2)}$}{$d\times d_{\mathrm{ff}}/p$}
|
||||||
|
\stcaption{Ha}{$\mathbf H^{(1)}$}{$BT\times d_{\mathrm{ff}}/p$}
|
||||||
|
\stcaption{Hb}{$\mathbf H^{(2)}$}{$BT\times d_{\mathrm{ff}}/p$}
|
||||||
|
\stnolane
|
||||||
|
|
||||||
|
% ============================================================ stage B row ===
|
||||||
|
\node[st stage, below=9mm of X-shape.south west, anchor=north west] (SB)
|
||||||
|
{行切 $\mathbf W_2$：一次 All-Reduce};
|
||||||
|
\coordinate (b) at ($(SB)+(-0.4,-1.9)$);
|
||||||
|
|
||||||
|
\stface[role=r1]{Ga}{(b)}{bt}{dffl}
|
||||||
|
\stface[role=r2]{Gb}{($(Ga.east)+(2*\stunit,0)$)}{bt}{dffl}
|
||||||
|
\node[st op, right=5mm of Gb] (mB) {$\times$};
|
||||||
|
% W_2 is split along the CONTRACTED axis: the two shards stack vertically and
|
||||||
|
% together have exactly the height of H's width. Splitting reverses concat.
|
||||||
|
\stface[role=r1]{W2a}{($(mB)+(1.35,0.46)$)}{dffl}{d}
|
||||||
|
\stface[role=r2]{W2b}{($(W2a.south)+(0,-2*\stunit)$)}{dffl}{d}
|
||||||
|
\node[st op, right=5mm of W2a.east |- W2a.south] (eB) {$=$};
|
||||||
|
\stface[role=r1]{Pa}{($(eB)+(1.3,0)$)}{bt}{d}
|
||||||
|
\node[st op, right=4mm of Pa] (plus) {$+$};
|
||||||
|
\stface[role=r2]{Pb}{($(plus)+(1.3,0)$)}{bt}{d}
|
||||||
|
|
||||||
|
\node[st comm, right=9mm of Pb] (ar) {All-Reduce};
|
||||||
|
\stface[role=act, bracket=true]{Y}{($(ar)+(1.9,0)$)}{bt}{d}
|
||||||
|
\starrow{Pb.east}{ar.west}
|
||||||
|
\starrow{ar.east}{Y.west}
|
||||||
|
|
||||||
|
\node[inner sep=0pt, fit=(Ga)(W2a)(W2b)(Pa)(Pb)(Y)] (rowB) {};
|
||||||
|
\stlane{rowB}
|
||||||
|
\stcaption{Ga}{$\mathbf G^{(1)}$}{$BT\times d_{\mathrm{ff}}/p$}
|
||||||
|
\stcaption{Gb}{$\mathbf G^{(2)}$}{$BT\times d_{\mathrm{ff}}/p$}
|
||||||
|
\stcaption{W2b}{$\mathbf W_2^{(r)}$}{$d_{\mathrm{ff}}/p\times d$}
|
||||||
|
\stcaption{Pa}{$\mathbf P^{(1)}$}{$BT\times d$}
|
||||||
|
\stcaption{Pb}{$\mathbf P^{(2)}$}{$BT\times d$}
|
||||||
|
\stcaption{Y}{$\mathbf Y$}{$BT\times d$}
|
||||||
|
\stnolane
|
||||||
|
|
||||||
|
% ============================================================== meaning box ==
|
||||||
|
\node[inner sep=0pt, fit=(F)(rowA)(rowB)(Y-shape)(Ga-shape)] (all) {};
|
||||||
|
\stmeaningbox{mb}{16.4cm}{all}
|
||||||
|
{$BT$ 展平后的 token 数，$d$ 模型宽度，$d_{\mathrm{ff}}$ 前馈中间宽度，
|
||||||
|
$p$ TP 并行度（图中 $p=2$）}
|
||||||
|
{$\mathbf X,\mathbf Y$ 每个 rank 完整持有；$\mathbf W_1^{(r)},\mathbf W_2^{(r)},
|
||||||
|
\mathbf H^{(r)},\mathbf P^{(r)}$ 仅本 rank 持有，$\mathbf P^{(r)}$ 是部分和而非最终输出}
|
||||||
|
{$\mathbf G^{(r)}=\mathrm{GeLU}(\mathbf H^{(r)})$ 逐元素、无跨 rank 依赖；列切 $\mathbf W_1$ 使 $\mathbf H$ 沿 $d_{\mathrm{ff}}$ 切分；行切 $\mathbf W_2$ 沿收缩维切分，
|
||||||
|
故 $\mathbf Y=\sum_r\mathbf P^{(r)}$ 需一次 All-Reduce，前向每层仅此一次通信}
|
||||||
|
\stsignature{TP-FFN（GeLU + All-Reduce）}{mb}
|
||||||
|
|
||||||
|
\end{tikzpicture}
|
||||||
|
\end{document}
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
# Anti-patterns
|
||||||
|
|
||||||
|
Every figure below compiles cleanly. `build.sh` is happy with all of them. They are still
|
||||||
|
wrong, because the compiler checks TeX syntax and not whether the picture is true.
|
||||||
|
|
||||||
|
Render `examples/antipatterns.tex` and look at
|
||||||
|
`examples/build/antipatterns.png` once before your first figure.
|
||||||
|
|
||||||
|
## 1. The transpose that only changed its label
|
||||||
|
|
||||||
|
A face captioned `Kᵀ` that is still `T × d_h`. The reader looks for the contracted axis,
|
||||||
|
finds two faces of the same height, and concludes the contraction runs along the wrong
|
||||||
|
dimension. **Fix:** swap the arguments — `\ststack{KT}{...}{dh}{T}{3}`. See `geometry.md` §3.
|
||||||
|
|
||||||
|
## 2. Shards that do not tile their parent
|
||||||
|
|
||||||
|
Two shards drawn with a gap, or stretched to fill a parent whose other shards were elided.
|
||||||
|
Both assert a width that the tensor does not have. **Fix:** place each shard from the
|
||||||
|
previous one's edge (`($(W1a.east)+(2*\stunit,0)$)`), and draw an ellipsis for anything
|
||||||
|
omitted. See `geometry.md` §5–6.
|
||||||
|
|
||||||
|
## 3. An index drawn as a heatmap
|
||||||
|
|
||||||
|
Expert ids or token positions rendered with a lightness ramp. The ramp is a magnitude
|
||||||
|
channel, so it says `expert 3 > expert 0`, which is meaningless. **Fix:** `\stindexface`.
|
||||||
|
See `semantics.md`.
|
||||||
|
|
||||||
|
Same family: a Boolean mask drawn with graded cells (it has one level, not three), and a
|
||||||
|
score matrix drawn as flat blocks (it has magnitude, and hiding it wastes the figure).
|
||||||
|
|
||||||
|
## 4. One pale level everywhere
|
||||||
|
|
||||||
|
A whole tensor in `role!10`. At full size it looks tasteful; at thumbnail size — which is
|
||||||
|
how it will be seen on a slide — it is a blank rectangle. **Fix:** three separated levels,
|
||||||
|
`role!30 / role!55 / role!80`. Contrast comes from lightness, not saturation.
|
||||||
|
See `style.md`.
|
||||||
|
|
||||||
|
## Not in the gallery, but just as common
|
||||||
|
|
||||||
|
- **Periodic texture.** A polynomial hash reduced mod 3 repeats every 3 rows, and the eye
|
||||||
|
reads the resulting stripe as real structure. `pattern=dense` avoids it; if you write
|
||||||
|
your own filler, check that rows 1, 2, 4, 5 of a tall face are not identical.
|
||||||
|
- **A label wider than its connector.** The white underlay then covers the target tensor.
|
||||||
|
Shorten the label or widen the gap — never let it sit on a face. See `layout.md`.
|
||||||
|
- **A new hue for a regrouped view of the same data.** `X` and the per-expert buffers
|
||||||
|
gathered out of `X` are the same object in a different order; a second hue claims they
|
||||||
|
are different tensors.
|
||||||
|
- **Captions hanging at different depths** because the faces in a row have different
|
||||||
|
heights. Use `\stlane`.
|
||||||
|
- **A floating commentary card between two operands.** If it is not a real operation, it
|
||||||
|
belongs in the stage subtitle or the bottom box.
|
||||||
|
- **A meaning box that repeats the shapes.** The shapes are already under every block. The
|
||||||
|
box is for what the axes *mean* and what the operation *does*.
|
||||||
|
- **Solving crowding by shrinking type.** The type hierarchy is a hard floor; move the
|
||||||
|
stage to another row instead.
|
||||||
@@ -0,0 +1,103 @@
|
|||||||
|
# supertensor.sty API
|
||||||
|
|
||||||
|
```tex
|
||||||
|
\documentclass[border=10pt]{standalone}
|
||||||
|
\usepackage[cjk]{supertensor} % cjk: ctex + fandol (XeLaTeX). en: English rail labels.
|
||||||
|
```
|
||||||
|
|
||||||
|
Build with `./scripts/build.sh fig.tex` — it puts `assets/` on `TEXINPUTS`, so the package
|
||||||
|
does not need to be installed into your texmf tree.
|
||||||
|
|
||||||
|
## Ledgers
|
||||||
|
|
||||||
|
```tex
|
||||||
|
\stsetrole{q}{stTeal} % role -> color. Macros take a ROLE, never a color.
|
||||||
|
\stdim{T}{6} % symbolic axis -> physical edge length in cells
|
||||||
|
\stsetauthor{...} % default 五道口纳什
|
||||||
|
\stsetlabels{A}{O}{M} % override the three meaning-box rail labels
|
||||||
|
\stsetrail{3.2em} % width of the bold label rail
|
||||||
|
```
|
||||||
|
|
||||||
|
Colors: `stTeal stOrange stCoral stViolet stGray stInk`. An unknown role falls back to gray
|
||||||
|
**and emits a package warning**, which `build.sh` turns into a failed build.
|
||||||
|
|
||||||
|
Lengths: `\stunit` (one cell, 4.6 mm) and `\sttilegap` (white gutter, 0.5 mm).
|
||||||
|
|
||||||
|
## Faces
|
||||||
|
|
||||||
|
```tex
|
||||||
|
\stface[keys]{name}{(coord)}{rows}{cols}
|
||||||
|
\ststack[keys]{name}{(coord)}{rows}{cols}{sheets}
|
||||||
|
\stindexface[keys]{name}{(coord)}{rows}{cols}{entries}
|
||||||
|
```
|
||||||
|
|
||||||
|
`rows`/`cols` accept a declared axis name or a raw integer. `(coord)` must include its own
|
||||||
|
parentheses — `{(0,0)}`, `{($(A.east)+(1.5,0)$)}`. `name` becomes a TikZ node you can
|
||||||
|
anchor against; `\ststack` also defines `name-front`.
|
||||||
|
|
||||||
|
Keys:
|
||||||
|
|
||||||
|
| key | default | meaning |
|
||||||
|
|---|---|---|
|
||||||
|
| `role=` | `neutral` | hue, via `\stsetrole` |
|
||||||
|
| `pattern=` | `dense` | `solid dense diag band lower upper causal empty data` |
|
||||||
|
| `data=` | — | with `pattern=data`: comma-separated rows, one digit per cell, `0`–`3` = level |
|
||||||
|
| `level=` | `2` | level for `pattern=solid` and the flat level of a mask |
|
||||||
|
| `bracket=` | `false` | thin neutral matrix brackets |
|
||||||
|
| `border=` | `true` | outer `black!60` border |
|
||||||
|
| `tiles=` | `true` | `false` = one flat filled rectangle |
|
||||||
|
|
||||||
|
`\stindexface` entries are row-major, `rows*cols` of them; `.` leaves a cell blank.
|
||||||
|
It deliberately has no lightness ramp — see `semantics.md`.
|
||||||
|
|
||||||
|
```tex
|
||||||
|
\stface[role=w, pattern=data, level=3,
|
||||||
|
data={3300,0330,0033,3003,3030,0303}]{D}{(0,0)}{T}{E}
|
||||||
|
\stindexface[role=idx]{I}{(b)}{T}{k}{0,1, 1,2, 2,3, 3,0, 0,2, 1,3}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Captions
|
||||||
|
|
||||||
|
```tex
|
||||||
|
\node[inner sep=0pt, fit=(A)(B)(C)] (rowA) {};
|
||||||
|
\stlane{rowA}
|
||||||
|
\stcaption{A}{$\mathbf A$}{$T\times d$} % symbol lane, shape lane
|
||||||
|
\stnolane
|
||||||
|
\stcaptiontop{A}{\stshapefont{token}} % occasional label above a face
|
||||||
|
```
|
||||||
|
|
||||||
|
`\stcaption` defines `name-sym` and `name-shape` nodes; anchor the next stage heading
|
||||||
|
against `name-shape.south`.
|
||||||
|
|
||||||
|
## Operators, connectors, nodes
|
||||||
|
|
||||||
|
```tex
|
||||||
|
\node[st op, right=6mm of A] (m) {$\times$};
|
||||||
|
\node[st comm, right=9mm of P] (ar) {All-Reduce};
|
||||||
|
\starrow{P.east}{ar.west}
|
||||||
|
\starrowlabel{M.east}{A.west}{softmax}
|
||||||
|
```
|
||||||
|
|
||||||
|
Styles: `st sym st shape st stage st op st note st arrow st comm st brace`.
|
||||||
|
Text helpers: `\stformula \ststagelabel \stoperator \stsymfont \stshapefont \stprose`.
|
||||||
|
Connectors route on the background layer automatically.
|
||||||
|
|
||||||
|
## Bottom
|
||||||
|
|
||||||
|
```tex
|
||||||
|
\node[inner sep=0pt, fit=(F)(rowA)(rowB)(Y-shape)] (all) {};
|
||||||
|
\stmeaningbox{mb}{16.6cm}{all}{axes text}{objects text}{mechanism text}
|
||||||
|
\stsignature{因果多头注意力（掩码 + 拼接投影）}{mb}
|
||||||
|
```
|
||||||
|
|
||||||
|
Arg 2 is the total box width; arg 3 is the node it hangs below — include every caption and
|
||||||
|
top label in that `fit` or the box will overlap them. An empty `{}` row is dropped.
|
||||||
|
|
||||||
|
## Gotchas
|
||||||
|
|
||||||
|
- `\ststack` never re-enters `\stface`; if you extend the package, do not pass
|
||||||
|
`role=\st@role` back through pgfkeys — it defines the macro in terms of itself and hangs.
|
||||||
|
- A coordinate expression inside `fit=` needs braces: `fit={(a) ($(b)+(1,0)$)}`.
|
||||||
|
- `\foreach {\macro,...,1}` cannot infer its direction from an unexpanded macro.
|
||||||
|
- `\strole` is expandable on purpose (it is used inside `\edef`); the warning lives in
|
||||||
|
`\stcheckrole`.
|
||||||
@@ -0,0 +1,59 @@
|
|||||||
|
# Pre-delivery checklist
|
||||||
|
|
||||||
|
A clean `build.sh` proves only that TeX was happy. Nothing below is checked by the compiler.
|
||||||
|
Work through it against the rendered PNG. Any mandatory violation means redraw, not patch.
|
||||||
|
|
||||||
|
## 1. Math (before looking at the picture)
|
||||||
|
|
||||||
|
- [ ] Every shape recomputed independently from the source formula or code.
|
||||||
|
- [ ] Block multiplication, broadcasting, reductions and sharding algebra verified.
|
||||||
|
- [ ] Axis identities that the figure asserts actually hold in the numbers drawn
|
||||||
|
(`d = h·d_h`, `Σ_e n_e = T·k`, shards summing to the parent).
|
||||||
|
|
||||||
|
## 2. Semantics ledger
|
||||||
|
|
||||||
|
- [ ] Every discrete or overloaded symbol has one type, domain and range.
|
||||||
|
- [ ] Score, index and mask are three distinct objects in three distinct grammars.
|
||||||
|
- [ ] Producer-to-consumer chain closed: scores → indices → gather/mask → values.
|
||||||
|
- [ ] Shared selectors drawn once, with the reuse axis marked.
|
||||||
|
|
||||||
|
## 3. Geometry ledger
|
||||||
|
|
||||||
|
- [ ] Equal shapes have identical faces everywhere.
|
||||||
|
- [ ] `a×a` is square; every transpose physically swaps the face.
|
||||||
|
- [ ] Both occurrences of each contracted axis have the same edge length.
|
||||||
|
- [ ] Shards tile their parent exactly; concat reverses split; elisions use an ellipsis.
|
||||||
|
|
||||||
|
## 4. Full-size visual audit
|
||||||
|
|
||||||
|
Open the PNG at 100 %.
|
||||||
|
|
||||||
|
- [ ] No forbidden intersection, tangency, clipping or occlusion — including stack offset
|
||||||
|
sheets, brackets, arrow labels and the meaning box.
|
||||||
|
- [ ] Every connector's white label underlay covers only its own connector.
|
||||||
|
- [ ] No connector crosses a box that is not its endpoint.
|
||||||
|
- [ ] Symbols and shapes sit on two flat lanes per row; stage headings share a left rail.
|
||||||
|
- [ ] Top zone compact (≤2 formula lines, no shape underbraces).
|
||||||
|
- [ ] Bottom box: one column, ≤3 rows, no overflow, no shrunken type.
|
||||||
|
- [ ] Signature outside the box, one line, names what the figure actually shows, not clipped
|
||||||
|
and not visually dominant.
|
||||||
|
- [ ] Structural support exact: known zeros unfilled, masks and diagonals exactly right.
|
||||||
|
|
||||||
|
## 5. Thumbnail audit
|
||||||
|
|
||||||
|
Open `*-thumb.png` (360 px).
|
||||||
|
|
||||||
|
- [ ] ≤4 active hue families per row plus gray; each role keeps one hue across stages.
|
||||||
|
- [ ] Base colors still muted, not saturated.
|
||||||
|
- [ ] Three visibly separated lightness levels where values vary; nothing is uniform pastel.
|
||||||
|
- [ ] Borders neutral, not tensor-colored.
|
||||||
|
- [ ] Dense texture reads as noise, not as stripes, checkerboard or a symmetric motif.
|
||||||
|
- [ ] The main structural claim of the figure is still legible at this size.
|
||||||
|
|
||||||
|
## 6. Delivery
|
||||||
|
|
||||||
|
- [ ] PNG preview shown.
|
||||||
|
- [ ] Short mechanism explanation in prose.
|
||||||
|
- [ ] `.tex` source and vector artifact (PDF/SVG) linked.
|
||||||
|
- [ ] Any degraded path stated explicitly — no CJK font, no `pdftocairo`, fallback renderer,
|
||||||
|
inferred shapes or conventions.
|
||||||
@@ -0,0 +1,77 @@
|
|||||||
|
# Geometry invariants
|
||||||
|
|
||||||
|
A figure that is drawn to the wrong geometry is not a stylistic problem — it teaches the
|
||||||
|
reader a false fact about the computation. These invariants are mandatory. Most of them
|
||||||
|
are automatic if you declare the geometry ledger and never pass a raw number.
|
||||||
|
|
||||||
|
## The ledger
|
||||||
|
|
||||||
|
```tex
|
||||||
|
\stdim{T}{6} % sequence length -> 6 cells, everywhere in the figure
|
||||||
|
\stdim{d}{9} % model width -> 9 cells, everywhere
|
||||||
|
\stdim{dh}{3} % head width -> 3 cells, and 3*dh = d holds visually
|
||||||
|
```
|
||||||
|
|
||||||
|
`\stface{...}{T}{dh}` resolves the names through the ledger, so **one symbolic axis maps
|
||||||
|
to exactly one physical edge length across the whole figure**. Equal shapes therefore form
|
||||||
|
an equivalence class automatically: `Q` and `V` at `T×d_h` come out identical without you
|
||||||
|
lining anything up by hand.
|
||||||
|
|
||||||
|
Raw integers are accepted (`\stface{A}{(0,0)}{4}{4}`) but they opt out of the guarantee.
|
||||||
|
Use them only for a face whose axis appears nowhere else.
|
||||||
|
|
||||||
|
## The rules
|
||||||
|
|
||||||
|
1. **Face orientation.** A matrix face `a×b` is height `a`, width `b`. Always. For batched
|
||||||
|
or stacked tensors, the *last two* axes make the face; leading axes become depth
|
||||||
|
(`\ststack`) or repeated panels — never a wider rectangle.
|
||||||
|
2. **Squares.** `a×a` renders as a square. Automatic when both arguments resolve to the
|
||||||
|
same declared axis.
|
||||||
|
3. **Transpose.** Draw `K^T` by physically swapping height and width:
|
||||||
|
`\ststack{KT}{...}{dh}{T}{3}` against `\ststack{K}{...}{T}{dh}{3}`. Relabelling a face
|
||||||
|
`K^T` while leaving its shape alone is invalid — it is the single most common lie in
|
||||||
|
attention figures.
|
||||||
|
4. **Contraction.** In `(m×k)(k×n)`, both occurrences of `k` get the same edge length.
|
||||||
|
With the ledger this is free: pass the same axis name to the width of the left face and
|
||||||
|
the height of the right one. The same applies to `einsum` and attention axes.
|
||||||
|
5. **Partition.** Explicit shards tile their parent exactly along the split axis, equal
|
||||||
|
shards are equal in size, and concatenation reverses the split. Place shards from the
|
||||||
|
previous face's edge so no gap can creep in:
|
||||||
|
|
||||||
|
```tex
|
||||||
|
\stface[role=r1]{W1a}{(...)}{d}{dffl}
|
||||||
|
\stface[role=r2]{W1b}{($(W1a.east)+(2*\stunit,0)$)}{d}{dffl}
|
||||||
|
```
|
||||||
|
|
||||||
|
The offset is `half-width of the next face` in `\stunit`, so the two faces are exactly
|
||||||
|
adjacent. Size concatenated parts from their *declared* shapes — Q/K/V are equal
|
||||||
|
segments only when their output shapes are equal.
|
||||||
|
6. **Elision.** If intermediate shards are omitted, draw an ellipsis. Never stretch the
|
||||||
|
visible shards to impersonate the full parent.
|
||||||
|
7. **Axis changes.** Geometry may change only at an explicit reshape, flatten, transpose,
|
||||||
|
split, or concat operator, and the axis identity must be stated — e.g. `(h/p)·d_h = d/p`.
|
||||||
|
Do not silently reuse one generic rectangle on both sides of an axis change.
|
||||||
|
8. **Illustrative counts.** Cell counts need not equal real dimensions. Choosing `d=9`
|
||||||
|
to stand for 4096 is fine. It never waives rules 1–7: the *ratios* you draw are read as
|
||||||
|
facts. If `d = h·d_h`, pick numbers where that arithmetic actually holds.
|
||||||
|
|
||||||
|
## Sharded matmul
|
||||||
|
|
||||||
|
When a matmul is sharded, expand the block algebra in the top formula as well as in the
|
||||||
|
middle row, otherwise the reader cannot check the figure:
|
||||||
|
|
||||||
|
```
|
||||||
|
XW = [XW^(1) | ... | XW^(p)] = [H^(1) | ... | H^(p)]
|
||||||
|
|
||||||
|
[H^(1) | ... | H^(p)] [W^(1); ...; W^(p)] = Σ_r H^(r) W^(r) = Σ_r P^(r)
|
||||||
|
```
|
||||||
|
|
||||||
|
Column sharding splits the *output* axis (shards sit side by side); row sharding splits the
|
||||||
|
*contracted* axis (shards stack vertically and require a reduction). `examples/tp-ffn-allreduce.tex`
|
||||||
|
draws both in one figure.
|
||||||
|
|
||||||
|
## Distinguish global from local
|
||||||
|
|
||||||
|
Per-rank shapes and global shapes are different objects. Label them differently
|
||||||
|
(`d_ff/p` vs `d_ff`) and, when both appear, say in the **Axes** row which one the figure
|
||||||
|
is drawing.
|
||||||
@@ -0,0 +1,89 @@
|
|||||||
|
# Layout invariants
|
||||||
|
|
||||||
|
Everything on the canvas is a bounding box: tensors, full offset stacks, brackets,
|
||||||
|
operators, arrow labels, annotations, symbols, shape labels, stage headings, the meaning
|
||||||
|
box, the signature. **Tangency counts as collision.**
|
||||||
|
|
||||||
|
## Gutters
|
||||||
|
|
||||||
|
Define one base gutter `g ≥ 1 em`. Unrelated boxes stay at least `g` apart; stage bands at
|
||||||
|
least `1.5g`. In practice: `right=5mm–9mm` between an operator and its operands, `7mm–9mm`
|
||||||
|
between the last caption of one row and the next stage heading.
|
||||||
|
|
||||||
|
Overlap is allowed only inside one declared composite:
|
||||||
|
|
||||||
|
- tiles inside their own face,
|
||||||
|
- shards tiling a parent,
|
||||||
|
- outline sheets in one `\ststack`,
|
||||||
|
- a bracket around its own tensor,
|
||||||
|
- a connector endpoint touching its source/target border.
|
||||||
|
|
||||||
|
Every other intersection or occlusion is forbidden.
|
||||||
|
|
||||||
|
## Lanes
|
||||||
|
|
||||||
|
Reserve separate vertical lanes and never put anything else in them:
|
||||||
|
|
||||||
|
```
|
||||||
|
stage heading
|
||||||
|
(connector annotations)
|
||||||
|
tensor / operator row
|
||||||
|
symbols <- \stcaption arg 2
|
||||||
|
shapes <- \stcaption arg 3
|
||||||
|
```
|
||||||
|
|
||||||
|
Faces of different heights would otherwise hang their captions at different depths. Fix it
|
||||||
|
with a shared baseline:
|
||||||
|
|
||||||
|
```tex
|
||||||
|
\node[inner sep=0pt, fit=(Q)(KT)(S)] (rowA) {};
|
||||||
|
\stlane{rowA}
|
||||||
|
\stcaption{Q}{$\mathbf Q^{(i)}$}{$h\times T\times d_h$}
|
||||||
|
...
|
||||||
|
\stnolane
|
||||||
|
```
|
||||||
|
|
||||||
|
Every `\stcaption` between `\stlane` and `\stnolane` hangs from the bottom of `rowA`, so
|
||||||
|
symbols and shapes form two flat lanes.
|
||||||
|
|
||||||
|
Stage headings share one left rail. Anchor each heading below the previous row but at the
|
||||||
|
previous *heading's* x, not at the previous row's content:
|
||||||
|
|
||||||
|
```tex
|
||||||
|
\coordinate (cy) at ($(I-shape.south)+(0,-9mm)$);
|
||||||
|
\node[st stage, anchor=north west] (SC) at (SB.west |- cy) {...};
|
||||||
|
```
|
||||||
|
|
||||||
|
Explanatory prose belongs in the stage subtitle, the bottom box, or above its own
|
||||||
|
connector. Never drop a floating commentary card between two operands unless it is a real
|
||||||
|
operation node (`st comm`).
|
||||||
|
|
||||||
|
## Layers
|
||||||
|
|
||||||
|
The package declares three: `stbg` (connectors), `main` (tensors, operators), `stfg`
|
||||||
|
(text). `\starrow` and `\starrowlabel` route on `stbg` automatically, so a connector can
|
||||||
|
never cover a face. Two consequences you still own:
|
||||||
|
|
||||||
|
- A connector may not cross a box that is not one of its endpoints. Move the row, don't
|
||||||
|
route over.
|
||||||
|
- A label's white underlay may cover only its own connector — never a tensor, never
|
||||||
|
another label. If the label is wider than the arrow, shorten the label or widen the gap.
|
||||||
|
This is the single most common failure after a first draft.
|
||||||
|
|
||||||
|
## Stacks
|
||||||
|
|
||||||
|
`\ststack` includes its offset sheets in the bounding box, so neighbours can be spaced
|
||||||
|
against the real extent. Back sheets are outline-only and must carry no semantic content
|
||||||
|
of their own. If individual slices need to be read, use separate panels instead of overlap.
|
||||||
|
|
||||||
|
## When it does not fit
|
||||||
|
|
||||||
|
In this order:
|
||||||
|
|
||||||
|
1. shorten or remove secondary annotation,
|
||||||
|
2. widen the natural crop,
|
||||||
|
3. increase row spacing,
|
||||||
|
4. move the whole stage to another row.
|
||||||
|
|
||||||
|
Never solve crowding by shrinking below the type hierarchy, closing the gutter, or covering
|
||||||
|
another object.
|
||||||
@@ -0,0 +1,67 @@
|
|||||||
|
# Symbol semantics
|
||||||
|
|
||||||
|
Shape is not meaning. Two tensors of shape `T×k` can be a score matrix, a list of token
|
||||||
|
positions, or a Boolean support, and drawing all three the same way is the fastest way to
|
||||||
|
mislead a reader who is trying to follow the mechanism.
|
||||||
|
|
||||||
|
## Classify every non-obvious symbol
|
||||||
|
|
||||||
|
For each symbol record: **semantic kind**, **dtype/domain**, **what one entry means**, and
|
||||||
|
its **range** when meaningful. Kinds worth separating:
|
||||||
|
|
||||||
|
| kind | example | domain |
|
||||||
|
|---|---|---|
|
||||||
|
| value / activation | `X`, `H` | ℝ |
|
||||||
|
| score / logit | `S = QKᵀ/√d_h` | ℝ |
|
||||||
|
| probability | `A = softmax(S)` | [0,1], rows sum to 1 |
|
||||||
|
| index / coordinate | `I = TopKIndices(G)` | {0,…,E−1} |
|
||||||
|
| rank / order | selected-slot axis `r` | {1,…,k} |
|
||||||
|
| count | `n_e` tokens per expert | ℕ |
|
||||||
|
| id | token id, device id | opaque |
|
||||||
|
| mask / support | `D`, causal `M` | {0,1} or {0,−∞} |
|
||||||
|
| permutation | gather order | bijection |
|
||||||
|
| shape parameter | `p`, `h` | ℕ, not drawn as a tensor |
|
||||||
|
|
||||||
|
## One block, one object
|
||||||
|
|
||||||
|
Never merge a score, an index list and a mask under a label like `M/S`. Each conversion
|
||||||
|
gets an explicit operator and arrow. For selection or routing, close the entire chain:
|
||||||
|
|
||||||
|
```
|
||||||
|
continuous scores → discrete indices/ids → gather / scatter / mask / route → selected values
|
||||||
|
```
|
||||||
|
|
||||||
|
`TopKValues` and `TopKIndices` are different tensors; if both are used, show both.
|
||||||
|
|
||||||
|
## Three grammars, deliberately different
|
||||||
|
|
||||||
|
| object | grammar | package |
|
||||||
|
|---|---|---|
|
||||||
|
| value / score / probability | magnitude — three separated lightness levels | `\stface[pattern=dense]` |
|
||||||
|
| index / id | discrete symbols in outlined cells, **no** lightness ramp | `\stindexface{...}{entries}` |
|
||||||
|
| mask / support | one flat level, exact structure, zeros unfilled | `\stface[pattern=causal, level=3]` or `pattern=data` |
|
||||||
|
|
||||||
|
`examples/moe-topk-gather.tex` puts all three in one figure on purpose. The reason indices
|
||||||
|
get no ramp: a ramp invites the reader to compare `expert 3 > expert 0` as if the number
|
||||||
|
were a size.
|
||||||
|
|
||||||
|
## Notation duties
|
||||||
|
|
||||||
|
- Define index notation and range at first use:
|
||||||
|
`S_t = (s_{t,1},…,s_{t,k})`, `s_{t,r} ∈ {0,…,t}`.
|
||||||
|
- Distinguish the *source-position* axis `s` from the *selected-slot* axis `r`, and say
|
||||||
|
whether ordering, duplicates, padding or variable cardinality matter.
|
||||||
|
- Show the address mapping once: `G[b,t,r,:] = X[b, S[b,t,r], :]`.
|
||||||
|
- If a mask is shown alongside the index tensor, state `M[b,t,s] = 1[s ∈ S_{b,t}]` — do not
|
||||||
|
let the figure imply they are the same object.
|
||||||
|
- When one selector is shared across heads, ranks or branches, draw it **once** and mark
|
||||||
|
the broadcast/reuse axis. A per-head copy of a shared mask is a false claim about memory
|
||||||
|
and about the computation (`examples/mha-causal.tex`: `M` is a single face while `A` is a
|
||||||
|
three-sheet stack).
|
||||||
|
|
||||||
|
## Colors carry semantics too
|
||||||
|
|
||||||
|
One tensor role keeps one hue for the whole figure — that is what `\stsetrole` is for. A
|
||||||
|
gathered, resharded or regrouped view of the same data keeps the *same* role color; a new
|
||||||
|
hue means a new object. Derived tensors may reuse their parent's family rather than
|
||||||
|
spending a hue (`V → O → Y` in the MHA example are all violet).
|
||||||
@@ -0,0 +1,94 @@
|
|||||||
|
# House style
|
||||||
|
|
||||||
|
The package ships these defaults; this file explains what you must still decide and what
|
||||||
|
you must not undo.
|
||||||
|
|
||||||
|
## Canvas
|
||||||
|
|
||||||
|
White background, natural `standalone` crop. No forced 16:9. No title by default. The top
|
||||||
|
holds at most two compact formula lines: the primary chain and, only if essential, one
|
||||||
|
companion definition.
|
||||||
|
|
||||||
|
Each stage is one horizontal algebraic row with a shared visual baseline. Use the fewest
|
||||||
|
stages that preserve the primary path. No unrelated branches, no dashboard panels.
|
||||||
|
|
||||||
|
## Type hierarchy
|
||||||
|
|
||||||
|
| element | size | macro / style |
|
||||||
|
|---|---|---|
|
||||||
|
| formula | `\large` | `\stformula` |
|
||||||
|
| stage label | `\small\bfseries`, muted | `st stage` |
|
||||||
|
| operator | `\Large` | `st op` |
|
||||||
|
| symbol | `\small` | `\stcaption` arg 2 |
|
||||||
|
| shape | `\scriptsize`, muted | `\stcaption` arg 3 |
|
||||||
|
| bottom prose | `\small` | `\stmeaningbox` |
|
||||||
|
| signature | `\scriptsize`, low contrast | `\stsignature` |
|
||||||
|
|
||||||
|
Never shrink below this to make something fit — see `layout.md`.
|
||||||
|
|
||||||
|
## Faces
|
||||||
|
|
||||||
|
Separate tiles with a small white gutter and 0.5–1 pt corner rounding (`\st@tile` does
|
||||||
|
this). Thin neutral brackets, `black!55`–`black!70` outer borders. No saturated
|
||||||
|
tensor-colored outlines, no continuous spreadsheet grid.
|
||||||
|
|
||||||
|
**Encode support before magnitude.** Every known zero stays white/unfilled; every shown
|
||||||
|
nonzero gets color. A diagonal matrix must read instantly as colored diagonal cells on a
|
||||||
|
white field. `pattern=diag/band/lower/upper/causal/data` fill exactly the structural
|
||||||
|
support.
|
||||||
|
|
||||||
|
## Color
|
||||||
|
|
||||||
|
Palette (already defined): `stTeal #4F8FA5`, `stOrange #EE995B`, `stCoral #C95B5B`,
|
||||||
|
`stViolet #8A74B5`, `stGray #85898F`. Muted, mid-chroma, paper-like. Do not add saturated
|
||||||
|
primaries.
|
||||||
|
|
||||||
|
- One semantic color per tensor role, held across every stage: `\stsetrole{q}{stTeal}`.
|
||||||
|
Macros take a **role**, never a color.
|
||||||
|
- At most four active hue families per algebraic row, plus neutral gray. Vary lightness or
|
||||||
|
reuse the related input/output family before spending a new hue.
|
||||||
|
- If sign matters: hue for sign, intensity for magnitude.
|
||||||
|
- Contrast comes from **lightness separation, not saturation**. Levels are `role!30`,
|
||||||
|
`role!55`, `role!80` (`\stlevelpct`); white is reserved for zero/absence. Do not render
|
||||||
|
a whole tensor in `role!5`–`role!15` pastel.
|
||||||
|
- Illustrative dense tensors use two or three **non-periodic** levels. `pattern=dense`
|
||||||
|
composes coprime moduli to avoid this; a polynomial hash mod 3 makes rows 1, 2, 4, 5
|
||||||
|
identical and the eye reads that stripe as structure in the data. No checkerboards, no
|
||||||
|
regular stripes, no symmetric motifs unless they encode real structure.
|
||||||
|
|
||||||
|
## Bottom box
|
||||||
|
|
||||||
|
One full-width, low-contrast box, one reading column, three fixed-label rows:
|
||||||
|
|
||||||
|
- **Axes** — what each dimension means.
|
||||||
|
- **Objects** — semantic kind / domain / range (see `semantics.md`).
|
||||||
|
- **Mechanism** — at most two essential mappings, contractions, broadcasts or boundaries.
|
||||||
|
|
||||||
|
Narrow bold label rail, left-aligned ragged-right `\small` content, 8–10 pt inner padding,
|
||||||
|
0.4–0.6 em row gaps. Rail labels localize with the package option (`zh` default, `en`).
|
||||||
|
Keep each row compact: prefer symbol semantics over numeric configuration. When it is too
|
||||||
|
long, **remove content** — never add cards, columns or smaller type. Pass `{}` to omit a row.
|
||||||
|
|
||||||
|
## Signature
|
||||||
|
|
||||||
|
One centered line below the box, outside it, low-contrast gray, `\scriptsize` or smaller:
|
||||||
|
`\stsignature{<subject>}{<fit node>}` renders `<subject>@五道口纳什`. The subject must name
|
||||||
|
what this figure actually visualizes. Keep it on one line, with a small but visible gap.
|
||||||
|
|
||||||
|
## Never
|
||||||
|
|
||||||
|
Charts or metric insets not present in the primary formula. Decorative pills, banners,
|
||||||
|
shadows, repeated separators, explanatory cards.
|
||||||
|
|
||||||
|
## Reference image
|
||||||
|
|
||||||
|
`assets/kimi-matrix-style-reference.png` — inspect it with an image viewer before drawing
|
||||||
|
when entry-level matrix blocks are central or Kimi-like styling is requested. Use it only
|
||||||
|
to calibrate tile spacing, rounding, neutral brackets, restrained hue, lightness separation
|
||||||
|
and structural whitespace. Do not copy its content, and do not embed it in the output.
|
||||||
|
|
||||||
|
## Language
|
||||||
|
|
||||||
|
Chinese figures use concise Chinese labels with standard English terms where helpful
|
||||||
|
(`softmax`, `All-Reduce`, `gather` stay in English). Switch the whole figure at once —
|
||||||
|
`\usepackage[en]{supertensor}` plus English stage headings and box text — never mix.
|
||||||
Executable
+74
@@ -0,0 +1,74 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Compile a supertensor figure and export every delivery artifact.
|
||||||
|
#
|
||||||
|
# ./scripts/build.sh figure.tex [outdir]
|
||||||
|
#
|
||||||
|
# Produces in outdir (default: alongside the source, in build/):
|
||||||
|
# figure.pdf vector master
|
||||||
|
# figure.svg vector, for slides/web
|
||||||
|
# figure.png white background, 300 dpi <- inspect this one
|
||||||
|
# figure-alpha.png transparent background
|
||||||
|
# figure-thumb.png 360 px wide <- inspect this for the color/hue audit
|
||||||
|
#
|
||||||
|
# The build FAILS on silent-corruption signals, not only on TeX errors:
|
||||||
|
# missing CJK glyphs and overfull boxes both produce figures that compile
|
||||||
|
# happily and read wrong.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SRC="${1:?usage: build.sh figure.tex [outdir]}"
|
||||||
|
[[ -f "$SRC" ]] || { echo "no such file: $SRC" >&2; exit 2; }
|
||||||
|
SRCDIR="$(cd "$(dirname "$SRC")" && pwd)"
|
||||||
|
BASE="$(basename "$SRC" .tex)"
|
||||||
|
OUT="${2:-$SRCDIR/build}"
|
||||||
|
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||||
|
mkdir -p "$OUT"
|
||||||
|
|
||||||
|
echo "==> xelatex $BASE"
|
||||||
|
# supertensor.sty lives in assets/; keep it off the user's texmf tree.
|
||||||
|
TEXINPUTS="$ROOT/assets:$SRCDIR:" \
|
||||||
|
xelatex -halt-on-error -interaction=nonstopmode \
|
||||||
|
-output-directory="$OUT" "$SRC" >/dev/null 2>&1 \
|
||||||
|
|| { echo "!! xelatex failed; last errors:" >&2
|
||||||
|
grep -n -A4 -m3 '^!' "$OUT/$BASE.log" >&2 || tail -30 "$OUT/$BASE.log" >&2
|
||||||
|
exit 1; }
|
||||||
|
|
||||||
|
LOG="$OUT/$BASE.log"
|
||||||
|
status=0
|
||||||
|
|
||||||
|
if grep -q "Missing character" "$LOG"; then
|
||||||
|
echo "!! missing glyphs (CJK font not applied?):" >&2
|
||||||
|
grep -m5 "Missing character" "$LOG" >&2
|
||||||
|
status=1
|
||||||
|
fi
|
||||||
|
if grep -qE "^(Overfull|Underfull) \\\\[hv]box" "$LOG"; then
|
||||||
|
echo "!! overfull/underfull boxes -- text is escaping its reserved lane:" >&2
|
||||||
|
grep -m5 -E "^(Overfull|Underfull) \\\\[hv]box" "$LOG" >&2
|
||||||
|
status=1
|
||||||
|
fi
|
||||||
|
if grep -q "Package supertensor Warning" "$LOG"; then
|
||||||
|
echo "!! supertensor warnings:" >&2
|
||||||
|
grep -m5 -A2 "Package supertensor Warning" "$LOG" >&2
|
||||||
|
status=1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if command -v pdftocairo >/dev/null 2>&1; then
|
||||||
|
echo "==> exporting svg / png"
|
||||||
|
pdftocairo -svg "$OUT/$BASE.pdf" "$OUT/$BASE.svg"
|
||||||
|
pdftocairo -png -r 300 -singlefile "$OUT/$BASE.pdf" "$OUT/$BASE"
|
||||||
|
pdftocairo -png -r 300 -singlefile -transp "$OUT/$BASE.pdf" "$OUT/$BASE-alpha"
|
||||||
|
pdftocairo -png -scale-to-x 360 -scale-to-y -1 -singlefile \
|
||||||
|
"$OUT/$BASE.pdf" "$OUT/$BASE-thumb"
|
||||||
|
else
|
||||||
|
echo "!! pdftocairo missing: PDF only, no SVG/PNG" >&2
|
||||||
|
status=1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "==> artifacts in $OUT"
|
||||||
|
ls -1 "$OUT/$BASE"*.{pdf,svg,png} 2>/dev/null | sed 's/^/ /'
|
||||||
|
if [[ $status -ne 0 ]]; then
|
||||||
|
echo "==> BUILD DIRTY: fix the warnings above before delivering." >&2
|
||||||
|
else
|
||||||
|
echo "==> clean. Now do the visual audit (references/checklist.md) --"
|
||||||
|
echo " a clean build says nothing about collisions or hue budget."
|
||||||
|
fi
|
||||||
|
exit $status
|
||||||
Executable
+54
@@ -0,0 +1,54 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Toolchain preflight for supertensor. Run this BEFORE drawing anything:
|
||||||
|
# it decides whether the TikZ path is available or the figure must fall back.
|
||||||
|
#
|
||||||
|
# ./scripts/preflight.sh human-readable report
|
||||||
|
# ./scripts/preflight.sh --quiet exit code only (0 = full TikZ path OK)
|
||||||
|
#
|
||||||
|
# Exit codes: 0 full path, 1 degraded (no CJK), 2 no LaTeX at all.
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
QUIET=0
|
||||||
|
[[ "${1:-}" == "--quiet" ]] && QUIET=1
|
||||||
|
say() { [[ $QUIET -eq 1 ]] || echo -e "$*"; }
|
||||||
|
|
||||||
|
ok=0; warn=0; fail=0
|
||||||
|
check() { # name, command
|
||||||
|
local name="$1"; shift
|
||||||
|
if "$@" >/dev/null 2>&1; then say " ok $name"; ok=$((ok+1)); return 0
|
||||||
|
else say " MISS $name"; return 1; fi
|
||||||
|
}
|
||||||
|
|
||||||
|
say "supertensor preflight"
|
||||||
|
say "--- required ---"
|
||||||
|
check "xelatex" command -v xelatex || fail=$((fail+1))
|
||||||
|
check "standalone.cls" kpsewhich standalone.cls || fail=$((fail+1))
|
||||||
|
check "tikz.sty" kpsewhich tikz.sty || fail=$((fail+1))
|
||||||
|
check "xstring.sty" kpsewhich xstring.sty || fail=$((fail+1))
|
||||||
|
|
||||||
|
say "--- chinese figures ---"
|
||||||
|
check "ctex.sty" kpsewhich ctex.sty || warn=$((warn+1))
|
||||||
|
check "fandol font" kpsewhich FandolSong-Regular.otf || warn=$((warn+1))
|
||||||
|
|
||||||
|
say "--- raster / vector export ---"
|
||||||
|
check "pdftocairo" command -v pdftocairo || warn=$((warn+1))
|
||||||
|
check "latexmk (optional)" command -v latexmk || true
|
||||||
|
|
||||||
|
if [[ $fail -gt 0 ]]; then
|
||||||
|
say ""
|
||||||
|
say "RESULT: no usable LaTeX path."
|
||||||
|
say "Fall back to an SVG or matplotlib figure and say so explicitly in the"
|
||||||
|
say "delivery; do not silently ship a lower-fidelity figure as if it were TikZ."
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
if [[ $warn -gt 0 ]]; then
|
||||||
|
say ""
|
||||||
|
say "RESULT: degraded."
|
||||||
|
say " - missing ctex/fandol -> English-label figures only; do not substitute"
|
||||||
|
say " an OS-specific CJK font without telling the user it costs portability."
|
||||||
|
say " - missing pdftocairo -> deliver PDF only, and say PNG/SVG were skipped."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
say ""
|
||||||
|
say "RESULT: full path available (TikZ + CJK + vector/raster export)."
|
||||||
|
exit 0
|
||||||
Executable
+28
@@ -0,0 +1,28 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Build every example and the smoke test. Any dirty build fails the run.
|
||||||
|
#
|
||||||
|
# ./scripts/test.sh
|
||||||
|
#
|
||||||
|
# This is a regression test for assets/supertensor.sty: the examples exercise
|
||||||
|
# faces, stacks, index faces, every pattern, shared caption lanes, connectors,
|
||||||
|
# the meaning box and the signature.
|
||||||
|
set -uo pipefail
|
||||||
|
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||||
|
fail=0
|
||||||
|
|
||||||
|
for f in "$ROOT"/tests/*.tex "$ROOT"/examples/*.tex; do
|
||||||
|
[[ -e "$f" ]] || continue
|
||||||
|
name="$(basename "$f")"
|
||||||
|
if "$ROOT/scripts/build.sh" "$f" >/dev/null 2>&1; then
|
||||||
|
echo " ok $name"
|
||||||
|
else
|
||||||
|
echo " FAIL $name"
|
||||||
|
fail=$((fail+1))
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
if [[ $fail -gt 0 ]]; then
|
||||||
|
echo "$fail failing figure(s); rerun scripts/build.sh on one to see why" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "all figures build clean"
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
\documentclass[border=8pt]{standalone}
|
||||||
|
\usepackage[cjk]{supertensor}
|
||||||
|
|
||||||
|
\stsetrole{x}{stTeal}
|
||||||
|
\stsetrole{w}{stOrange}
|
||||||
|
\stsetrole{h}{stViolet}
|
||||||
|
|
||||||
|
\stdim{m}{6}
|
||||||
|
\stdim{k}{4}
|
||||||
|
\stdim{n}{5}
|
||||||
|
|
||||||
|
\begin{document}
|
||||||
|
\begin{tikzpicture}
|
||||||
|
\stface[role=x, pattern=dense, bracket=true]{X}{(0,0)}{m}{k}
|
||||||
|
\stcaption{X}{$\mathbf X$}{$m\times k$}
|
||||||
|
|
||||||
|
\node[st op, right=4mm of X] (mul) {$\times$};
|
||||||
|
|
||||||
|
\stface[role=w, pattern=dense, bracket=true]{W}{($(mul)+(1.6,0)$)}{k}{n}
|
||||||
|
\stcaption{W}{$\mathbf W$}{$k\times n$}
|
||||||
|
|
||||||
|
\node[st op, right=4mm of W] (eq) {$=$};
|
||||||
|
|
||||||
|
\stface[role=h, pattern=dense, bracket=true]{H}{($(eq)+(1.8,0)$)}{m}{n}
|
||||||
|
\stcaption{H}{$\mathbf H$}{$m\times n$}
|
||||||
|
|
||||||
|
\ststack[role=x, pattern=causal]{S}{($(H)+(3.4,0)$)}{k}{k}{3}
|
||||||
|
\stcaption{S}{$\mathbf S$}{$b\times k\times k$}
|
||||||
|
|
||||||
|
\node[inner sep=0pt, fit=(X)(S)(H-shape)] (all) {};
|
||||||
|
\stmeaningbox{mb}{13cm}{all}
|
||||||
|
{$m$ 行、$k$ 收缩维、$n$ 输出维}
|
||||||
|
{$\mathbf X$ 激活值，$\mathbf W$ 权重，$\mathbf S$ 因果分数}
|
||||||
|
{$(m\times k)(k\times n)\rightarrow(m\times n)$}
|
||||||
|
\stsignature{Smoke test}{mb}
|
||||||
|
\end{tikzpicture}
|
||||||
|
\end{document}
|
||||||
Reference in New Issue
Block a user