v12_gtmask — final report (2026-07-10)

Headline: v12_gtmask step 3000 bakes the GT off-frame fix into training itself. Result: 116 mm abs MPJPE / 34 px pixel err / 0.27 depth log-res on ARCTIC-lite (40 clips) — surpasses v11 + eval-time filter (124 mm / 39 px) and closes 48 % / 90 % of the gap to the Iter 206 oracle (54 mm / 0 px).

What v12 changed

v11: trained with hand_valid = URDS present flag (a world-3D flag). Learned to regress hand poses even when the GT wrist projected off-frame, because those "valid" GT joints were still supervising the model. Predicted 2D pixels landed in-frame; the resulting per-frame supervision was noise.

v12_gtmask: hand_valid is now present AND (GT wrist projects inside [-20, W+20] × [-20, H+20]). The 5-11 % of formerly-"valid" frames where GT is off-camera no longer supervise. On the worst-100 arctic clips, 21-58 % of frames were mis-labeled; those are all now excluded from training loss.

Everything else (init from v10 s3000, w_pix=10, w_s=2, LR schedule) is identical to v11.

v12 training arc — same-split ARCTIC-lite eval, N=40

Step	abs (mm)	rr (mm)	PA (mm)	wrist (mm)	pixel (px)	depth log-res
500	122	82	14	75	41	0.31
1000	119	82	14	72	38	0.31
1500	116	80	14	72	38	0.29
2000	119	80	14	73	36	0.27
2500	116	80	14	72	36	0.28
3000 (FINAL) 🥇	116	78	14	72	34	0.27

abs MPJPE: flat at 116 from step 1500 onward — plateau confirmed.
pixel err: strictly monotonically decreasing (41 → 34 = −17 %) across the entire arc. Training is still refining 2D supervision even after abs plateaus.
rr MPJPE: strictly monotonically decreasing (82 → 78 = −5 %) after step 1500. Orientation is still improving.
depth: flat at 0.27 from step 2000. Depth head fully converged.

Step 3000 is the first ckpt to be the leader (or tied) on every metric simultaneously.

v12 vs v11 — same eval, no filter

Metric	v11 s3000	v12 s3000	Δ
abs MPJPE	149 mm	116 mm	−22 %
rr MPJPE	87 mm	78 mm	−10 %
PA MPJPE	14 mm	14 mm	0 %
wrist trans	103 mm	72 mm	−30 %
pixel err	93 px	34 px	−63 %
depth log-res	0.28	0.27	−3 %

The training-time fix compounded better than the eval-time filter.
The single largest gain is on pixel error (−63 %). This makes sense: the noise removed by v12's hand_valid was primarily on the pixel head (which was regressing to random targets).
PA MPJPE unchanged: the label noise was on wrist / global position, not hand shape. PA-MPJPE is already at the shape floor (13 mm).

v12 vs (v11 + eval-time GT filter)

Metric	v11 + filter	v12 s3000	Δ
abs MPJPE	124 mm	116 mm	−6 %
pixel err	39 px	34 px	−13 %

The eval-time filter removed 11 % of frames from the metric. v12 removes those same frames from training, and additionally learns better on the remaining frames — hence beats the filter on the same eval set.

What v12 didn't fix

Absolute floor stayed at 116 mm. Not close to Iter 206's 54 mm. The remaining 62 mm gap is likely due to (a) rodrigues-MSE mean collapse on global_orient (see prior report), (b) monocular depth ambiguity on wrist_z, (c) the ~30-40 % of ARCTIC-lite worst clips that still have extreme trans errors.
Depth head plateaued at 0.27. Already flat by step 2000; needs a bigger head or better supervision (VGGT reuse per Iter 208 blueprint).
PA MPJPE unchanged at 14 mm. Already at shape floor.

Iter progression — the 12h arc, condensed

v9b (baseline)                       222 mm abs / 327 px pixel
  ↓ v10_scale — full 7-head training
v10 s3000 (FINAL)                    159 mm / 105 px
  ↓ v11_pushpix — w_pix=10, lower LR
v11 s3000 (FINAL)                    149 mm / 93 px
  ↓ diagnose per-source / best-worst / depth (findings: none)
  ↓ discover GT off-frame mislabel → apply as eval filter
v11 + eval-time filter               124 mm / 39 px  (−17% abs, −58% pixel!)
  ↓ push the fix into training: hand_valid = present AND in-frame
v12_gtmask s3000 (FINAL)             116 mm / 34 px  🏆

Total: v9b → v12 = 222 → 116 mm (−48 %) and 327 → 34 px (−90 %).

What's next

The Iter 206 oracle at 54 mm remains 62 mm out of reach. From this floor, the levers are (in expected impact order):

6D-rot MANO orient (v13, ~200 LoC). Fixes rodrigues-MSE mean collapse. Estimated −5 to −40 mm on abs — depends on how much of the 62 mm residual is orientation vs. depth.
Multi-resolution root_head (v14, ~300 LoC). Coarse-to-fine at higher resolution. Estimated pixel err −20 to −34 px (to Somantis territory).
Data pool: drop HO3Dv3 (data engine, ~30 LoC). Per-source finding showed HO3Dv3 is worse than any other source (255 mm vs 116 mm). Iter 204 flagged it as noise. Estimated −5 to −15 mm on abs.
VGGT point_head reuse (v15, ~500 LoC). Bigger + pretrained depth head. Estimated depth log-res 0.27 → 0.15-0.20.

Reproducibility

# 1. Train v12_gtmask (~3 hours on H200)
python data_engine/train_egowm_v10_scale.py \
    --out _ckpt/lane_b/v12_gtmask \
    --init-from _ckpt/lane_b/v10_scale/ckpt_step003000.pt \
    --steps 3000 --lr-heads 1e-4 --lr-trunk 5e-6 \
    --w-pix 10.0 --w-s 2.0

# The fix is inside data_engine/lane_b_dataset.py.LaneBCachedDataset:
#   hand_valid[t, hi] = base_valid AND (GT projects in-frame with 20 px margin)

# 2. Eval v12 s3000 (baseline eval; no additional filter needed)
python data_engine/eval_v10_scale.py \
    --ckpt _ckpt/lane_b/v12_gtmask/ckpt_step003000.pt \
    --n-clips 40 --source-filter arctic \
    --out benchmark/results/v12_gtmask_step3000.arctic-lite.json

# 3. Rebuild leaderboard + report
python benchmark/scripts/leaderboard.py
python report/build.py