v14 postmortem — Somantis-lift direction did not deliver

Date: 2026-07-11 · Verdict: v14 s3000 = 88 mm (v13 pipeline on v14 ckpt) vs. v13 s3000 = 90 mm. 2 mm improvement, entirely from continued hand_head training. The Somantis lift_v2_head + PnP pipeline did NOT work.

The v13 gap diagnostic picked Somantis-style 2D + PnP lift as the v14 direction (predicted 60-70 mm target). This report explains why it landed at 88 mm instead and picks the next direction from what we learned.

v14 s3000 numbers — three pipelines on the same ckpt

Eval pipeline	abs	rr	PA	wrist	pix (wrist)	depth
PnP-only (lift_v2_head + solve_transl)	641 mm	42	12	632	49 px	0.18
Hybrid (v14 wrist uv + v13 depth)	126 mm	42	12	107	49 px	0.18
v13-style (dense_root_head + v13 depth) 🥇	88 mm	42	12	67	35 px	0.18

Only the v13-style pipeline is competitive. The 2 mm improvement over v13 s3000 (90 mm) comes entirely from continued hand_head training during v14 (3000 more steps of rotation-matrix loss) — rr dropped 45 → 42.5 mm and PA dropped 12.8 → 12.3 mm. The new lift_v2_head added nothing usable.

Why PnP failed

init_transl estimates depth via weak-perspective:

z_init = fx * s_3D / s_2D

Where s_3D = mean joint distance from wrist in canonical J0 (~50 mm), s_2D = mean joint distance from wrist in predicted 2D (in pixels).

The pathology: at early training, joint SPREAD predictions are noisy. If s_2D is even slightly small (mode-collapsed joints), z_init blows up by 10-20×. Even with sanity clips (min_span_px=10, z in [0.05, 3.0]), 32/40 frames triggered fallback at s1000; at s3000 only 3/40 triggered but the successful PnP frames still landed with 632 mm mean wrist error.

Even a "clean" case: 24 px wrist error → PnP result 217 mm 3D wrist error (mostly depth: 780 mm predicted vs 573 mm GT). Depth is fundamentally under- determined by 2D landmarks alone in single-frame monocular; the leverage |dZ/dSpread| at typical hand scale is ~5×.

Iter 206's oracle achieved 54 mm because it used per-clip s* (median GT wrist depth over clip). Without that clip-level aggregation, PnP has no anchor.

Why the hybrid didn't win

hybrid = v14 wrist uv + v13 pointmap depth. This bypasses PnP entirely. Should be v13-equivalent if v14's wrist prediction is as good as v13's.

But v14 wrist uv error is 49 px vs v13's 35 px (both at s3000). The lift_v2_head spreads capacity across 21 joints; the dense_root_head is a wrist specialist. 3000 v14 steps produced weaker wrist prediction than 6000 combined steps (v9b→v10→v11→v12→v13) of specialized wrist training.

Result: 14 px more pixel error → ~10 mm more 3D error (at typical 0.5 m depth, 1 px ≈ 0.7 mm) → 88 → 126 mm.

What v14 DID improve

Continued matrix-hand-loss training for 3000 more steps helped the pose head:

metric	v13 s3000	v14 s3000 (v13-pipeline)	Δ
abs	90 mm	88 mm	-2 mm
rr	45 mm	42 mm	-3 mm
PA	12.8 mm	12.3 mm	-0.5 mm
wrist	67 mm	67 mm	0
pixel	35 px	35 px	0
depth	0.21	0.18	-0.03

Marginal but real. The depth head is still improving (pointmap keeps refining). The MANO pose is approaching but not quite at the PA floor.

The map of what still hurts

Given v14 s3000 = 88 mm and Iter 206 oracle = 54 mm, the 34 mm gap is composed of (rough decomposition, from Iter 205/206 ablations):

Wrist depth error — dominant contributor, ~20 mm. wrist_MPJPE = 67 mm total. Iter 205's per-clip s* recovers most of this.
Wrist pixel error — 35 px @ 0.5 m depth ≈ 10-12 mm.
Joint pose residual — rr − PA = 42 − 12 = 30 mm, contributes ~3-5 mm to abs after wrist correction.

Only 2 of these had a lever pulled in v14 (pose via rotation-matrix continued training; depth via pointmap refinement). Neither delivered enough. The biggest untouched levers are wrist pixel and per-clip s* aggregation.

Next direction candidates

Iter	Move	Predicted gain	Cost
v15a	Multi-res dense_root_head — wrist-only, 60×104 → 240×416 second pass with residual coarse-to-fine	pixel 35→15 px, abs -8 to -12 mm	250 LoC
~~v15b~~	~~Per-clip s* aggregation~~ — TESTED, DID NOT HELP	v13 per-frame 89.9 mm; per-clip-median 90.8 mm (+0.8); per-clip-hand-median 90.9 mm (+1.0)	tested
v15c	6D-rot for global_orient — replace axis-angle (still under matrix loss) with explicit 6D representation	rr -5 to -10 mm	100 LoC
v15d	Log_scale-based depth — use v14's converged log_scale for analytical depth (bypasses PnP)	unknown, quick test	30 LoC

v15b tested (per-clip s aggregation, 30 min, no training): No help — 89.9 → 90.8 mm. Model's per-frame s is already stable within a clip (much lower variance than we thought). Iter 205's per-clip s* stability was w.r.t. GT-derived s*, not model-predicted s. Recorded as null result.

Revised recommendation: v15a (multi-res dense_root_head) is now the sole viable next move. Predicted 5-12 mm on abs by cutting pixel error 35 → 15-20 px. Requires training (~1h) but reuses the v13 trainer recipe. If it delivers, v15a lands at ~75-83 mm. Beyond that, single-frame monocular depth ambiguity becomes the floor (needs temporal / clip-level modeling).

What v14 taught us that will inform v15+

PnP-derived depth is unsafe in this data regime. Any lift pipeline should use pointmap or log_scale as the depth anchor, not joint-spread ratio.
Per-joint dense soft-argmax is under-trained at 3000 steps. For 21 joints on 60×104 grid, need more steps or better inductive bias (e.g., temporal smoothing, MANO shape prior).
The v13 specialist head (dense_root_head for wrist) is the strongest component we have. Multi-res upgrade of it, not replacement, is the right move.
hand_head continued training still buys 2-3 mm every 3000 steps. Cheap. Should keep going.