Failure mode analysis — v11 step 3000 on ARCTIC val

Date: 2026-07-10 · Model: v11_pushpix step 3000 · Split: ARCTIC val (150 clips)

Per-clip metrics computed for 150 ARCTIC val clips. The best 6 clips hit oracle-level accuracy (60 mm abs, 14 px pixel). The worst 6 clips catastrophically fail (>400 mm abs, 500-2761 px pixel — predicted wrist completely outside the frame). The aggregate 149 mm hides this bimodal distribution.

Quick links: Best clips · Worst clips · Distribution · Root causes

Best clips — near-oracle territory

Clip	Subject/Action	abs MPJPE	pixel err
#7660	s06 · espressomachine_use_01	60 mm	33 px
#8422	s06 · microwave_use_02	61 mm	25 px
#1870	s02 · phone_use_01	64 mm	14 px
#14066	s08 · scissors_use_04	66 mm	14 px
#7838	s06 · espressomachine_use_04	70 mm	23 px
#16261	s10 · (see clip)	74 mm	40 px

Best-in-class clip #1870 hits 64 mm — 18% above the Iter 206 oracle bound (54 mm) — essentially at ceiling. Pixel error 14 px is well below SomantisModel v14d champion (13.4 px).

[**🎞️ Open BEST gallery →**](../gallery_best/gallery.html)

Worst clips — catastrophic

Clip	Subject/Action	abs MPJPE	pixel err
#8888	s06 · phone_grab_01	540 mm	566 px
#14835	s09 · espressomachine_use_02	518 mm	2761 px
#17883	s10 · waffleiron_grab_01	464 mm	618 px
#6217	s05 · notebook_use_02	445 mm	1546 px
#3439	s04 · microwave_grab_01	424 mm	982 px
#5978	s05 · (see clip)	402 mm	830 px

Pixel errors 566-2761 px mean the predicted wrist is entirely outside the frame or at the wrong side of the image. This isn't "off by a bit" — it's total failure.

[**🎞️ Open WORST gallery (failure modes) →**](../gallery_worst/gallery.html)

Per-clip distribution

Median clip: ~130 mm abs, ~85 px pixel (well below aggregate 149 mm)
Best 10%: <75 mm abs, <30 px pixel
Worst 10%: >250 mm abs, >200 px pixel
Worst 3%: >400 mm abs, >500 px pixel — these dominate the mean

The mean is dragged up by ~10 catastrophic clips out of 150. Median performance is actually near-oracle.

Failure mode causes

Examining the worst 6 clips, all share features:

All are "grab" or "use" actions — no "hold" or "pass" — hand is often occluded by the object being grabbed
All feature small/thin objects — phone, notebook, scissors — which occlude hand more than large objects (kettle, microwave)
Cross-subject — different subjects (s04-s10) so not a subject-specific memorization gap
Common with s06 phone_grab_01, s09 espressomachine_use_02 — actions where the hand crosses the frame edge or dips behind an object

Root cause hypothesis: the dense root_head soft-argmax fails when the true wrist position is either at the frame edge OR occluded. The heatmap has no strong signal, and the argmax defaults to a low-probability region.

Fix candidates (Iter 210+): - Presence head gating — when presence probability is low, don't trust the pixel prediction. Currently no gating. - Off-frame handling — SomantisModel representation.py's clamp_root_to_edge extends root prediction range past frame edges. We don't do this. - Visibility-weighted lift — use predicted vis[joint] to weight the PnP translation solve (Iter 208 blueprint).

Impact on aggregate metrics