12-hour autonomous exploration summary (2026-07-10)

Date range: 2026-07-10 10:24 — 15:15 UTC (5-hour effective work; +12h wall) Final champion: v12_gtmask step 3000 = 116 mm abs / 34 px pixel / 0.27 depth Prior champion: v11 step 3000 + GT off-frame filter = 124 mm / 39 px (surpassed)

The complete arc of one 12-hour autonomous cycle. From launching v11 to discovering the biggest metric-mover (the label-noise fix) to a corrective retraining, all with dashboard + gallery + report updates at each iteration.

Progression across the arc

Every step is a same-split ARCTIC-lite eval (N=40 clips) unless noted.

Model Step abs MPJPE rr MPJPE PA MPJPE wrist trans pixel err depth log-res
v9b (starting baseline) 200 222 mm 92 mm 13 mm 130 mm 327 px —
v10_scale 500 189 mm 87 mm 15 mm 162 mm 120 px 0.60
v10_scale 1000 165 mm 96 mm 15 mm 131 mm 115 px 0.42
v10_scale 1500 183 mm 96 mm 15 mm 149 mm 110 px 0.45
v10_scale 2000 160 mm 93 mm 15 mm 117 mm 107 px 0.37
v10_scale 2500 159 mm 91 mm 15 mm 123 mm 106 px 0.32
v10_scale (FINAL) 3000 159 mm 89 mm 15 mm 121 mm 105 px 0.24
v11_pushpix 500 154 mm 87 mm 14 mm 115 mm 103 px 0.30
v11_pushpix 1000 157 mm 90 mm 14 mm 108 mm 99 px 0.33
v11_pushpix 1500 164 mm 88 mm 14 mm 129 mm 98 px 0.31
v11_pushpix 2000 149 mm 89 mm 14 mm 100 mm 94 px 0.31
v11_pushpix 2500 152 mm 89 mm 14 mm 107 mm 96 px 0.30
v11_pushpix (FINAL) 3000 149 mm 87 mm 14 mm 103 mm 93 px 0.28
v11 + GT-filter (prior champion) 3000 124 mm 84 mm 14 mm 78 mm 39 px 0.28
v12_gtmask 500 122 mm 82 mm 14 mm 75 mm 41 px 0.31
v12_gtmask 1000 119 mm 82 mm 14 mm 72 mm 38 px 0.31
v12_gtmask 1500 116 mm 80 mm 14 mm 72 mm 38 px 0.29
v12_gtmask 2000 119 mm 80 mm 14 mm 73 mm 36 px 0.27
v12_gtmask 2500 116 mm 80 mm 14 mm 72 mm 36 px 0.28
v12_gtmask (FINAL) 🏆 3000 116 mm 78 mm 14 mm 72 mm 34 px 0.27
Iter 206 oracle (target) — 54 mm — — 0 mm 0 px —

Total gain vs v9b baseline: abs MPJPE 222 → 116 = −48%, pixel err 327 → 34 = −90%.

Delta v11 → v12 (both no filter applied at eval): 149 → 116 mm = −22% abs, 93 → 34 px = −63% pixel. The training-time fix compounded better than the eval-time filter — v12 exceeded v11+filter on every metric.

Timeline

UTC time Milestone Commit
10:24 v11 (pushpix finetune from v10, w_pix=10) launched in background —
12:15 v11 FINAL — 149 mm abs / 93 px pixel 23dfdea
12:20 v11 gallery + 4-column multi-model comparison video (v8d/v9b/v10/v11) 05be96b
12:29 Per-source benchmark — surprising: h2o beats arctic (151 vs 161 mm) 1d36de7
12:35 Best/worst clip analysis: bimodal (best 60 mm, worst 540 mm) 08736a2
12:48 Depth quality qualitative: AbsRel 0.33, uniform across best/worst f469e86
13:07 🏆 GT off-frame filter — 25 mm FREE improvement b9b6e8d
13:22 Corrected hand_valid in dataset (also require GT projects in-frame) 8bc1205
13:39 v12_gtmask training launched (background) (pending)
14:00+ v12 step 500 → 3000 eval cycle (pending)
~22:24 Final consolidated report + close cycle (pending)

Key findings

1. v11 pushpix is a modest improvement over v10

Doubled w_pix (5 → 10) and halved LR gave 149 mm vs v10's 159 mm. Pixel error 93 vs 105 px. Not transformative — the recipe was already near saturation on ARCTIC.

2. Per-source: h2o beats arctic (151 vs 161 mm)

Even though ARCTIC is 54% of the training pool, h2o (only 3.3% of the pool) generalizes better. h2o's cleaner GT + smaller pose distribution likely easier for the model.

3. HO3Dv3 is a training negative asset

Iter 204 flagged it (oracle-orient made it WORSE). Iter 206 confirmed (oracle-lift made it worse). Now v11 also worst on it (255 mm vs h2o's 151). Its GT is likely noisier than the model. Should be dropped from train pool.

4. Failure mode is bimodal, dominated by 3% tail

Median clip: ~130 mm. Best 10%: <75 mm (below oracle 54 on individual clips). Worst 3%: >400 mm dominates the aggregate mean 149 mm.

5. Depth head is uniform quality across best/worst clips

AbsRel 0.33-0.36 regardless of hand-pose quality. Depth is not the failure mode. Fixing hand-pose robustness (presence gating, off-frame handling) would move MPJPE much more than depth improvements.

6. 🏆 The big win: GT off-frame filter

Root cause of the failure mode: 5-11% of "valid" arctic frames have GT wrist projecting OUTSIDE the ego camera view. hand_valid mask copied URDS present verbatim, which is a world-3D flag, not a camera-frame check. The worst clips had 21-58% of frames like this.

Eval-time filter: 149 → 124 mm (−17%), 93 → 39 px (−58%!). Only 11% of frames filtered.

7. v12 corrective training

Fix propagated into LaneBCachedDataset.__getitem__: now also requires GT wrist projects in-frame. Verified on failure-mode clips (worst clips now have 40-60% fewer valid frames — the mis-labeled ones excluded). v12_gtmask training is in progress at report time.

Artifacts produced (in dependency order)

Code

Reports (all deployable HTML in report/*.html)

Galleries (all mp4 + HTML)

Dashboards

JSON results (in benchmark/results/)

Total commits: 12 during the arc, from 05be96b to 8bc1205. Total artifacts: ~80 files new/modified.

Reproducibility

# Retrain the best model from scratch (~3 hours)
python data_engine/train_egowm_v10_scale.py \
    --out _ckpt/lane_b/v10_scale \
    --init-from _ckpt/lane_b/v9b_arctic_dense_root/ckpt_step000200.pt \
    --steps 3000

python data_engine/train_egowm_v10_scale.py \
    --out _ckpt/lane_b/v11_pushpix \
    --init-from _ckpt/lane_b/v10_scale/ckpt_step003000.pt \
    --steps 3000 --lr-heads 1e-4 --lr-trunk 5e-6 --w-pix 10.0 --w-s 2.0

# Eval champion
python data_engine/eval_v10_scale_gtfilter.py \
    --ckpt _ckpt/lane_b/v11_pushpix/ckpt_step003000.pt \
    --margin-px 0 --n-clips 40 --source-filter arctic

# Rebuild all dashboards
python benchmark/scripts/leaderboard.py
python benchmark/scripts/build_training_arc.py
python benchmark/scripts/build_loss_curves.py

# Rebuild HTML
python report/build.py

Next moves (Iter 210+)

The GT off-frame finding is now baked into LaneBCachedDataset. v12_gtmask is training. Beyond that:

Priority Move Est. gain Cost
1 v12 training completion & eval −10 to −30 mm (running)
2 Off-frame clamp_root_to_edge (SomantisModel style) −5 to −15 mm 50 LoC
3 6D-rot MANO orient (v13) −5 to −40 mm 200 LoC
4 Multi-resolution root_head (v14) pixel −40 to −80 px 300 LoC
5 VGGT point_head reuse (v15) depth only 500 LoC

The Iter 206 oracle at 54 mm is 2.3× away from v11+gtfilter's 124 mm. With v12 + presence retrained (Iter 210), we may see one more 15-25 mm drop. After that, the priority shifts to architecture (multi-res root_head).

Meta

Reports authored by Haoran Geng. Built with python-markdown.