EgoWM Lane B — Iter 208 v10 Scale Training

Date: 2026-07-10 · Author: Haoran Geng (@geng-haoran) with Claude Opus 4.7 (1M context) · Branch: h200_1_dev · Status: 🔄 training in progress

Full-scale training of the v10 architecture (all 7 heads) on the 33 077-clip all-7-dataset pool. Warm-started from v9b step 200. Uses per-head GT-availability masking so heads with missing GT for a given sample are silently skipped rather than corrupting the loss.

Quick links: Setup · Masking · Convergence · Metrics · Video · Overfit ↗ · Benchmark ↗ · Blueprint ↗

Setup

Model: v9b architecture + PointmapHead + PresenceHead + VisHead (18.4 M head params, 186 M trainable trunk params after unfreezing last-4 blocks + patch_embed).
Dataset: all-7-dataset pool — 29 770 train / 3 307 val clips (v8d's split).
Warm-start: v9b step 200 (hand_head + dense_root_head + lift_head already trained on ARCTIC; new heads fresh + zero-init).
Optimizer: AdamW, LR 3e-4 heads / 1e-5 trunk, weight decay 1e-3, linear warmup 100 steps.
Steps: 3000 (val every 250, save every 500, log every 20).
Compute: single GH200 96 GB, ~0.45 it/s → ~110 min total.

Per-head GT masking (the "nb" feature)

Each head only contributes to the loss on samples where its GT is available. Missing-GT contributions are exactly zero (not NaN), preserving gradient integrity.

Head	GT source	Mask condition	Empty-mask behaviour
`cam` (7-d pose)	URDS cam_pose	Always active	N/A
`hand` (61-d MANO)	URDS MANO + jc[:,0] wrist	`hand_valid[t, hi]`	return zero-loss
`dense_root` (u, v)	K · jc[:,0,:3] projected	`hand_valid` AND GT-pixel-in-frame AND pmap-finite	return zero-loss
`lift` (s_log)	log(gt_wrist_z / pmap_z_at_pixel)	Same as `dense_root`	return zero-loss
`presence`	`hand_valid` IS the GT	Always active	N/A
`vis` (per-joint)	Per-joint in-frame + finite	`hand_valid` AND per-joint in-frame	return zero-loss
`pointmap` (X, Y, Z, conf)	DA3 cache	`da3_finite AND z > 0`	pixel-level mask

This is critical for the all7 pool because different sources have different GT quality — e.g. AssemblyHands has ~29% NaN-orient frames from label noise (Iter 200 finding), HO3Dv3 GT depth may be unreliable (Iter 204 finding). Without per-head masking, those samples would poison the losses. With it, they simply contribute nothing.

Verified in smoke test: a batch with n_hv=0 (no valid hands) at step 20 produced loss_hand = 0.0, loss_pix = 0.0, loss_vis = 0.0 — clean zeros, not NaN.

Convergence

Training log (_ckpt/lane_b/v10_scale/train_log.jsonl) captures per-step {loss, hand, cam, pix, s, present, vis, depth, n_hv, n_uv}. Val runs every 250 steps on 16 clips.

Loss curves and val metrics filled in as ckpts are produced. Section will be regenerated at the end of the run.

Eval metrics

Ran eval_v10_scale.py on each ckpt as it saves, filtered to ARCTIC val for apples-to-apples comparison with the previous 3-way benchmark:

ckpt	abs MPJPE	rr MPJPE	PA MPJPE	wrist trans	pixel err (mean)	depth log-res
v9b step 200 (ref)	222.2	91.7	13.4	~130	327	—
v10 step 500	188.5	86.6	14.9	162	120	0.60
v10 step 1000	165.0	95.7	15.0	131	115	0.42
v10 step 1500	183.4	96.4	15.4	149	110	0.45
v10 step 2000	159.9	93.1	15.1	117	107	0.37
v10 step 2500	158.8	91.0	14.6	123	106	0.32
v10 step 3000 (final) 🏁	158.9	88.6	14.9	121	105	0.24

Step-500 first result (2026-07-10 06:14 UTC)

Just 500 steps of training and the pixel error dropped 327 → 120 px (−63 %), absolute MPJPE 222 → 189 mm (−34 mm / −15 %), root-relative 92 → 87 mm. Presence loss is essentially perfectly converged (val = 0.001). This confirms the overfit-test prediction: the dense root_head can pixel-localize on unseen clips too, given enough training. Extrapolating linearly, step 3000 should reach pixel_err ≈ 20-40 px and absolute MPJPE ≈ 100-130 mm.

Table populated as evaluation runs complete.

Before / after video

Rendered at end of training: 3-column comparison (GT / v9b step 200 baseline / v10 step 3000 final) on 3 ARCTIC val clips.

Placeholder while training runs.

What this test is designed to prove

The overfit sanity check was correct. If v10 can push val pixel error down significantly from v9b's 327 px, we've confirmed the pipeline scales.
Per-head masking is safe. If val metrics are clean (not corrupted by missing-GT rows), the mask logic is correct.
Depth head + presence + vis add value without breaking existing losses. Progression of the depth head's SSI residual, and of vis/presence toward near-zero, is a proxy for architectural soundness.
The all7 pool improves generalization. If v10 on ARCTIC-val beats v9b despite the trunk seeing 15× more diverse training data, scaling works.