Technical Field Note #08 · October 2026

How Much Egocentric Capture Is Usable? | Origin Data Lab

Integrity, task, sharpness and both-hands checks combined into frame yield, clip yield and capture hours per accepted hour — measured on 290 s of stereo and 132 s of phone capture

Buyers of egocentric robot-learning data pay per accepted hour, so the share of captured footage that survives acceptance sets the real cost. We measured it frame by frame on 12,651 frames: a full 290-second head-mounted stereo session and two 66-second phone recordings. Integrity checks removed almost nothing; sharpness and the both-hands rule did the filtering. Frame yield was 79% on the stereo rig and 57%–67% on the phones, but requiring clips of at least 3 seconds cut it to 46%, 12% and 43%. How hands are checked mattered as much as the footage: replacing a three-pass consensus with a single detector pass raised capture cost from 1.26 to 2.06 hours per accepted hour on the rig. Hand checks are AI-only and preliminary.

79% → 46%Stereo rig: frame yield → share in accepted clips ≥3 s (290 s session)

1.26 vs 2.06 hCapture hours per accepted hour on the rig: AI-consensus vs single-pass hand check

46% → 71%Rig 3 s clip yield when gaps of ≤6 frames (0.2 s) are tolerated

61%–89%Per-minute frame yield within one stereo session

Problem

Accepted Hours, Not Recorded Hours

Egocentric data for robot learning is increasingly bought per accepted hour. What a capture partner can deliver, and at what price, depends on the yield — the share of recorded footage that survives the buyer's checks.

\text{capture hours per accepted hour} = \frac{1}{\text{yield}}

Data

Three Recordings, Four Layers

Layer	Rule	Source	Status
Integrity	Valid flags, frame interval 16.7–50 ms, ≥3 IMU samples per exposure (rig) / gyro coverage ±16.7 ms (phone), stereo sync ≤1 ms	Device timestamps and IMU	Verified
Task	Not inside a non-task keystep	Field Note #05 annotation (phones only)	Human annotation
Sharpness	Relative Laplacian sharpness ≥ 0.6; gyro rule (≤1.5 px or ≤20 °/s) when no still-frame reference	Field Note #06 method	Verified measurement (proxy)
Both hands	Both hands present under AI consensus	Field Note #07 method	AI-only preliminary

Method

From Frames to Clips to Cost

accepted[t] = integrity_ok[t]                  # frame flags, timing, IMU coverage, stereo sync
            and not non_task[t]                # human keystep annotation (phones only)
            and rel_sharpness[t] >= 0.6        # vs low-motion frames within ±3 s; gyro rule if no reference
            and both_hands[t]                  # AI consensus: 3 MediaPipe passes, ±2 frames
clip_accepted = runs of accepted frames ≥ L seconds (optionally bridging gaps ≤ g frames)

Frames rejected by exactly one criterion are counted separately, to show which rule is binding.

Results

Integrity Is Free; Sharpness and Hands Do the Filtering

Grouped bar chart showing how the share of usable frames falls as integrity, task, sharpness, both-hands and clip-length rules are applied to three recordings — Cumulative share of frames that pass each rule, and the share inside accepted clips of at least 3 seconds. The rig session has no task annotation.

Recording	Frames	Integrity	+ On task	+ Sharp	+ Both hands	In clips ≥3 s
Stereo rig, full session	8,694	100.0%	n/a	90.2%	79.3%	45.8%
Phone · knife prep	1,979	99.7%	95.7%	66.0%	56.6%	11.7%
Phone · seasoning	1,978	99.6%	89.2%	69.8%	66.7%	42.9%

Stacked bars of frames rejected by exactly one criterion for three recordings — Frames rejected by exactly one criterion: the binding rule differs between the rig (hands) and the phones (sharpness).

Clips

The Acceptance Rule Moves Yield as Much as the Footage

Gap tolerance (frames)	Rig, clips ≥3 s	Knife prep, clips ≥3 s	Seasoning, clips ≥3 s	Rig, clips ≥5 s
0	45.8%	11.7%	42.9%	27.7%
3 (100 ms)	64.2%	17.8%	61.3%	49.6%
6 (200 ms)	70.5%	30.3%	62.0%	57.9%
15 (500 ms)	84.8%	63.3%	68.0%	80.4%

Cost

Capture Hours per Accepted Hour

Recording	No hand check	Hands: consensus	Hands: single pass	Clips ≥3 s, consensus	Clips ≥3 s, single pass
Stereo rig	1.11	1.26	2.06	2.18	29.37
Phone · knife prep	1.51	1.77	3.75	8.57	no clips
Phone · seasoning	1.43	1.50	1.69	2.33	4.30

Stability

Yield Changes Within a Session

Timeline of sharp, both-hands and accepted frame shares in 5 second bins over a 290 second stereo session — Stereo session in 5-second bins: sharp frames, both-hands frames, accepted frames, and frames inside accepted 3-second clips.

For Buyers

Five Terms to Fix Before Collection

Term	Why it matters (this note)
Frame vs clip acceptance	Rig: 79% of frames vs 46% in clips ≥3 s
Gap tolerance	Rig 3 s clips: 46% → 71% with 6-frame tolerance
Hand-check method	Rig cost: 1.26 vs 2.06 capture h per accepted h
Sharpness threshold	Rig frame yield 86% at 0.4 vs 60% at 0.8
Human sampling	Hand checks here are AI-only; misses depress yield (last minute of the session)

Evidence Status

What Is Verified, Estimated or Not Evaluated

Claim	Status
Integrity checks (flags, frame interval, IMU samples per exposure, stereo sync)	Verified — measured on every frame
Sharpness criterion (relative Laplacian sharpness ≥ 0.6)	Verified measurement; proxy for usability (Field Note #06)
Both-hands criterion	AI-only preliminary — MediaPipe consensus; per-hand recall vs AI review 92–99% (Field Note #07); no human verification
Task / non-task	Human annotation (phones only, Field Note #05)
Frame and clip yields, capture hours per accepted hour	Verified computation on the criteria above; inherits their status
Yield on other tasks, operators, sites	Not evaluated

Reproducibility

Re-running the Analysis

python analysis/yield_analysis.py analysis/data/yield_frames.csv OUT_DIR
# OUT_DIR/yield_results.json is byte-for-byte comparable with analysis/yield_results.json

Limitations

Hand checks are AI-only (MediaPipe consensus); no human verification. Missed hands lower the measured yield, as the last minute of the stereo session shows.
About 7 minutes of footage from three recordings; two task families; yields are method demonstrations, not production rates.
Sharpness is a proxy (relative Laplacian variance on H.264 video); it does not measure the effect on a specific model.
The stereo session has no task annotation, so its yield excludes the task filter.
Device, mounting, task and operator differ between the rig and the phones; differences are observations, not causal comparisons.
Thresholds (0.6 sharpness, ±2-frame consensus, clip lengths, gap tolerance) are illustrative; buyers' acceptance rules may differ.

Frequently asked questions

What share of egocentric capture is usable for robot-learning data?

It depends on the acceptance rule. On our recordings, 79% of frames from a head-mounted stereo rig and 57%–67% from two phone recordings passed integrity, task, sharpness and both-hands checks. Requiring clips of at least 3 s cut that to 46%, 12% and 43%. Hand checks are AI-only and preliminary.

How many capture hours does one accepted hour cost?

At frame level with a 0.6 sharpness threshold and AI-consensus hand checks: 1.26 capture hours per accepted hour on the stereo rig and 1.50–1.77 on the phones. With a single hand-detector pass instead, the same footage costs 2.06 and 1.69–3.75 hours.

Why does the hand-detection method change the cost so much?

A single MediaPipe pass misses many visible hands (Field Note #07), so more frames fail the both-hands rule. Because one miss also breaks a clip, the effect is larger for clips: on the stereo rig, 3-second clip yield was 46% with consensus detection and 3.4% with a single pass.

Should acceptance rules allow short gaps inside clips?

Our data suggest the rule matters as much as the footage. Bridging rejected gaps of up to 6 frames (0.2 s) raised 3-second clip yield on the stereo rig from 46% to 71%. Bridged frames are themselves below threshold, so the tolerance should be agreed explicitly with the buyer.

Which criterion rejects the most footage?

Sharpness on the phone recordings (22%–31% of on-task frames failed) and the both-hands rule on the stereo rig. Integrity checks rejected 0% of rig frames and about 0.3% of phone frames (the last ~0.2 s without IMU and one long frame interval).

Does yield stay constant during a session?

No. Per-minute frame yield on the 290-second stereo session ranged from 61% to 89%. The lowest minute coincided with many missed hands while the wearer held a drill; an AI visual spot check of six frames from that minute saw both hands in all six, while the consensus missed both hands in three.

Are these yields guaranteed for a production project?

No. They come from three recordings (about 7 minutes), one task family per recording and AI-only hand checks. They demonstrate the method; production yields must be measured on the buyer's tasks and agreed acceptance rules.

RESEARCH RESOURCES

Explore the Technical Resources

Access the code, datasets and evaluation materials.

GitHubSource code & reproducibilityExplore resource ↗ Hugging FaceDataset & documentationExplore resource ↗ KaggleData Explorer & notebookExplore resource ↗