TECHNICAL FIELD NOTE #03 · OCTOBER 2026

Stereo + IMU Egocentric Capture

Based on a 120-second recording of a real manual task captured with a head-mounted stereo camera and six-axis IMU, we examine left–right frame correspondence, exposure-midpoint timing, IMU continuity, camera–IMU time-offset estimation, and calibration limitations.

This is a technical evaluation of one device in one capture mode. It does not constitute final approval of production-grade stereo calibration or precise camera–IMU synchronization.

Synchronization, Timing and Calibration Evaluation

Capture Configuration

We recorded a two-handed case-assembly task at an indoor workbench using a power drill and hand tools. Each left and right video stream contains 3,600 frames at 1600×1200 and 30 fps, sharing device frame ticks. Exposure start time and duration are recorded for each frame in H.264 SEI messages.

Figure 1. Synchronized left and right frames at frame 1,500. The monitor in the workspace has been blurred for public release.
Figure 1. Synchronized left and right frames at frame 1,500. The monitor in the workspace has been blurred for public release.

Exposure Midpoints, Not Frame Ticks

A camera frame integrates light over an exposure interval rather than representing a single instant. Due to automatic exposure, exposure durations in this recording range from approximately 8.1 to 12.0 ms. Therefore, video and sensor timing should be compared using theexposure midpoint (exposure start + half the exposure duration). MP4 presentation timestamps (PTS) occur at regular 1/30-second intervals and are not a substitute for the device-recorded exposure timestamps.

Figure 2. Left and right exposure durations and exposure-midpoint differences across 3,600 frame pairs. The p1–p99 range is 16.5–31.0 µs.
Figure 2. Left and right exposure durations and exposure-midpoint differences across 3,600 frame pairs. The p1–p99 range is 16.5–31.0 µs.

Left–Right Stereo Synchronization Checks

CheckMethodResultStatus
Frame ticksSEI tick comparison3,600/3,600 pairs matched; 33,333 µs intervalVerified
Frame continuityStrobe counterIncrement of +1 per frame on both sides; zero missingVerified
Visual frame correspondenceCorrelation of global image motion between viewsPeak at zero-frame lag; r=0.997Verified
Exposure-midpoint differenceComparison of device-reported valuesMedian 20.5 µs; mean 21.1 ± 2.9 µsDevice-reported

Multiple independent checks support frame-level correspondence. However, microsecond-scale exposure differences are firmware-reported values and have not been independently verified from 30 fps video alone. External validation would require, for example, a flashing LED visible to both cameras.

IMU Stream

The six-axis IMU was sampled at an average rate of approximately501.7Hzin a recording that uses a shared clock with the video. No interval outliers, duplicates, missing samples, or sensor saturation were observed within the tested scope. IMU coverage begins approximately 48 ms before the first video exposure and ends approximately 48 ms after the last exposure.

Figure 3. Raw three-axis gyroscope and accelerometer signals. Mean acceleration magnitude: 9.893 m/s².
Figure 3. Raw three-axis gyroscope and accelerometer signals. Mean acceleration magnitude: 9.893 m/s².
Figure 4. IMU sample-interval distribution and head angular velocity.
Figure 4. IMU sample-interval distribution and head angular velocity.

Target-Free Camera–IMU Time-Offset Estimation

Even with a common device clock, relative delays may arise along the camera and IMU processing paths. Weestimatedthe offset by comparing visually derived head rotation with gyroscope measurements.

  1. Computed inter-frame motion at 400×300 resolution using Lucas–Kanade optical flow and a RANSAC similarity transform.
  2. Averaged gyroscope signals over intervals between exposure midpoints.
  3. Fitted a 3×3 linear mapping and bias between visual motion and angular velocity using least squares.
  4. Searched −6 to +6 ms in 0.1 ms increments and checked stability across six 30-second windows.
Figure 5. Comparison of video-estimated head rotation and gyroscope signals. Per-axis R²: 0.971 / 0.972 / 0.979.
Figure 5. Comparison of video-estimated head rotation and gyroscope signals. Per-axis R²: 0.971 / 0.972 / 0.979.
Figure 6. Estimated offsets in 30-second windows. Between-window variation is within 0.8 ms.
Figure 6. Estimated offsets in 30-second windows. Between-window variation is within 0.8 ms.
MetricLeft cameraRight camera
Best full-recording offset−1.1ms−1.2ms
Fit R²0.9730.974
30-second window range−1.5~−0.7ms−1.5~−0.8ms

These results support the possibility of camera–IMU alignment within a few milliseconds for this device and mode, but do not establish a final production time offset. Manufacturer calibration performed in a different sensor mode reports +0.89/+0.95 ms; the discrepancy remains unresolved. Update (23 October 2026): A later comparison against Kalibr calibration on another recording found a several-millisecond disagreement. These estimates are not a validated production camera–IMU time shift.

Calibration Finding

Stereo Calibration Must Match the Sensor Mode

The calibration file supplied with the device reports an excellent stereo RMS of 0.362 px. Its principal points, however, sit near the centre of a 1920×1080 (16:9) image, while this recording used a 1600×1200 (4:3) mode. Applied to the recorded frames, the parameters leave a median vertical residual of 3.409 px (p95 10.388 px) between matched left/right points after rectification, on 1,496 points from four held-out timestamps.

Scatter map of vertical rectification residuals across the image and residual versus horizontal position
Figure 7. Vertical residual after rectification with the supplied parameters. The error varies systematically across the image (about −10 px on the right, +10 px lower left) — the signature of intrinsics from a different sensor mode, not noise.
Bar chart of median and 95th percentile vertical residual for supplied parameters and a diagnostic rotation
Figure 8. Held-out vertical residual. The 3-DOF diagnostic rotation reduces the median to 0.570 px, but it does so mostly through a −1.41° rotation about the vertical axis that shifts right-image points by −20.9 px horizontally and changes disparity.
LESSONA low calibration RMS says nothing about whether the file matches the resolution, crop or binning you recorded in.
LESSONCheck rectification on your own frames with held-out matches and look at the spatial residual map, not only the median.
LESSONA small correction that lowers vertical residual can still corrupt depth if it moves points horizontally.
LESSONCalibrate per device and per sensor mode, and recalibrate when the mode or firmware changes.
Update, 23 October 2026

A Kalibr calibration produced in the same 1600×1200 capture mode was later validated on this recording without any adjustment: median angular epipolar error 0.41 px over the full field of view, with a stereo-rotation correction below 0.02°. The method and results are in Technical Field Note #04.

Evidence Status

What Is Verified, Estimated or Deferred

ItemResultStatus
Video decode and format0 errors · 1600×1200 · 30/1 fps · 3,600 frames per channelVerified
Source traceabilityStream copy · frame hashes and per-frame SEI preserved 3,600/3,600Verified
L/R frame correspondenceFrame tick 100% match · 0 counter gaps · motion correlation peak at lag 0Verified
IMU continuity and units501.7 Hz · 0 gaps / duplicates · 0 saturationVerified
Video–IMU coverageIMU spans 48 ms before first and after last exposureVerified
Parsing reproducibilityDecoded CSVs regenerate byte-for-byte from raw filesVerified
L/R exposure synchronizationMedian 20.5 µs (firmware record)Device-recorded
Camera–IMU time offset−1.1 / −1.2 msEstimated
Stereo epipolar consistency (this footage)Mode-matched Kalibr parameters: 0.41 px median angular epipolar residual on 19,386 unfiltered matches (Field Note #04). Baseline scale and metric depth not verified.Verified on evaluated footage
Camera–IMU extrinsics and time shiftKalibr calibration available; not verifiable on this recording without a targetKalibr-calibrated

What This Evaluation Does Not Claim

This evaluation does not claim synchronization performance across all devices or modes, externally validated microsecond-level exposure simultaneity, production acceptance of stereo calibration, finalized camera–IMU extrinsics, or improved robot-learning performance.

Citable Technical Findings

Origin Data Lab evaluated a 120-second head-mounted stereo + IMU egocentric recording (2 × 1600×1200 at 30 fps; six-axis IMU). All 3,600 stereo frame-tick pairs matched; the device-reported median exposure-centre difference was 20.5 µs; IMU sampling averaged 501.7 Hz. The video–gyro time shifts of −1.1/−1.2 ms are estimates, not validated production offsets. A mode-matched stereo calibration achieved 0.41 px median angular epipolar residual on 19,386 unfiltered SIFT matches; this verifies epipolar consistency on the evaluated footage, not baseline scale, metric depth, or camera–IMU extrinsics.

How to Cite

Origin Data Lab (2026). “Stereo + IMU Egocentric Capture: Synchronization, Timing and Calibration,” Technical Field Note #03. Original technical note.

Related Technical Field Notes

Field Note #01: Video + IMU Capture Evaluation · Field Note #02: Egocentric RGB + IMU Task Sample · Field Note #04: Checking a Stereo Calibration on Ordinary Footage

Technical Evidence

Explore our public stereo and IMU evaluation evidence.

🤗 Hugging Face Evidence GitHub GitHub Evidence

Request the Full Evaluation Package

Research institutions and robotics teams may inquire about access conditions for the source bitstreams, per-frame device timestamps, raw IMU measurements, decoders, validation scripts, QA records and technical evaluation materials.

Request Evaluation Package →