Technical Field Note #10 · October 2026
Can you trust the clocks? Frame, stereo and IMU timing of a stereo+IMU egocentric rig, measured
Field Note #10 · Timestamp integrity and camera–IMU alignment (ODL-EGO S1 capture rig)
Robot-learning pipelines fuse video and inertial data on the assumption that every frame has a correct timestamp and that the two cameras and the IMU agree in time and orientation. We checked that assumption on 13 recordings (0.6 h, 67,746 stereo pairs) from one R8 stereo+IMU unit using the per-frame exposure timestamps embedded in the video stream. No frame was dropped, left and right exposures always started within 56 µs of each other, and the IMU ran gap-free at ~501.5 Hz on the same clock. A hand-held rotation session against a wall target recovered the camera–IMU rotation within 0.63° of the factory value and a time offset of 0 ms (factory: 0.66 ms). One caveat appeared: when the two cameras chose different exposure lengths, their exposure centres drifted apart by up to 1.2 ms.
Why this matters
Bad timestamps are silent
A dropped frame, a few milliseconds of left/right skew or an IMU gap do not show up when you watch a video, yet they corrupt stereo depth on moving hands, visual-inertial odometry and any action label aligned to sensor time. A data buyer usually cannot check this from delivered files alone.
This note documents what we check on every ODL-EGO S1 recording before delivery and what the numbers were on the 2026-10-26 calibration day.
Data
Where the timestamps come from
The R8 unit writes, for every video frame of each eye, a small header inside the H.264 stream (an SEI message) carrying the exposure start time, exposure length and a strobe sequence number on the device clock. The IMU file (ZXIMU v6, 40-byte records) carries gyro and accelerometer samples on the same clock. We extracted both for 13 recordings: 135,498 frames, 67,746 stereo pairs, 1,095,094 gyro and as many accelerometer samples.
One recording (145221, a distance session) arrived without its IMU file and is evaluated for video timing only.
Method
Four checks per recording
1) Frame continuity: interval between consecutive exposure centres (expected 33.33 ms) and strict +1 increments of the strobe sequence. 2) Stereo alignment: right minus left exposure start and exposure centre for every pair. 3) IMU integrity: sample intervals, record sequence breaks and whether the video lies inside the IMU time span. 4) Camera–IMU alignment: in a 76 s session the rig was rotated by hand in front of a wall-mounted target; we estimated camera rotation from the target pose (OpenCV PnP) between frames ~0.35 s apart, integrated the gyro over the same interval, and solved for the rotation that best maps gyro rotations onto camera rotations while sweeping the time offset from −40 to +40 ms.
\hat R_{ci} = \arg\min_R \sum_k \lVert \theta^{cam}_k - R\,\theta^{imu}_k(\delta t) \rVert^2Result 1
Frames: none missing, stereo exposures start together
All 13 recordings had exactly 33.33 ms between frames with no gap above 35.4 ms and a continuous strobe sequence: 0 dropped frames. Left and right exposure starts differed by at most 56 µs in every one of the 67,746 pairs.
Exposure centres differed by a median of 20.5 µs. In all afternoon recordings the maximum was 51 µs. In one morning recording facing a bright window the two cameras chose different exposure lengths (up to 2.4 ms apart), so the centres separated by up to 1.23 ms; 22% of that recording's pairs exceeded 80 µs.
| Recording | Frames L/R | Dropped | Strobe seq. continuous | L/R centre Δ median (µs) | L/R centre Δ max (µs) | IMU rate (Hz) | IMU max gap (µs) |
|---|---|---|---|---|---|---|---|
| 124235_00000 | 9000/9000 | 0 | yes | 20.5 | 44 | 501.50 | 2111 |
| 124235_00001 | 9000/9000 | 0 | yes | 20.5 | 470 | 501.50 | 2111 |
| 124235_00002 | 9000/9000 | 0 | yes | 21.0 | 532 | 501.50 | 2111 |
| 124235_00003 | 5625/5624 | 0 | yes | 21.5 | 1234 | 501.50 | 2111 |
| 142800_00000 | 7122/7122 | 0 | yes | 20.5 | 48 | 501.76 | 2110 |
| 143229_00000 | 9000/9000 | 0 | yes | 20.5 | 43 | 501.50 | 2111 |
| 143229_00001 | 6733/6734 | 0 | yes | 20.5 | 46 | 501.50 | 2111 |
| 145221_00000 | 2252/2252 | 0 | yes | 20.5 | 38 | IMU file not delivered | — |
| 145446_00000 | 1289/1288 | 0 | yes | 20.5 | 44 | 501.50 | 2106 |
| 145719_00000 | 2271/2271 | 0 | yes | 20.5 | 38 | 501.76 | 2109 |
| 145935_00000 | 2468/2467 | 0 | yes | 20.5 | 41 | 501.50 | 2110 |
| 150150_00000 | 1945/1946 | 0 | yes | 20.5 | 48 | 501.50 | 2111 |
| 150437_00000 | 2044/2045 | 0 | yes | 20.5 | 51 | 501.50 | 2109 |
Result 2
IMU: continuous and on the same clock as video
Every IMU file ran at ~501.5 Hz with a maximum interval of 2.11 ms and no record-sequence break. Video started 3.1–4.5 ms after the first IMU sample and ended inside the IMU span in every recording, so each frame has IMU coverage on both sides.
Result 3
Camera–IMU rotation and time offset agree with the factory file
Using 111 of 123 frame pairs (median rotation 31° per pair), the rotation residual was smallest at 0 ms and stayed within 10% of the minimum from -6 to +4 ms. The factory calibration states 0.66 ms. The estimated rotation is 0.63° from the factory rotation (bootstrap 95% interval 0.41–0.90°).
An earlier attempt on slow head motion (median ~6°/s) could not resolve the rotation; deliberate fast rotations (~30° between frames 0.35 s apart) were needed. We report that as a protocol lesson.
| Quantity | This measurement | Factory calibration file |
|---|---|---|
| Camera(left)–IMU rotation | 0.63° from factory (bootstrap 95%: 0.41–0.90°) | 0.25° from identity |
| Camera→IMU time offset | 0 ms (within 10% of minimum: -6 to +4 ms) | 0.66 ms |
| Rotation pairs used / total | 111 / 123 (median pair rotation 30.8°) | — |
| Median rotation residual | 0.80° | — |
Interpretation
What this means for users of the data
Verified: on this unit and day, frame timing, stereo exposure start and IMU continuity are clean, and the factory camera–IMU parameters are confirmed to within about half a degree and a few milliseconds.
Interpretation: the remaining timing risk is exposure-length mismatch between eyes under strong backlight. For fast hand motion, 1 ms of centre skew is about 1 mm of apparent disparity error at 1 m/s. We therefore lock exposure for production captures and flag recordings whose per-eye exposures differ.
- Ship per-frame exposure timestamps with every recording, not just frame indices.
- Flag recordings where left/right exposure lengths differ by more than 0.1 ms.
- Include a 60–90 s fast-rotation sequence in each calibration check.
Limitations
- Single R8 stereo+IMU unit, single indoor office, single day (2026-10-26); results are not a population estimate for other units.
- All measurements are automated (OpenCV detection, triangulation, fitting). No independent human re-measurement was performed: AI/algorithm-only preliminary analysis.
- The factory calibration file is third-party material; it is used for evaluation but is not redistributed. Raw videos are not published (they contain people and a private office).
- Timestamps are as reported by the device; there is no external reference clock, so absolute latency to the real world is not measured.
- The camera–IMU check uses one 76 s session and the left camera only; translation (lever arm) was not estimated.
- The time-offset resolution is about ±5 ms (flat minimum from -6 to +4 ms); the sub-millisecond factory value cannot be confirmed or rejected at that resolution.
- IMU noise parameters in the factory file are priors, not measured here.
Frequently asked questions
Does the ODL-EGO S1 rig drop frames?
No frame was dropped in 13 recordings (135,498 frames) checked by frame interval and strobe sequence on 2026-10-26.
How well are the two cameras synchronized?
Exposure starts differed by at most 56 µs across 67,746 pairs. Exposure centres can differ up to 1.2 ms when the cameras pick different exposure lengths, which we avoid by locking exposure.
Is the IMU on the same clock as the video?
Yes. Video frame timestamps and IMU samples share the device clock; video started 3–5 ms after the first IMU sample and every frame has IMU samples on both sides.
Was the factory camera–IMU calibration confirmed?
Our independent estimate differs from the factory rotation by 0.63° and finds a time offset of 0 ± ~5 ms, consistent with the factory 0.66 ms.
What motion is needed to check camera–IMU alignment?
Deliberate fast rotations about all three axes with a calibration target in view; slow head motion (~6°/s) was not enough in our test.
RESEARCH RESOURCES
Explore the Technical Resources
Access the code, datasets and evaluation materials.
Want this measured on your data?
Request a sample from our stereo+IMU or phone capture, or ask us to run the same checks on your rig.