Technical Field Note #21 · November 2026

Head pose from stereo alone during real work: rotation checked against the gyroscope, translation against the accelerometer

By Hoonhoe Kim, Co-Founder & CTO, Origin Data Lab

Field Note #21 · Camera 6DoF from a head-mounted stereo pair, checked with two independent IMU sensors (ODL-EGO S1)

Robot learning from egocentric video needs the camera's pose, not just the images, and pose estimates are rarely shown to be right on real work footage. We estimated the head-mounted camera's 6DoF motion from the stereo pair alone, with no IMU in the estimate, on three 60 s clips of real produce portioning on two days, and checked it against the rig's own gyroscope and accelerometer on the same clock. 2690 of 2694 steps (67 ms each) were solved. Per-step rotation agreed with the gyroscope to a median 0.04° (p95 0.21°), about 3% of a typical 1.4° step. Without loop closure the orientation difference accumulated to 3.6–7.5° over 60 s, once the gyroscope's own bias, measured at rest in Field Note #14, was removed; vision recovered that bias independently. For translation, we differentiated the stereo positions twice and compared them with the accelerometer after removing gravity: correlation 0.98 and a scale of 0.98 (vision/IMU), so the metric scale from the 65 mm baseline holds to within a few percent. The accelerometer checks the motion at 0.25–2 Hz; slow position drift is not checked.

Published November 2026⏱ 15 min readStatus Preliminary · automated analysisOpen data GitHub · Hugging Face · Kaggle
0.04°Rotation per step, vision − gyroscope, medianp95 0.21°; 2,690 steps of 67 msVerified (automated)
99.9%Steps solved2690 of 2694, 3 clips, 180 sVerified (automated)
r = 0.98Acceleration, vision vs accelerometerscale 0.98 (clips 0.97–0.99); same band-pass on bothVerified (automated)
3.6–7.5°Orientation drift over 60 sgyro bias removed; no loop closureVerified (automated)

Why this matters

A pose estimate is only useful if it has been checked

To learn manipulation from egocentric video, a model needs to know how the camera moved: where the hands and objects were in a fixed frame, not only in each image. That pose usually comes from visual SLAM or visual-inertial odometry, and it is rarely checked on real work footage, where the head turns fast, the hands fill the view and the scene is cluttered.

Our rig records two independent sensors on the same clock as the cameras: a gyroscope, which measures rotation directly, and an accelerometer, which measures acceleration plus gravity. If the pose comes from the images alone, both sensors become a free check: the gyroscope for rotation, the accelerometer for translation and its metric scale.

Setup

180 seconds of real work, vision only

Data: three 60 s clips from three recording sessions on 2026-11-02 and 2026-11-04 (bagging carrots, tying green onions, bagging vegetables), from the same five days of real produce portioning as Field Notes #18 and #19. One worker, rig worn on the head.

Estimate: both fisheye images rectified with the factory calibration to a pinhole view (f = 240 px at 800×600); Shi–Tomasi corners matched left to right by optical flow with a forward–backward check and triangulated with the 65.1 mm baseline; the same corners tracked to the next step and the camera motion solved by PnP with RANSAC; every second frame (15 Hz). The wearer's own blue jacket was masked. No IMU data went into the estimate, and there was no loop closure or bundle adjustment.

Frames were paired by exposure time, not by frame counter (Field Note #18); IMU and camera times were aligned with the factory camera–IMU time offset. For the comparison, the gyroscope's constant bias was removed using the value measured at rest in Field Note #14 (0.36°/s), nothing fitted to these clips.

Rotation

Vision and gyroscope agree to 0.04° per step

For each step, the rotation solved from the images was compared with the gyroscope integrated over the same exposure-centre interval and rotated into the camera frame with the factory camera–IMU calibration. 2690 of 2694 steps were solved; the rest had too few inlier points, mostly during fast head turns, and were left out (the pose was held).

The difference had a median of 0.04° and a 95th percentile of 0.21° (0.05° with the raw gyroscope), while a typical step turned the head 1.4° (the head turned 6,234° in total over the three clips). The two curves in the figure are hard to tell apart.

Errors add up without loop closure: over each 60 s clip the vision orientation moved 3.6–7.5° away from the bias-corrected gyroscope (2.0–4.7° after 30 s); about 5° of clip A's total came from four failed steps during one fast turn. Against the raw gyroscope the difference grows steadily to 14–21°, and that slope is the gyroscope's bias, not vision.

Vision also measures that bias: the constant rotation-rate offset between gyroscope and vision was 0.34–0.45°/s in the three clips, with the same sign on each axis as the 0.36°/s measured at rest a week or more earlier (Field Note #14). Short windows are reliable; minutes-long trajectories still need loop closure or IMU fusion.

Time series of per-step head rotation from the gyroscope and from stereo vision lying on top of each other, and a histogram of their difference with a median near 0.05 degrees
Figure 1. Left: rotation per 67 ms step in clip A, gyroscope (solid) and stereo vision (dashed). Right: vision − gyroscope difference for every solved step, three clips.
Three curves of accumulated orientation difference rising over 60 seconds
Figure 2. Accumulated orientation difference between vision and the integrated gyroscope from the start of each clip; solid: gyroscope bias from Field Note #14 removed, dashed: raw gyroscope.

Translation

Acceleration from the stereo positions matches the accelerometer (r = 0.98)

Translation cannot be checked against the gyroscope, and we had no external tracking. The accelerometer can check it indirectly: the second derivative of the camera position is its acceleration. We rotated the accelerometer into the vision frame with the vision orientation, removed gravity, and passed both signals through the same band-pass filter (1 s Savitzky–Golay smoothing minus a 4 s trend). The trend removal matters: without it, the slow orientation drift leaks a little gravity into the rotated accelerometer and lowers the agreement.

The two agreed in shape (correlation 0.98 over all axes and clips, best at zero time lag) and in size: the fitted scale, vision over accelerometer, was 0.98 overall and 0.97–0.99 per clip. The metric scale that the stereo baseline gives is therefore right to within a few percent for head motion at roughly 0.25–2 Hz.

This does not check slow drift of position over tens of seconds, which an accelerometer cannot see. The head-travel box from vision (about 1 m on a side for a worker standing at a bench) is plausible but unverified.

Time series of camera acceleration from the accelerometer and from the stereo positions following each other, and a scatter plot of the two clustered along the equality line
Figure 3. Left: acceleration on the axis with the most motion in clip A, both band-passed the same way. Right: all filtered samples, three axes, three clips; the line marks equality.
Head path drawn as a blue line through a sparse coloured point cloud of the work area, with camera frustums along the path
Figure 4. Clip A, first 36 s: head path and a sparse stereo point cloud (one keyframe per second), gravity-aligned with the accelerometer.
Table 1. Acceleration from the stereo positions matches the accelerometer (r = 0.98)
ClipDateTaskSecondsSteps solvedRotation diff., medianp95Drift at 30 sDrift at endAccel. correlationAccel. scale (vision/IMU)
A2026-11-04bagging carrots60894/8980.04°0.26°2.5°7.5°0.970.98
B2026-11-02tying green onions60898/8980.04°0.14°2.0°4.4°0.990.97
C2026-11-04bagging vegetables60898/8980.04°0.18°4.7°3.6°0.990.99

What it means

Usable for short windows; checked, not assumed

Verified: on this rig, stereo alone gives a camera rotation that matches the gyroscope to a few hundredths of a degree per step on real work, and a metric translation whose acceleration matches the accelerometer within a few percent.

For buyers: the IMU in every recording is enough to check any pose estimate you run on our data, with the scripts in this note. The same check applies to your own SLAM or VIO output.

Next: loop closure or visual–inertial fusion for minutes-long trajectories, and a tape-measured walking path to check position drift directly.

  • Per-step rotation: median difference to the gyroscope well under 0.1°.
  • Translation scale: within a few percent of the accelerometer.
  • Long trajectories: not yet; a few degrees per minute accumulate without loop closure.

SCOPE AND LIMITS

Limitations

What this note does not show, stated up front.

  • One rig, one worker, three 60 s clips (180 s) from two days of one kind of task; other tasks, lighting and head motion may differ.
  • The gyroscope bias was taken from static recordings a week or more earlier (Field Note #14) and assumed constant; any change in bias with temperature remains in the drift curves.
  • Translation is checked through acceleration at about 0.25–2 Hz only. Slow position drift, absolute position and the head-travel extent are not verified.
  • Steps that failed (4 of 2694) were left out of the statistics; they cluster in fast turns, so the error distribution is for solved steps.
  • Visual odometry only: no loop closure, bundle adjustment or IMU fusion. Better estimators exist; this note shows what the raw data supports and how to check any estimator.
  • Preliminary: the analysis was run by an AI-assisted automated pipeline and has not been independently verified by a third party or a second lab.
  • The trajectory figure shows a sparse point cloud of the workroom with the site owner's consent; no person is identifiable. Raw video is not published.

FAQ

Questions data buyers ask

Q01How accurate is stereo visual odometry on head-mounted egocentric video?
On 180 s of real produce-portioning work, per-step rotation matched the gyroscope to a median 0.04° (p95 0.21°), with 99.9% of 67 ms steps solved. Orientation drifted 3.6–7.5° over 60 s without loop closure.
Q02How can camera pose from egocentric video be validated without motion capture?
With the IMU recorded on the same clock: compare vision rotation with the integrated gyroscope, and the second derivative of vision position with the gravity-removed accelerometer after an identical band-pass filter.
Q03Is the metric scale of stereo visual odometry correct?
In this test, acceleration from the stereo positions matched the accelerometer with a scale of 0.98 (vision/IMU) and correlation 0.98, so the scale from the 65 mm baseline held to within a few percent.
Q04Why does visual odometry seem to drift against the gyroscope?
Partly because the gyroscope drifts: its constant bias (about 0.36°/s on this rig) integrates to about 20° per minute. With the bias removed, vision and gyroscope stayed within 3.6–7.5° over 60 s.
Q05Does the IMU need to be used in the pose estimate?
Not for these numbers: the IMU was used only to check. Fusing it would reduce drift and help during fast head turns, at the cost of losing it as an independent check.

OPEN DATA AND CODE

Reproduce and inspect the evidence

Every number in this note can be checked: derived tables, analysis code, figures and a runnable notebook are public.

GitHubAnalysis code and reproduction stepsorigin-data-lab/stereo-visual-odometry-head-pose-imu-validation-real-workView repository Hugging FaceDataset card, tables and figuresorigin-data-lab/stereo-visual-odometry-head-pose-imu-validation-real-workOpen dataset KaggleData explorer and runnable notebookorigindatalab/odl-stereo-vo-checked-with-imuExplore on Kaggle
Run the notebook on Kaggle

Licence: CC BY 4.0 (data and figures) · MIT (code)

Cite this note

Origin Data Lab (2026). Technical Field Note #21: Head pose from stereo alone during real work, checked against the gyroscope and accelerometer. https://origindatalab.io/technical-field-notes/stereo-visual-odometry-head-pose-imu-validation-real-work.html

CONTINUE READING

Related field notes

FIELD NOTE #20Stable is not the same as correct: a calibration file for the wrong resolutionFIELD NOTE #19Four days of real work: does a head-mounted stereo+IMU rig behave the same every day?FIELD NOTE #18Don't pair stereo frames by their counters: per-session offsets, true sync 15 µs

Want this measured on your data?

Origin Data Lab plans and runs egocentric, stereo+IMU and field video capture with documented QA, for physical AI and robotics teams.

Request sample dataContact usData services