Recalibration-Free Stereo PTU
Calibration and state estimation for a moving stereo pair
A stereo pan-tilt unit whose extrinsics stay trustworthy as it moves, so the rig does not need to be recalibrated by hand every time it is used.
Problem
A stereo rig on a pan-tilt mount loses its calibration the moment it moves. Servo backlash and drift accumulate, the extrinsics stop describing the actual geometry, and depth quietly degrades until someone recalibrates by hand.
Treating this as a state estimation problem rather than a maintenance chore is the point of the project.
What I built
The rig, and the estimator that keeps it honest.
- Stereo pair on a pan-tilt mount with MPU9250 IMU reference
- Laser anchor (VL53L0X) providing an absolute correction
- Six-state EKF with explicit backlash estimation and drift correction
- SGBM block matching fused with MiDaS monocular depth priors
- CAD, plus synthetic and real evaluation
- Reprojection analysis and ablation analysis
Keeping the extrinsics honest
The rig loses calibration the moment it moves. Rather than recalibrating by hand, the errors that cause the drift are estimated as part of the state.
Command a pan/tilt move
Servo backlash and drift accumulate
The extrinsics stop describing the actual geometry.
Six-state EKF
Backlash estimated explicitly rather than assumed away; IMU (MPU9250) provides the inertial reference.
Laser anchor correction
A VL53L0X range to a known anchor gives an absolute correction, not just a relative one.
Depth from stereo
SGBM block matching fused with MiDaS monocular priors.
Reprojection error stays bounded — no manual recalibration
The rig
| Part | Role |
|---|---|
| Stereo pair on a pan-tilt mount | The sensor whose extrinsics are the problem |
| MPU9250 IMU | Inertial reference for the filter |
| VL53L0X laser anchor | Absolute correction against a known point |
| Jetson Orin Nano | Runs estimation and depth |
| Six-state EKF | Backlash estimation and drift correction |
Experiments
Synthetic and real evaluation
- Question
- Does modelling backlash and drift keep the extrinsics valid without manual recalibration?
- Method
- Synthetic and real evaluation with reprojection error analysis, plus ablations isolating the contribution of each estimator component.
- Result
- Ablation analysis separates what the backlash estimation, drift correction and laser anchor each contribute.
Limitations and failure modes
- Evaluated on one rig; the backlash model is fitted to this mount's characteristics.
Gallery
Attribution
- Built by me
- Hardware, CAD, estimator and evaluation.
- Based on external research
- SGBM and MiDaS are upstream methods.
Related
- OpenDroneKit
Offline drone inspection — mission planning, flight, geospatial reconstruction and defect intelligence
- SMR300
Autonomy stack for a 300 kg industrial AMR