Skip to content

README.md

Latest commit

 

History

History
208 lines (122 loc) · 3.09 KB

README.md

File metadata and controls

208 lines (122 loc) · 3.09 KB
Raw

📸 3D Reconstruction using Stereo Vision

🎯 Objective

This project implements a complete pipeline for 3D reconstruction from stereo images (left/right). It estimates depth (Z) and generates a 3D point cloud in millimeters.

🧠 Principle

The system is based on stereo vision:

  • Two images of the same scene are captured
  • Corresponding points are detected
  • The horizontal difference (disparity) is used to compute depth

[ Z = \frac{f \cdot B}{d} ]

Where:

  • Z = depth
  • f = focal length (pixels)
  • B = baseline (mm)
  • d = disparity

⚙️ Project Pipeline

1. 📷 Image Loading

  • Check file existence
  • Load images using OpenCV
  • Convert to landscape orientation

2. 📐 Camera Calibration

  • Load intrinsic matrix K

  • Convert portrait → landscape

  • Parameters:

    • focal lengths (fx, fy)
    • optical center (cx, cy)

3. 🔍 Feature Detection & Matching (SIFT + FLANN)

  • Detect keypoints using SIFT
  • Match descriptors using FLANN
  • Apply Lowe’s ratio test (0.70)

4. 🧮 Essential Matrix Estimation

cv2.findEssentialMat(...)
  • Computed using RANSAC
  • Encodes the geometric relationship between views

5. 🧭 Camera Pose Recovery

cv2.recoverPose(...)

  • Recovers:

    • Rotation R
    • Translation t

6. 🔄 Epipolar Rectification

cv2.stereoRectifyUncalibrated(...)
  • Aligns points horizontally
  • Reduces vertical disparity
  • Update intrinsic matrices:
K_rect = H @ K

7. 📉 Disparity Computation

disp = pts_l[:,0] - pts_r[:,0]
  • Horizontal difference between matched points

8. 📦 3D Triangulation

cv2.triangulatePoints(P1, P2, ...)

Using:

  • P1 = K[I | 0]

  • P2 = K[R | t]

  • Output is in homogeneous coordinates → converted to 3D

9. 📏 Real-World Scaling

scale = BASELINE / ||t||
pts3d *= scale
  • Converts reconstruction to real-world units (mm)

10. 🧹 Point Cloud Filtering

  • Remove:

    • points with Z < 0 (behind camera)
    • NaN / infinite values

  • Apply outlier filtering (3σ rule)

11. 📊 Visualization

  • 2D projections (X-Z, Z-Y)

  • 3D scatter plot

  • Export:

    • resultat_3d.png
    • nuage_points.ply

📁 Required Files

  • image_left_undist4.jpg
  • image_right_undist4.jpg
  • camera_K.npy
  • camera_dist.npy (optional)

▶️ Run the Project

python main.py

📌 Outputs

  • 3D point cloud

  • Generated files:

    • points_3d.npy
    • points_3d.txt
    • nuage_points.ply
    • resultat_3d.png

⚠️ Notes

  • Accuracy depends on:

    • image quality
    • calibration precision
    • number of matches

  • Translation vector t is normalized → scaling with baseline is required

🚀 Possible Improvements

  • Use full stereo calibration (stereoCalibrate)
  • Compute dense disparity maps (StereoBM / StereoSGBM)
  • Add color/texture to point cloud
  • Apply bundle adjustment for optimization

👨‍💻 Author

This project was developed as part of studies in computer vision and stereo vision.