360Cam-PGM-3DGS-Tools

A consolidated toolkit for turning 360° video captures into photogrammetry (PGM) datasets and optimized point clouds that seed 3D Gaussian Splatting (3DGS) projects. The scripts cover every stage from frame extraction to PLY refinement, and the desktop GUI orchestrates the full flow for multiple photogrammetry and 3DGS apps.

Project status: actively edited and expanded. Expect interface polish, richer presets, and additional documentation updates in upcoming revisions. The code in this repository was generated with the assistance of OpenAI Codex.

---

Installation & Environment Setup

Requirements

• Python 3.7 or newer
• pip for installing Python packages
• Python dependencies listed in requirements.txt
• FFmpeg available on your PATH
• PyTorch (torch 1.10+ and matching torchvision build) for the human masking tool
• GPU/CUDA is optional for masking, but accelerates large batches.

Setup Steps

1. Clone the repository

   git clone https://github.com/Mistral-Yu/360Cam-PGM-3DGS-Tools.git
   cd 360Cam-PGM-3DGS-Tools

2. Create and activate a Conda environment

   conda create -n gs360 python=3.7
   conda activate gs360

   Tip: pick any supported Python version (3.7+) that matches your GPU drivers/CUDA toolkits.

3. Install Python dependencies

   pip install -r requirements.txt

   Note: PyTorch wheels are platform-specific. If pip cannot find a build for your OS/Python combo, install torch/torchvision manually inside the Conda environment before rerunning the command.

4. Verify FFmpeg

   ffmpeg -version

---

Workflow at a Glance

GUI Explanation: gs360_360GUI.py (Tabs: Video2Frames, FrameSelector, 360PerspCut, SegmentationMaskTool, PointCloudOptimizer, MS360xmlToPerspCams).

• Video2Frames tab: Sample equirectangular frames from a 360° video.
• FrameSelector tab: Score and retain the sharpest frames for Structure-from-Motion (SfM).
• 360PerspCut tab: Convert panoramas into perspective views.
• SegmentationMaskTool tab: Use SegmentationMaskTool to preview and refine masks for unwanted subjects or rig elements before reconstruction to reduce ghosts and cleanup artifacts.
• Use exported views to align in your photogrammetry software (RealityScan, Metashape, etc.) and export the PLY point cloud plus camera metadata.
• PointCloudOptimizer tab: Optimize initial point cloud, Downsample/merge the PLY or Colmap PointCloud3D for your 3DGS tool (PostShot, gsplat, etc.) . Delete sky point clouds generated with photogrammetry software, add clean Sky point clouds.
• MS360xmlToPerspCams tab: Experimental tool for convert Metashape spherical camera XML into perspective camera parameters. Colmap, Metashape xml, transform.json, RealityScan xmp.
• DualFisheyePipeline tab: Experimental tool for correcting dual-fisheye distortion from Metashape XML calibration and exporting perspective views.

Workflows (GUI)

Rapid

1. Create a 360 video using DJI Studio or similar software (This GitHub repository has been tested with the Osmo 360. Recommended settings: apply D-Log only, keep all other settings at their default value of 0 if correct color values are required under a linear workflow. And turn RockSteady off.)
2. Launch gs360_360GUI.py.
3. In the 360PerspCut tab, click Browse Video and select a video file.
4. Choose a preset.
5. Under Video (direct export), set the FPS.
6. Click Run Export to write the images.
7. RealityScan: launch the app, import images, then select all. Set Prior Calibration -> Prior to Fixed or Prior to Approximate, and change Focal Length to the value shown in the 360PerspCut log (Preset default: 12 mm, fisheyelike: 17mm, full360coverage: 14 mm). Set Prior Lens Distortion -> Prior to Fixed.
8. Or Metashape: launch the app, import images, then go to Tools -> Camera Calibration (Initial tab). Set Type to Precalibration and update f to the value shown in the 360PerspCut log (Preset default: 533.33333, fisheyelike: 755.55556, full360coverage: 622.22222). Next, click the Fixed parameters Select button and Check all or check distortion parameters except f.
9. Bring the RealityScan or Metashape alignment results into a 3DGS tool such as PostShot.

Faster but low quality

1. Create a 360 video using DJI Studio or similar software.
2. Launch gs360_360GUI.py.
3. In the Video2Frames tab, extract 360 frames by setting the FPS and running the export.
4. In Metashape, import the frames, then go to Tools -> Camera Calibration and set Camera Type = Spherical.
5. Align the cameras.
6. Export from Metashape: File -> Export -> Export Cameras (Agisoft XML) and File -> Export -> Point Cloud (PLY).
7. Back in gs360_360GUI.py, open the MS360xmlToPerspCams tab and set Input XML to the exported XML.
8. Set Format = transforms, enable PerspCut, set PerspCut input to the 360 image folder, and set Points PLY to the exported PLY. Run the tool.
9. Use PerspCut out, transforms.json, and the rotated PLY in PostShot (or similar 3DGS tools).

Recommended Workflow

TODO

• Metashape Pro, Multi-Cameras-System. (Known bug: If you import perspective_cams_Multi-Camera-System.xml with MS360xmlToPerspCams and no tie points appear after alignment(sometimes), re-importing the XML file fixes the issue.).
• RealityCapture or Metashape Standard with spherical Metashape Alignment.xml.
• Hybrid scan combining mirrorless camera and 360-degree camera.

NOTES

• After converting to the RealityScan format (XMP) using MS360xmlToPersCams, be careful with the Position Accuracy setting in the alignment settings when running alignment in RealityScan. The default value is too large — it is intended for aerial capture.

---

License

This project is released under the MIT License. Copyright (c) 2025 Yu.

---

Work in progress

• Dual fisheye extraction → Undistort, apply D-log M to sRGB, Correct color differents → Convert to 5 perspective images → Update Metashape Multi-Camera System workflow
• CameraFormatConverter. RealityScan(CSV, Ply, XMP) ←→ Metashape(XML, Ply) ←→ Colmap

TODO / Ideas

• Flesh out a full GUI walkthrough (tab descriptions, launch parameters, screenshot gallery).
• Redesign the GUI using PySide or a similar framework
• Create the workflow using Mermaid notation
• Fisheye lens calibration and color correction, and extract perspective images directly from fisheye images