|
2 | 2 |
|
3 | 3 | # Caliscope |
4 | 4 |
|
5 | | -*Multicamera Calibration + Pose Estimation --> Open Source Motion Capture* |
| 5 | +*Multicamera Calibration for Research Workflows* |
6 | 6 |
|
7 | 7 | [](https://pypi.org/project/caliscope/) |
8 | 8 | [](https://opensource.org/license/bsd-2-clause/) |
|
12 | 12 |  |
13 | 13 | </div> |
14 | 14 |
|
15 | | -Caliscope is a GUI-based multicamera calibration package. |
16 | | -It simplifies the process of determining camera properties to enable 3D motion capture in a hardware-agnostic pipeline. |
| 15 | +Caliscope is a permissively licensed multicamera calibration tool for markerless motion capture workflows. It allows visual assessment and detailed quality metrics at each stage of the calibration workflow to allow high quality output. The approach to initializing parameters for bundle adjustment ([see docs](https://mprib.github.io/caliscope/extrinsic_calibration/)) allows rapid and reliable calibration. |
17 | 16 |
|
18 | 17 | ## Demo |
19 | 18 |
|
20 | | -NOTE: the demo below represents the current version on `main`, which has not yet been formally released on PyPI yet. |
21 | | - |
22 | 19 | https://github.com/user-attachments/assets/037c6237-0955-41e2-979e-a4247f7677e6 |
23 | 20 |
|
24 | 21 | ## Quick Start |
25 | 22 |
|
26 | | -Basic installation instructions can be found in our [docs](https://mprib.github.io/caliscope/installation/). |
27 | | -Please note that installation can take a while due to large dependencies like OpenCV and PySide6. |
28 | | -We are working to streamline the installation in an upcoming release. |
| 23 | +Installation instructions are in the [docs](https://mprib.github.io/caliscope/installation/). |
29 | 24 |
|
30 | | -For a complete overview of the entire workflow, please see the [sample project](https://mprib.github.io/caliscope/sample_project/). |
31 | | -A [video walk through](https://www.youtube.com/watch?v=voE3IKYtuIQ) demonstrates the process with an example dataset. |
| 25 | +For a walkthrough with test data after installing, see the [sample project](https://mprib.github.io/caliscope/sample_project/). |
32 | 26 |
|
33 | 27 | --- |
34 | 28 |
|
35 | | -## How It Works |
36 | | - |
37 | | -To triangulate 3D landmarks from synchronized video, you must know the intrinsic and extrinsic properties of your camera system. |
38 | | -Intrinsic properties include each camera's focal length, optical center, and lens distortion. |
39 | | -Extrinsic properties describe the relative rotation and translation of all cameras in the system. |
40 | | -Using more cameras makes 3D tracking more robust to occlusion and other inevitable errors, but increases the computational demands of optimizing their relative position. |
41 | | - |
42 | | -While OpenCV provides functions for single-camera intrinsics, estimating extrinsics for more than two cameras is not straightforward. |
43 | | -This multi-camera process requires a technique called [bundle adjustment](https://scipy-cookbook.readthedocs.io/items/bundle_adjustment.html), which demands extensive tracking of camera parameters and 2D point estimates. |
44 | | - |
45 | | -Caliscope automates this calibration process from only raw video and a definition of your calibration board. |
46 | | -It provides visual feedback at each stage, helping you verify the parameter estimates. |
47 | | - |
48 | | -## Key Features |
| 29 | +## Features |
49 | 30 |
|
50 | 31 | #### Calibration |
51 | 32 |
|
52 | | -- Easy creation of `png` files for ChArUco calibration boards. |
53 | | -- Automated calculation of camera intrinsic properties from input video. |
54 | | -- Visualization of the distortion model to ensure reasonableness. |
55 | | -- Automated bundle adjustment to estimate the 6-DoF relative position of all cameras. |
56 | | -- A 3D visualizer to inspect camera position estimates. |
57 | | -- Tools to set the world origin within the visualizer to simplify data processing. |
58 | | - |
59 | | -#### 3D Tracking |
| 33 | +- ChArUco, ArUco, and chessboard calibration targets |
| 34 | +- Automated intrinsic calibration from video with distortion model visualization |
| 35 | +- Pairwise extrinsic initialization for reliable bundle adjustment across 3+ cameras |
| 36 | +- Mirror board support for camera arrangements where no single board position is visible to all cameras |
| 37 | +- 3D visualizer for inspecting camera positions and setting the world origin |
| 38 | +- Reprojection error display and outlier filtering after optimization |
| 39 | +- Exports `camera_array.toml` (native) and `camera_array_aniposelib.toml` for use with [aniposelib](https://github.com/lambdaloop/aniposelib)-compatible tools |
60 | 40 |
|
61 | | -- A general Tracker interface for integrating alternate 2D tracking methods. |
62 | | -- Three sample implementations using Google Mediapipe (Hands/Pose/Holistic). |
63 | | -- Automated application of 2D landmark tracking to synchronized videos. |
64 | | -- Triangulation of 3D landmark positions based on the full camera system calibration. |
65 | | -- Trajectory smoothing through gap-filling and Butterworth filtering. |
| 41 | +#### Tracking and Reconstruction |
66 | 42 |
|
67 | | -#### Data Export |
68 | | - |
69 | | -- Output to the `.trc` file format for use in biomechanical modeling. |
70 | | -- Output to a tidy `.csv` format for integration with other analysis workflows. |
71 | | - |
72 | | -## Roadmap & Integrations |
73 | | - |
74 | | -The current tracker implementations provide a proof-of-concept pipeline using Google's Mediapipe. |
75 | | -While Mediapipe is an easy and efficient method for human tracking, it has limitations in accuracy and precision. |
76 | | -The planned roadmap includes integration with more powerful tools like [MMPose](https://github.com/open-mmlab/mmpose), [DeepLabCut](https://github.com/DeepLabCut/DeepLabCut), and [SLEAP](https://github.com/talmolab/sleap). |
| 43 | +- Built-in MediaPipe trackers (Hands, Pose, Holistic) |
| 44 | +- ONNX model support for custom pose estimators exported from SLEAP, DeepLabCut, RTMPose, and other frameworks |
| 45 | +- Output in CSV and TRC (OpenSim) formats |
77 | 46 |
|
78 | 47 | ## Community & Support |
79 | 48 |
|
80 | | -To report a bug or request a feature, please [open an issue](https://github.com/mprib/caliscope/issues). |
81 | | -Please keep in mind this is an open-source project supported by volunteer effort, so your patience is appreciated. |
82 | | - |
83 | | -For general questions and conversation, please post in the [Discussions](https://github.com/mprib/caliscope/discussions) section of the repo. |
| 49 | +To report a bug or request a feature, please [open an issue](https://github.com/mprib/caliscope/issues). For questions, post in [Discussions](https://github.com/mprib/caliscope/discussions). This is an open-source project supported by volunteer effort. |
84 | 50 |
|
85 | 51 | ## Acknowledgments |
86 | 52 |
|
87 | 53 | This project was inspired by [FreeMoCap](https://github.com/freemocap/freemocap) (FMC), which is spearheaded by [Jon Matthis, PhD](https://jonmatthis.com/) of the HuMoN Research Lab. |
88 | 54 | The FMC calibration and triangulation system is built upon [Anipose](https://github.com/lambdaloop/anipose), created by Lili Karushchek, PhD. |
89 | 55 | Caliscope was originally envisioned as an alternative calibration tool to Anipose that would allow more granular estimation and visual feedback. |
90 | | - |
91 | 56 | Several lines of the original Anipose triangulation code are used in this code base, though it was otherwise written from the ground up. |
92 | | -I'm grateful to Dr. Matthis for his time developing FreeMoCap, discussing it with me, and providing a great deal of information regarding open-source project management. |
93 | 57 |
|
94 | 58 | ## License |
95 | 59 |
|
96 | 60 | Caliscope is licensed under the permissive [BSD 2-Clause license](https://opensource.org/license/bsd-2-clause/). |
97 | 61 | The triangulation function was adapted from the [Anipose](https://github.com/lambdaloop/anipose) code base which is also licensed under the BSD-2 Clause. |
98 | | -A primary dependency of this project is PySide6 which provides the GUI front end. |
99 | | -PySide6 is licensed under the [LGPLv3](https://www.gnu.org/licenses/lgpl-3.0.html). |
100 | | -Caliscope does not modify the underlying source code of PySide6 which is available via [PyPI](https://pypi.org/project/PySide6/). |
0 commit comments