An error occurred when executing MASt3R-Fusion.

[ryujin62]

Dear Sir,

When running MASt3R-Fusion using Example OverExposure,
the following error occurred in the solve_GN_calib function within the run_backend() process.

Please advise on how to resolve this issue. Thank you in advance.
Note: In mast3r_fusion/global_opt.py, BA2GTSAM_PYTHON_IMPLEMENTATION is set to True.
(BA2GTSAM_PYTHON_IMPLEMENTATION = True)

Thank you

---

[INFO] add factor. 1763429065.1761956
solve_GN_calib!!!!
[INFO] marg 1763429065.1769028
[INFO] marg. 1763429065.1769083
[INFO] before optim. 1763429065.1813862
Traceback (most recent call last):
File "/home/keiji/MASt3R-Fusion/main.py", line 326, in
run_backend(states, keyframes)
File "/home/keiji/MASt3R-Fusion/main.py", line 99, in run_backend
factor_graph.solve_GN_calib(config["use_calib"])
File "/home/keiji/MASt3R-Fusion/mast3r_fusion/global_opt.py", line 750, in solve_GN_calib
vfactors = Align2GTSAM_factors(H11.numpy(),v11.numpy(),self.wTcs[pin:],self.ss[pin:],ii.cpu().numpy(),jj.cpu().numpy(),pin)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/keiji/MASt3R-Fusion/mast3r_fusion/global_opt.py", line 88, in Align2GTSAM_factors
factors.append(CustomHessianFactor(symbols,initials,HHH/1e6,-vvv/1e6))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/keiji/MASt3R-Fusion/mast3r_fusion/global_opt.py", line 102, in CustomHessianFactor
h_f = gtsam.HessianFactor(symbols,dims,info_expand)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: init(): incompatible constructor arguments. The following argument types are supported:
1. gtsam.gtsam.HessianFactor()
2. gtsam.gtsam.HessianFactor(factor: gtsam.gtsam.GaussianFactor)
3. gtsam.gtsam.HessianFactor(j: int, G: numpy.ndarray[numpy.float64[m, n]], g: numpy.ndarray[numpy.float64[m, 1]], f: float)
4. gtsam.gtsam.HessianFactor(j: int, mu: numpy.ndarray[numpy.float64[m, 1]], Sigma: numpy.ndarray[numpy.float64[m, n]])
5. gtsam.gtsam.HessianFactor(j1: int, j2: int, G11: numpy.ndarray[numpy.float64[m, n]], G12: numpy.ndarray[numpy.float64[m, n]], g1: numpy.ndarray[numpy.float64[m, 1]], G22: numpy.ndarray[numpy.float64[m, n]], g2: numpy.ndarray[numpy.float64[m, 1]], f: float)
6. gtsam.gtsam.HessianFactor(j1: int, j2: int, j3: int, G11: numpy.ndarray[numpy.float64[m, n]], G12: numpy.ndarray[numpy.float64[m, n]], G13: numpy.ndarray[numpy.float64[m, n]], g1: numpy.ndarray[numpy.float64[m, 1]], G22: numpy.ndarray[numpy.float64[m, n]], G23: numpy.ndarray[numpy.float64[m, n]], g2: numpy.ndarray[numpy.float64[m, 1]], G33: numpy.ndarray[numpy.float64[m, n]], g3: numpy.ndarray[numpy.float64[m, 1]], f: float)
7. gtsam.gtsam.HessianFactor(factors: gtsam::GaussianFactorGraph)

Invoked with: [8286623314361712640, 8646911284551352320, 8286623314361712641, 8646911284551352321], [1, 6, 1, 6], array([[ 1.52762677e-02, -4.68112565e-03, -5.76944905e-04,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
7.24357178e-03, -8.51022974e-03, 4.99979661e-04,
4.95881841e-04, 1.28382268e-04, 1.51154417e-03,
3.08280630e-04, -7.07739566e-03, -6.16361098e-04],
[-4.68112565e-03, 1.72535565e+08, -6.48451665e+06,
1.67784677e+06, -6.63138949e+05, -6.44535600e+07,
-2.75439813e+07, 1.47401426e+07, -1.03253955e+08,
2.17049232e+07, -2.33214334e+07, 3.48438841e+06,
6.24435966e+07, 3.16548311e+07, 5.35125548e+04],
[-5.76944905e-04, -6.48451665e+06, 2.39649415e+08,

---

[ryujin62]

Dir Sir,
This is a follow-up to the previous question.
When setting BA2GTSAM_PYTHON_IMPLEMENTATION=False, the error message is as follows:
Note that the example used was written in Handheld2, not OverExposure.
Thank you

---

[INFO] track 1763440078.1858685
[INFO] track. 1763440078.2032745
[INFO] backend-1 1763440078.20476
[INFO] add factor 1763440078.2215576
-- 1763440078.3283598
[INFO] add factor. 1763440078.4038653
solve_GN_calib!!!!
[INFO] marg 1763440078.4046438
[INFO] marg. 1763440078.4046507
[INFO] before optim. 1763440078.4094322
Traceback (most recent call last):
File "/home/keiji/MASt3R-Fusion/main.py", line 326, in
run_backend(states, keyframes)
File "/home/keiji/MASt3R-Fusion/main.py", line 99, in run_backend
factor_graph.solve_GN_calib(config["use_calib"])
File "/home/keiji/MASt3R-Fusion/mast3r_fusion/global_opt.py", line 750, in solve_GN_calib
vfactors = Align2GTSAM_factors(H11.numpy(),v11.numpy(),self.wTcs[pin:],self.ss[pin:],ii.cpu().numpy(),jj.cpu().numpy(),pin)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/keiji/MASt3R-Fusion/mast3r_fusion/global_opt.py", line 35, in Align2GTSAM_factors
factors = gtsam.Align2GTSAM_factors(list(H11.reshape(-1, 7,7)),
^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'gtsam' has no attribute 'Align2GTSAM_factors'
Process Process-2:
Traceback (most recent call last):
File "/home/keiji/anaconda3/envs/mast3r_fusion_imu/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/home/keiji/anaconda3/envs/mast3r_fusion_imu/lib/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/home/keiji/MASt3R-Fusion/mast3r_fusion/visualization.py", line 637, in run_visualization
if states.get_mode() == Mode.TERMINATED:
^^^^^^^^^^^^^^^^^
File "/home/keiji/MASt3R-Fusion/mast3r_fusion/frame.py", line 202, in get_mode
with self.lock:
File "/home/keiji/anaconda3/envs/mast3r_fusion_imu/lib/python3.11/multiprocessing/managers.py", line 1056, in enter
return self._callmethod('acquire')
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/keiji/anaconda3/envs/mast3r_fusion_imu/lib/python3.11/multiprocessing/managers.py", line 837, in _callmethod
raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError:

---

[yuxuanzhou97]

Hello! Thanks for your feedback.

I believe this is because the official version of GTSAM is used. Please install the provided repository (https://github.com/yuxuanzhou97/gtsam.git), which is a modified version of GTSAM, and try again.

Please let me know if it still doesn't work.

[ryujin62]

Thank you for your advise.

We will try MASt3r-Fusion by using your gtsam codes (https://github.com/yuxuanzhou97/gtsam.git).