ERROR - ModuleNotFoundError: No module named 'ipd'

[WSobo]

The make_conda.sh script is missing the ipd dependency, which causes the tests to fail when running ./rf_diffusion/ca_rfd_tests.sh.

I tried installing the https://github.com/baker-laboratory/ipd, but it requires python>=3.10.

Error Message

========================================================================================= test session starts =========================================================================================
platform linux -- Python 3.9.22, pytest-7.4.0, pluggy-1.6.0
rootdir: ////CA_RFDiffusion
plugins: hypothesis-6.131.30, anyio-4.9.0, hydra-core-1.3.1
collected 0 items / 1 error

=============================================================================================== ERRORS ================================================================================================
____________________________________________________________________________________ ERROR collecting test session ____________________________________________________________________________________
../../../docker/conda/ca_rfd_release/lib/python3.9/site-packages/_pytest/config/init.py:642: in _importconftest
mod = import_path(conftestpath, mode=importmode, root=rootpath)
../../../docker/conda/ca_rfd_release/lib/python3.9/site-packages/_pytest/pathlib.py:565: in import_path
importlib.import_module(module_name)
../../../docker/conda/ca_rfd_release/lib/python3.9/importlib/init.py:127: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
:1030: in _gcd_import
???
:1007: in _find_and_load
???
:972: in _find_and_load_unlocked
???
:228: in _call_with_frames_removed
???
:1030: in _gcd_import
???
:1007: in _find_and_load
???
:986: in _find_and_load_unlocked
???
:680: in _load_unlocked
???
:850: in exec_module
???
:228: in _call_with_frames_removed
???
lib/rf2aa/rf2aa/tests/init.py:3: in
import ipd
E ModuleNotFoundError: No module named 'ipd'
======================================================================================= short test summary info =======================================================================================
ERROR - ModuleNotFoundError: No module named 'ipd'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
========================================================================================== 1 error in 0.78s ===========================================================================================

[davidcjuergens]

Thank you for posting this, I tried to get rid of all the ipd dependencies here and in rf2aa but seems like something snuck through. I think what I need to do is alter the remote rf2aa. I will work on this

[davidcjuergens]

Okay, first thing I have discovered is that the data which the tests depend on was accidentally not uploaded, so I just uploaded those. Now looking into if I can alter rf2aa to remove that rogue ipd dependency in tests/__init__.py

For now, can you also try removing the import ipd statement and the def load(... function definition in your local copy of the init file in rf2aa submodule? Let me know how that goes.

[blauemond]

Okay, first thing I have discovered is that the data which the tests depend on was accidentally not uploaded, so I just uploaded those. Now looking into if I can alter rf2aa to remove that rogue ipd dependency in tests/__init__.py

For now, can you also try removing the import ipd statement and the def load(... function definition in your local copy of the init file in rf2aa submodule? Let me know how that goes.

Hi, I tried to delete import pid and der load() in tests/__init__.py and tests, but a new mistake occurred in the following:

================================================ ERRORS ================================================ ____________________________________ ERROR collecting test session _____________________________________ ../../../miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/runner.py:341: in from_call result: Optional[TResult] = func() ../../../miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/runner.py:372: in <lambda> call = CallInfo.from_call(lambda: list(collector.collect()), "collect") ../../../miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/main.py:735: in collect for x in self._collectfile(pkginit): ../../../miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/main.py:582: in _collectfile ihook = self.gethookproxy(fspath) ../../../miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/main.py:549: in gethookproxy my_conftestmodules = pm._getconftestmodules( ../../../miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/config/__init__.py:610: in _getconftestmodules mod = self._importconftest(conftestpath, importmode, rootpath) ../../../miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/config/__init__.py:658: in _importconftest self.consider_conftest(mod) ../../../miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/config/__init__.py:740: in consider_conftest self.register(conftestmodule, name=conftestmodule.__file__) ../../../miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/config/__init__.py:488: in register ret: Optional[str] = super().register(plugin, name) ../../../miniconda3/envs/carfd/lib/python3.10/site-packages/pluggy/_manager.py:140: in register raise ValueError( E ValueError: Plugin name already registered: /xcfhome/yzmeng/tools/design/CA_RFDiffusion/lib/rf2aa/rf2aa/tests/conftest.py=<module 'rf2aa.tests.conftest' from '/xcfhome/yzmeng/tools/design/CA_RFDiffusion/lib/rf2aa/rf2aa/tests/conftest.py'> E {'140349740914272': <_pytest.config.PytestPluginManager object at 0x7fa5b8739660>, 'pytestconfig': <_pytest.config.Config object at 0x7fa5b80ae890>, 'mark': <module '_pytest.mark' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/mark/__init__.py'>, 'main': <module '_pytest.main' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/main.py'>, 'runner': <module '_pytest.runner' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/runner.py'>, 'fixtures': <module '_pytest.fixtures' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/fixtures.py'>, 'helpconfig': <module '_pytest.helpconfig' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/helpconfig.py'>, 'python': <module '_pytest.python' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/python.py'>, 'terminal': <module '_pytest.terminal' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/terminal.py'>, 'debugging': <module '_pytest.debugging' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/debugging.py'>, 'unittest': <module '_pytest.unittest' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/unittest.py'>, 'capture': <module '_pytest.capture' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/capture.py'>, 'skipping': <module '_pytest.skipping' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/skipping.py'>, 'legacypath': <module '_pytest.legacypath' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/legacypath.py'>, 'tmpdir': <module '_pytest.tmpdir' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/tmpdir.py'>, 'monkeypatch': <module '_pytest.monkeypatch' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/monkeypatch.py'>, 'recwarn': <module '_pytest.recwarn' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/recwarn.py'>, 'pastebin': <module '_pytest.pastebin' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/pastebin.py'>, 'nose': <module '_pytest.nose' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/nose.py'>, 'assertion': <module '_pytest.assertion' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/assertion/__init__.py'>, 'junitxml': <module '_pytest.junitxml' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/junitxml.py'>, 'doctest': <module '_pytest.doctest' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/doctest.py'>, 'cacheprovider': <module '_pytest.cacheprovider' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/cacheprovider.py'>, 'freeze_support': <module '_pytest.freeze_support' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/freeze_support.py'>, 'setuponly': <module '_pytest.setuponly' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/setuponly.py'>, 'setupplan': <module '_pytest.setupplan' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/setupplan.py'>, 'stepwise': <module '_pytest.stepwise' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/stepwise.py'>, 'warnings': <module '_pytest.warnings' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/warnings.py'>, 'logging': <module '_pytest.logging' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/logging.py'>, 'reports': <module '_pytest.reports' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/reports.py'>, 'python_path': <module '_pytest.python_path' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/python_path.py'>, 'unraisableexception': <module '_pytest.unraisableexception' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/unraisableexception.py'>, 'threadexception': <module '_pytest.threadexception' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/threadexception.py'>, 'faulthandler': <module '_pytest.faulthandler' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/_pytest/faulthandler.py'>, 'hydra_pytest': <module 'hydra.extra.pytest_plugin' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/hydra/extra/pytest_plugin.py'>, 'anyio': <module 'anyio.pytest_plugin' from '/xcfhome/yzmeng/miniconda3/envs/carfd/lib/python3.10/site-packages/anyio/pytest_plugin.py'>, 'capturemanager': <CaptureManager _method='fd' _global_capturing=<MultiCapture out=<FDCapture 1 oldfd=5 _state='suspended' tmpfile=<_io.TextIOWrapper name="<_io.FileIO name=6 mode='rb+' closefd=True>" mode='r+' encoding='utf-8'>> err=<FDCapture 2 oldfd=7 _state='suspended' tmpfile=<_io.TextIOWrapper name="<_io.FileIO name=8 mode='rb+' closefd=True>" mode='r+' encoding='utf-8'>> in_=<FDCapture 0 oldfd=3 _state='started' tmpfile=<_io.TextIOWrapper name='/dev/null' mode='r' encoding='utf-8'>> _state='suspended' _in_suspended=False> _capture_fixture=None>, 'session': <Session CA_RFDiffusion exitstatus=<ExitCode.OK: 0> testsfailed=0 testscollected=0>, 'lfplugin': <_pytest.cacheprovider.LFPlugin object at 0x7fa5b7e14af0>, 'nfplugin': <_pytest.cacheprovider.NFPlugin object at 0x7fa5b7e17c40>, 'legacypath-tmpdir': <class '_pytest.legacypath.LegacyTmpdirPlugin'>, 'terminalreporter': <_pytest.terminal.TerminalReporter object at 0x7fa5b7e17df0>, 'logging-plugin': <_pytest.logging.LoggingPlugin object at 0x7fa5b7e17b80>, 'funcmanage': <_pytest.fixtures.FixtureManager object at 0x7fa5b7fdd0f0>, '/xcfhome/yzmeng/tools/design/CA_RFDiffusion/lib/rf2aa/rf2aa/tests/conftest.py': <module 'rf2aa.tests.conftest' from '/xcfhome/yzmeng/tools/design/CA_RFDiffusion/lib/rf2aa/rf2aa/tests/conftest.py'>} ======================================= short test summary info ======================================== ERROR - ValueError: Plugin name already registered: /xcfhome/yzmeng/tools/design/CA_RFDiffusion/lib/rf2aa/r... !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

It seems that rf2aa has something wrong.

[davidcjuergens]

Thanks for posting Yanzheng. One question: What directory are you in when you execute the tests? It might be my fault for saying execute ./rf_diffusion/ca_rfd_tests.sh in the README. What happens if you cd rf_diffusion/ first and then ./ca_rfd_tests.sh? It might matter which directory you are in.

[blauemond]

Thanks for posting Yanzheng. One question: What directory are you in when you execute the tests? It might be my fault for saying execute ./rf_diffusion/ca_rfd_tests.sh in the README. What happens if you cd rf_diffusion/ first and then ./ca_rfd_tests.sh? It might matter which directory you are in.

Yes, I did execute ./rf_diffusion/ca_rfd_tests.sh in the CA_RFDiffusion directory, when I changed to rf_diffusion to run this script, some new mistakes occurs in the follow:
_________________________________________________________________________________________________________________________________________ ERROR collecting test_ca_rfd_inference.py _________________________________________________________________________________________________________________________________________ ImportError while importing test module '/xcfhome/yzmeng/tools/design/CA_RFDiffusion/rf_diffusion/test_ca_rfd_inference.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: ../../../../miniconda3/envs/carfd/lib/python3.10/importlib/__init__.py:126: in import_module return _bootstrap._gcd_import(name[level:], package, level) test_ca_rfd_inference.py:12: in <module> import run_inference run_inference.py:34: in <module> from rf_diffusion.inference import model_runners inference/__init__.py:1: in <module> from rf_diffusion.inference import utils, model_runners inference/utils.py:4: in <module> from rf_diffusion.kinematics import xyz_to_t2d kinematics.py:3: in <module> from rf_diffusion.chemical import ChemicalData as ChemData chemical.py:2: in <module> import rf2aa.chemical E ModuleNotFoundError: No module named 'rf2aa.chemical' ________________________________________________________________________________________________________________________________________ ERROR collecting test_ca_rfd_refinement.py _________________________________________________________________________________________________________________________________________ ImportError while importing test module '/xcfhome/yzmeng/tools/design/CA_RFDiffusion/rf_diffusion/test_ca_rfd_refinement.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: ../../../../miniconda3/envs/carfd/lib/python3.10/importlib/__init__.py:126: in import_module return _bootstrap._gcd_import(name[level:], package, level) test_ca_rfd_refinement.py:14: in <module> from rf2aa.model.RoseTTAFoldModel import LegacyRoseTTAFoldModule E ModuleNotFoundError: No module named 'rf2aa.model' _________________________________________________________________________________________________________________________________________ ERROR collecting test_ca_rfd_sm_train.py __________________________________________________________________________________________________________________________________________ ImportError while importing test module '/xcfhome/yzmeng/tools/design/CA_RFDiffusion/rf_diffusion/test_ca_rfd_sm_train.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: ../../../../miniconda3/envs/carfd/lib/python3.10/importlib/__init__.py:126: in import_module return _bootstrap._gcd_import(name[level:], package, level) test_ca_rfd_sm_train.py:10: in <module> from rf2aa.model.RoseTTAFoldModel import LegacyRoseTTAFoldModule E ModuleNotFoundError: No module named 'rf2aa.model' ____________________________________________________________________________________________________________________________________ ERROR collecting test_ca_rfd_unconditional_train.py ____________________________________________________________________________________________________________________________________ ImportError while importing test module '/xcfhome/yzmeng/tools/design/CA_RFDiffusion/rf_diffusion/test_ca_rfd_unconditional_train.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: ../../../../miniconda3/envs/carfd/lib/python3.10/importlib/__init__.py:126: in import_module return _bootstrap._gcd_import(name[level:], package, level) test_ca_rfd_unconditional_train.py:10: in <module> from rf2aa.model.RoseTTAFoldModel import LegacyRoseTTAFoldModule E ModuleNotFoundError: No module named 'rf2aa.model' ______________________________________________________________________________________________________________________________________________ ERROR collecting test_utils.py _______________________________________________________________________________________________________________________________________________ ImportError while importing test module '/xcfhome/yzmeng/tools/design/CA_RFDiffusion/rf_diffusion/test_utils.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: ../../../../miniconda3/envs/carfd/lib/python3.10/importlib/__init__.py:126: in import_module return _bootstrap._gcd_import(name[level:], package, level) test_utils.py:14: in <module> from rf2aa import tensor_util E ImportError: cannot import name 'tensor_util' from 'rf2aa' (unknown location)

It seems that these mistakes is still related to rf2aa, I am trying to modified such related codes.

[davidcjuergens]

Okay this is better. At this point, it seems like maybe rf2aa is not in your PYTHONPATH since it's unsure where rf2aa is. Is export PYTHONPATH="${PYTHONPATH}:../" still in your copy of ca_rfd_test.sh? I'm trying to replicate this on my end also.

[blauemond]

Yes, there are export PYTHONPATH="${PYTHONPATH}:../" export PYTHONPATH="${PYTHONPATH}:../lib/se3_flow_matching/" in ca_rfd_test.sh. As rf2aa has softlinked to the CA_RFDiffusion directory, I have not tested modifying it. Now let me try.

[blauemond]

After installing einops, evn and ipd, modifying evn's path and importing in ipd, inference scripts now pass the tests. Thank you!! But train script still had mistakes. It seems that \net directory could not be made because of permission.

[davidcjuergens]

Got it! okay that's really helpful, thank you for your help and feedback on this! Ideally we can remove the ipd dependency at some point in the future, but im glad for now it allowed you to proceed and the inference tests are passing!

I will look into those other missing dependencies as well.

For the training, I am working on getting the entire set of crystal structures that were in the training set available for download. It's really huge, so it's going to take some effort but ideally i can have that done in the next week or two. However, if you want to see the identities of the structures used, they are in the dataset .pkl file already available for download.

I will try to update the repo instructions/scripts with your findings soon!