Cpp extensions, why needed?

[SaxElectronics]

Hi,

This is more like a general question. I really liked the library that it was purely pythonic based . Since the major update to process DAQ lists with c++ extension this adds additional dependency and I do not understand the need for it. Maybe @christoph2 as the expert here can ellaborate more on this decision he made?

[christoph2]

@SaxElectronics

Well, performance and scalability in the first place - while sub-interpreters are a hot-topic in the latest Python versions, parallelism and other things are much easier to express in C++.
In my opinion, modern C++ and modern Python complement each other very well.
I hope these C++ extensions are causing you not to much trouble.

[SaxElectronics]

i am just not familiar with c++ extensions but it is a good occasion to get into. Do i have to install visual studio c++ compiler extra or is this not a hard requirement because the binaries are somehow precompiled and available when latest pyxcp gets installed?

[christoph2]

• The binaries are automatically installed from the Python package index (PyPI) if you run pip install pyxcp on Windows or Linux (currently only Intel/AMD).
• In other cases, e.g. MacOS or Windows/ARM, a build from source process is invoked; the same holds for local builds:
  pip install -e .
  This step is required if you cloned/forked or downloaded pyxcp; this is basically the modern equivalent of python setup.py develop
• the build process uses CMake as backend, so one could install Visual Studio on Windows, but GCC or Clang / LLVM is also fine.

[SaxElectronics]

thanks, i am trying it out. currently i have a bit of an issue, because i use also poetry for another repo and the pyxcp fork of mine is a dependency on the other repo and i completely rely on automatic installation.

When i did the pip install -e . on the fork itsilf it worked fine, but when the fork is as a dependency of another repo it is not that easy.
I am trying to resolve this issue somehow...

[SaxElectronics]

@christoph2 which files would be missing if i just download the source? I see the .pyd files are there but i am getting a DLL import error.
from pyxcp.cpp_ext.cpp_ext import DaqList
E ImportError: DLL load failed while importing cpp_ext: The specified module could not be found.
The binaries seem to be there, but i don't get why there is an import issue with a dll

[christoph2]

@SaxElectronics
Maybe a version issue (if you have installed multiple Python versions)?

[christoph2]

@SaxElectronics

but when the fork is as a dependency of another repo it is not that easy.
I am trying to resolve this issue somehow...

There is a poetry source add command, I think it can be used to bypass package installation from PyPI.

[SaxElectronics]

no, it was not the python version.

I was struggling a bit but now i made it work, it seems poetry cannot use git repositories and install them as custom packages on the fly, you have to create an extra wheel in the fork and reference to this dev whl. What poetry does with git referencies is just to clone the repo and remove .git and any build information (big limitation from my point of view), so as long as you do not have to build anything inside the fork or the git repos - it works all fine. However if you have to invoke builds the only option i found is to create a wheel for the fork itself.

In this case i created a dev (developement) wheel and it looks like this:
"pyxcp @ https://github.com/SaxElectronics/pyxcp/releases/download/dev/pyxcp-0.22.28.dev0-cp312-cp312-win_amd64.whl",
Now poetry can use this and install it as custom package.
Since my fork had a pretty old state, i have to fix some inports at my side, i had before
config = readConfiguration(detected_config)
what is now the substitute for "readConfiguration" ?

[christoph2]

@SaxElectronics
Honestly, I can't remember a function named readConfiguration()...
The configuration system is completly different for some time now and transparently handled behind the scenes, there is no user intervention required.

[SaxElectronics]

Yes, there was:

I am trying to adopt the new way of configuration, also these two functions were there:
try_to_install_system_supplied_drivers()

CAN_DRIVERS = registered_drivers()

is there any readme or example to help me here ? I see the yaml/toml-way of configuration is now a legacy one, correct?

[christoph2]

@SaxElectronics

The old config. files can still be used, but there is a tool (xcp-profile) that can be used to convert configuration files; it is also useful to get a quick overview of the configuration parameters:

xcp-profile create | less

Well, the README is currently (for strange reasons...) part of this discussion

It's also possible to copy the example code to a user directory:

xcp-examples <dir-name>