API, embedding and language bindings

[amb5l]

I've been experimenting with driving NVC from Python. My initial goals were to expose the same functionality that a user sees from the command line executable - analysis, elaboration, simulation etc - plus some visibility of internal data structures. For example: analyse a source file and report the ports found in any entity declarations. It seems that this would require a clear C interface to which a Python module would bind.

So, I experimented with using NVC internals in C and succeeded in producing an entity ports report example. This involved figuring out the applicable functions and header files, and the libraries and object files from a completed NVC build to link against.

I am now playing with wrapping up the includes, and some code sequences, and producing a shim library so that a user application and its build process can be streamlined.

Exposing the command line functionality is one thing - I assume that side of things is settled (?), but exposing internals (e.g. the tree structure that is created by analysis) may create expectations of stability - or a maintenance overhead - that @nickg may not want to deal with?

Maybe I could share the outcome of my experiments as an addition to contrib/ that is clearly documented as experimental and unsupported as a first step. Any thoughts welcome...

I'm not a software engineer so please excuse my naivety.

[nickg]

Exposing the command line functionality is one thing - I assume that side of things is settled (?), but exposing internals (e.g. the tree structure that is created by analysis) may create expectations of stability - or a maintenance overhead - that @nickg may not want to deal with?

Yeah, I'm perfectly happy for people to take the code and do whatever they want with it but I don't really want to maintain a C library interface in addition to the command line interface, maybe TCL, and what's mandated by the standard (e.g. VHPI). The two main reasons are:

• NVC isn't really written in a way that makes it suitable for use as a library: it has a lot of global state, expects to be able to take over the signal handlers, often exits after serious errors rather than returning an error code back to the user, etc. etc. There are also more subtle things like the memory allocated for design units is never freed which is fine for a batch CLI program but becomes a problem if you have a long running process that analyses the same file repeatedly (e.g. language server use case). All these things could be fixed, but it would be a lot of effort and make the code much more complex than it is now.

• Hiding the internals means I can throw away large parts of the code and rewrite it. I do this quite a lot because I'm also not an expert in this and the first time I do something is usually quite bad, but at least I learn enough to know how to improve it. Exposing the internal interfaces makes it slow and difficult to change them without annoying users who may be depending on them, and if I made no backwards compatibility guarantees it's really no better than the current situation where someone can just take a snapshot of the code and build what they want on top of it.

[amb5l]

Absolutely fine and thanks for considering this idea. The agenda that drove me to look at this has shifted, so I don't actually need Python bindings anymore, but could probably create them - and they'd be great until the very next commit you make! My C example might be useful to someone trying to understand your codebase; anyone who wants to see it - comment below. I don't plan a PR.