Splitting mesa into smaller, independent packages

[Corvince]

This is something that has been sitting on the top of my head for quite some time and I would like to discuss.

The idea is to split up mesa into several packages, for example

• mesa-core
• mesa-visualization
• mesa-examples
• mesa-agents (this one doesn't make sense at the moment, but could potentially host some out-of-the box agents, such as RandomWalker)

mesa would become a meta-package that merely installs the above packages.

There are some advantages to this approach

1. mesa-core could be relatively stable, which could accelerate a version 1.0.0 release. Of course it depends what is inside and outside of mesa-core but we could exclude most of the moving parts from the core development.
2. mesa-core could be used standalone as a minimal interface without much overheat.
3. mesa-examples can be versioned, so every release of mesa has fitting examples.
4. mesa-visualization could be developed independently and rapidly without interfering with the core development cycle (I have quite some plans for mesa-viz, but this will be another discussion topic)
5. New branches/packages could be discussed without worrying to much about the size of mesa, since mesa-core can remain minimal.

To overall goal would be to accelerate development without breaking core functionalities.

The obvious downside is that the release stories will get more complicated. Also, we would have to decide if we would want to split the repo for each package or keep the individual packages inside this repository. Personally I think it just isn`t enough code to justify the split and it is much easier to tie things together during the development if we keep it in a single repo. However, I think it makes sense to discuss splitting in general and the technical aspects of the split together.

Looking forward to discussions.

[rht]

Looking at precedents:

• scipy is a monorepo. It depends only on Numpy in its setup.py
• matplotlib is a monorepo, see the setup.py
• d3.js is modularized with lots of components, see the package.json. It is very likely that you can import each component and use them on their own. But the reason could be for modularity's sake. Not sure, I need to dig into d3.js's threads.
• Agents.jl mainly contains the non-viz stuff, while you can use InteractiveDynamics.jl for viz stuff, but InteractiveDynamics.jl is also used for any kind of dynamical systems, not just for Agents.jl

scipy contains independent moving parts, but it remains as a monorepo. Could be because scipy is an older project.

I think separating to separate repos is well justified if the components themselves can be used as a standalone package. With this constraint, mesa-core can surely be extracted from the current mesa, since it works on its own.

[rht]

I'm adding one more data point: hash.ai's own hEngine is part of a monorepo containing multiple independent subpackages. This is notable, because hEngine is also an ABM framework.

[tpike3]

I agree I think the modular approach is is the way to go (or at least try). I think of it as allowing for the diversity of the user base to evolve Mesa along multiple paths. The great challenge will be can we get the contributor base for each path and possible variations (e.g. vis with javascript, viz with pure python).

To account for the contributor density I am assuming that ABMs will become a commonplace tool for research and decision making. To me, the challenge is an ABM ecosystem is an order of magnitude harder than say an ML ecosystem.

So I would add -
mesa repo has the key components -
mesa-core
mesa-visualizations etc

then you go out a layer and you would get repos dedicated to different parts
mesadata
mesabehaviors