Introduce Topology to check biomass graph disconnections.

[iago-lito]

Start work on addressing #151.

[STATUS] The feature is complete and about to land on main. Before documenting it an considering it "released" though:

• Try it out, making sure bikeshedding is over.
• Write basic documentation (not much technical about code, but rather high-level about science) [comment].
• Write a first CHANGELOG.md entry to land this into main as our v0.2.1 :)

[iago-lito]

So, this PR addresses #151 in a very deep way by creating the new Topology type.
Values of this type represent the ecological network under a strict topological perspective (neighbourhood and connections) according to the model sketched in #151 (comment).

The reason to put so much effort into Topology semantics and testing is that I don't intend to only use it to solve #151. I expect this type to become a solid candidate to tackle #141 when it'll be time: all the internal representation of what users mean by an "EcologicalNetwork" can be based on such a Topology.

Review welcome @alaindanet @ismael-lajaaiti ;) There is a lot of code so I won't mind if you don't thoroughly scan all of it of course. But if you want to help: please test the feature to check whether it's comfy and whether it corresponds to something that would at least solve #151 :)

Entrypoint into the feature:

• (check out the disconnections branch)
• You can try model.topology to retrieve a topology value, then test the methods in src/topology.jl.
• After simulation you can also try get_topology(solution) to retrieve a similar-looking topology, but with the extinct species removed. If simulation has resulted in the introduction of isolated producers / starving consumers / disconnected components, then you should have seen an @info message about that and you can query it with the same set of methods ;)

Feedback welcome! (here or on the channel ;)

[ismael-lajaaiti]

I started to tried your feature but I quickly encountered an error. Here is my code

using Distributions
using EcologicalNetworksDynamics
using Random
Random.seed!(1234) # For reproducibility.

foodweb = Foodweb(:niche, S=50, C=0.2)
m = default_model(foodweb)
g = get_topology(m)
g == m.topology # Sanity check. Expected to be true.

simulate(m, rand(Uniform(0.1, 1), m.S), 1_000)

which gives me the following error

ERROR: KeyError: key EcologicalNetworksDynamics.Topologies.Abs(3) not found
Stacktrace:
 [1] pop!
   @ ./dict.jl:604 [inlined]
 [2] pop!(s::Set{EcologicalNetworksDynamics.Topologies.Abs}, x::EcologicalNetworksDynamics.Topologies.Abs)
   @ Base ./set.jl:104
 [3] starving_consumers(m::<inner parms>, g::EcologicalNetworksDynamics.Topologies.Topology)
   @ EcologicalNetworksDynamics ~/Documents/Projects/Julia/edn/src/topology.jl:117
 [4] show_degenerated_biomass_graph_properties(model::<inner parms>, biomass::Vector{Float64}, arg::Symbol)
   @ EcologicalNetworksDynamics ~/Documents/Projects/Julia/edn/src/simulate.jl:141
 [5] _simulate(model::<inner parms>, u0::Vector{Float64}, tmax::Int64; kwargs::@Kwargs{model::Model})
   @ EcologicalNetworksDynamics ~/Documents/Projects/Julia/edn/src/simulate.jl:44
 [6] _simulate(::Model, ::Vector{Float64}, ::Vararg{Any}; kwargs::@Kwargs{model::Model})
   @ EcologicalNetworksDynamics ~/Documents/Projects/Julia/edn/src/Framework/method_macro.jl:294
 [7] _simulate
   @ ~/Documents/Projects/Julia/edn/src/Framework/method_macro.jl:288 [inlined]
 [8] simulate(model::Model, u0::Vector{Float64}, tmax::Int64)
   @ EcologicalNetworksDynamics ~/Documents/Projects/Julia/edn/src/simulate.jl:74
 [9] top-level scope
   @ REPL[59]:1

Did I do something wrong?
I wanted to run a simulation and remove some (disconnected) species or to see how I could create a new model from a single disconnected components, or simply from the network of alive species.

Otherwise I was wondering if it would be possible, to have a compact view of the topology for large networks? Because currently it is a bit overhelming (see screenshot below). But this is a very cosmetic feature and in any way shouldn't be (y)our priority.

[iago-lito]

Wops! I can reproduce your error with a simple graph indeed:

m = default_model(Foodweb([:a => [:b, :c], :b => :c]))
g = m.topology
starving_consumers(m, g)

This is a bug in the graph visit implementation. I'm fixing it right now. Thank you for reporting.

Otherwise I was wondering if it would be possible, to have a compact view of the topology for large networks? Because currently it is a bit overhelming
Wow, of course! I already have a few utils to elide long vectors.. and maybe I won't display the whole adjacency list if it's long.

BTW are you happy with the default :s1, :s2, .., s:2048 naming, or should we produce shorter identifiers like :a, :b, .. :z, :aa, :ab etc.? Or random fixed-width identifiers like :ux, :bw, :pa, :oj.. ?

[ismael-lajaaiti]

This is a bug in the graph visit implementation. I'm fixing it right now. Thank you for reporting.

Great, thank you for fixing it. 😉
Let me know when it's done, so I can continue experimenting the new feature. 🤓

BTW are you happy with the default :s1, :s2, .., s:2048 naming, or should we produce shorter identifiers like :a, :b, .. :z, :aa, :ab etc.? Or random fixed-width identifiers like :ux, :bw, :pa, :oj.. ?

Yes, I like it. In particular, because with this default it is easy to distinguish between species and nutrients.

[iago-lito]

Thank you for feedback @ismael-lajaaiti :)

we can improve the error message when requesting an adjacency matrix of interaction type that is not present within the model

ERROR: ArgumentError: Invalid edge type label: :facilitation. Valid labels are :refuge and :trophic.

I'm not sure we can do much better because the list of possible "interaction types" is open. Today, neither the topology or the model values know about all possible interaction types, and so there is no (simple) way they could make a difference between:

• :facilitation: a valid interaction name that just happen not to exist within this model.
• :rfeuge: incorrect spelling of an interaction that happens to exist within this model.

Thus the error message you are seing now. Maybe I can still clarify a little though:

Invalid edge type label: :facilitation. Valid labels in this model are :refuge and :trophic. ..?

Why not return an empty adjacency matrix, instead of raising an error?

For the same reason: in principle, anyone could add the interaction type they want to the system by creating their own custom components (although it's (very) not ready yet). So given input like :rfeuge, there is no way the system could decide whether to error (incorrect name) or to return an empty matrix (valid name), because the list of names is open. So it needs to always take the same decision in both cases. My opinion now is that issuing an error with :facilitation if there is no facilitation in the model is ok, but returning an empty matrix with :rfeuge if it is a typo is absolutely unacceptable ^ ^"

I'm curious though: what would you use this empty matrix for? How come your code needs to request facilitation links in a model without a Facilitation component?

---

update the method when generating a network with the niche or cascade model so it produces a matrix with no starving consumers or isolated producers

Oh, I thought these were guaranteed not to generate such degenerated networks. If they aren't, then of course we could feature that. The algorithm would just be bruteforce like the one to avoid cycles, right? Draw random networks until we find one without this kind of species.
Would you be okay to leave this for another github issue? I am going to screen through every component again in the context of #139. I could maybe pick this new feature up when I get to rewriting the Foodweb blueprints :)

---

We (I) should also work on expanding the documentation

Of course :) My first (modest) contribution was this comment.
It was good not to dive into the documentation too soon because the design is still being discussed here, but as you seem to be mostly satisfied with it now, I'll be happy to review a related doc-labeled PR :)
Again, although the documentation should cover the basics of using topology values to address #151, I would advise against putting too much effort into documenting very sophisticated code snippets using it, because possibly #141 could lead to Topology becoming the internal structure of Model, and this would possibly surface into breaking changes to the API introduced here in #152.
So, yeah, focus on the science between these new features and their meaning (that should not change). I hope this comment should be helpful in this respect. Then don't be overly precise about the code details ;)

What I can do on my side is to start keeping track of API changes in a toplevel CHANGELOG.md file. This disconnections branch may become our first version bump to EcologicalNetworksDynamics v0.2.1 ;)

Unfortunately, I am leaving tonight for three weeks off. What I can do is merge this PR into dev right now so that we can sync up on this version which also happens to bundle a few bugfixes like f1b740d. But I'll leave it open and won't merge it into main until we are confident with the design and we have written the corresponding doc and the CHANGELOG :)

See you about end of August <3

[ismael-lajaaiti]

Invalid edge type label: :facilitation. Valid labels in this model are :refuge and :trophic. ..?

Yes, IMO this is already better!

For the same reason: in principle, anyone could add the interaction type they want to the system by creating their own custom components (although it's (very) not ready yet). So given input like :rfeuge, there is no way the system could decide whether to error (incorrect name) or to return an empty matrix (valid name), because the list of names is open. So it needs to always take the same decision in both cases. My opinion now is that issuing an error with :facilitation if there is no facilitation in the model is ok, but returning an empty matrix with :rfeuge if it is a typo is absolutely unacceptable ^ ^"

OK, I see.

I'm curious though: what would you use this empty matrix for? How come your code needs to request facilitation links in a model without a Facilitation component?

First, my reasoning was that a model without a Facilitation component and a model with an empty Facilitation component is essentially the same model. Second, for instance, let's say you have simulated lots of models with and without different non-trophic components, and you want to investigate the impact of the number of non-trophic links on a given property (e.g., species diversity). We may want to loop on this set of simulated models, extract their number of non-trophic links at the end of the simulation to plot that against the final diversity. Or something along those lines.

Oh, I thought these were guaranteed not to generate such degenerated networks. If they aren't, then of course we could feature that. The algorithm would just be bruteforce like the one to avoid cycles, right? Draw random networks until we find one without this kind of species. Would you be okay to leave this for another github issue? I am going to screen through every component again in the context of #139. I could maybe pick this new feature up when I get to rewriting the Foodweb blueprints :)

Yes, and yes. ;)
Issue opened #159.

Unfortunately, I am leaving tonight for three weeks off. What I can do is merge this PR into dev right now so that we can sync up on this version which also happens to bundle a few bugfixes like f1b740d. But I'll leave it open and won't merge it into main until we are confident with the design and we have written the corresponding doc and the CHANGELOG :)

See you by the end of August <3

Enjoy your (well deserved) holidays! 🌴 ☀️

[iago-lito]

I'm curious though: what would you use this empty matrix for? How come your code needs to request facilitation links in a model without a Facilitation component?

First, my reasoning was that a model without a Facilitation component and a model with an empty Facilitation component is essentially the same model.

That is.. very true. They are the same in principle, but they differ in the code because the former does not store extra facilitation data whereas the latter does. This is an unfortunate quirk.. :\ But I'm not sure what to do about it without also stating that "every model is a model with an empty Brakbaorgas layer" 8\ Luckily this is rather a theoretical/philosophical quirk, and we haven't yet come across a situation where it's really annoying. But yeah, m1 == m2 would fail to capture the identity between two such models :(

say you have simulated lots of models with and without different non-trophic components, and you want to investigate the impact of the number of non-trophic links on a given property (e.g., species diversity). We may want to loop on this set of simulated models, extract their number of non-trophic links at the end of the simulation to plot that against the final diversity.

Okay, I see two approaches in this situations:

1. Simulate all your models with a FacilitationLayer, some of which happen to be empty. Then species_adjacency_matrix(g, :facilitation) should never error and return null matrices when empty.
2. OR guard your final extraction with something like nl = has_component(model, FacilitationLayer) ? model.n_facilitation_links : 0.

Would either be okay ? :)

[iago-lito]

Hi @alaindanet @ismael-lajaaiti! Following our recent discussion, I shortened verbosity arguments to simulate in the last commit on this branch (3fefa15 at the time of writing).

But it makes us realize that this PR is kind of stalled right now. The blocker was to have someone try the feature and write the associated (science-)documentation.

If this is not something you think we can do before landing #139, then one option is to land the feature as part of #139 but without actually documenting it, so it'd be present but not exactly "released", and then maybe report its documentation to #141?

[iago-lito]

Wops, this is an accidental close, yet definitive it seems :(
Continued in #177.