[FAQ] GNNs frequent questions

[matteobettini]

Here is a collection of frequent questions about GNN models in BenchMARL

Here #161 is an example of using a GNN with a dynamic topology in VMAS

[matteobettini]

When trying to use the GNN model for the actor network for the MAPPO algorithm, we do not pass in the topology, the position vector, velocity vector, etc. How is the GNN model supposed to account for all the environments available in VMAS, MPE, etc.?

The gnn model can take 2 special arguments: positions and velocities (which it uses to build edge features). If you do not pass them, then all of the agent observations will be node features and there will be no edge features. The topology can be: full, empty or from_pos. The default is “full” as in a fully connected graph.
 
Hence, if you leave all defaults, a fully connected gnn will be used for each group (groups are always kept separate) with no edge features and observations as node features.

This can be applied to any environmnent

[matteobettini]

The code mentions that the GNN class can only be used in the case of decentralized actors or critics. Why is this?

BenchMARL has 4 ways a model can be used https://benchmarl.readthedocs.io/en/latest/concepts/components.html#models
 

• As a policy (decentralized): gnn can do this
• As a decentralized critic (same as policy but for computing values): gnn can do this
• As a centralised critic from local agent inputs (aka you have access to all agents observations in the critic): gnn can do this and does it using mean graph pooling
• As a centralized critic from global input (let’s say a global vector or something that does not have agent dimensions): GNN does not do this as how could it build a graph from only one input?

BenchMARL will automatically construct the GNN in the right way for the different uses based on your algorithm and env (e.g. avaiability of a global state). If it can't, it will tell you

[matteobettini]

How are the features for the nodes and the edges constructed for the GNN model. The topology, positions, velocities that the GNN is using do not seem to be passed on in the algorithm. How does the GNN work in this case? 

The algorithms and models are completely decoupled in benchmarl, so each component will always be agnostic of each other. This is the core tenet of the library.
 
The features are constructed as the following:

• Node features: all agent observations are node features by default. The only exception is if you pass position_key and exclude_pos_from_node_features=True, where position will be excluded from node features.
• Edge features (you will need a gnn kernel compatible with them): if you pass position_key, relative positions and distance will be added to the edge features ($x_i-x_j$, $||x_i-x_j||$). if you pass velocity_key, relative velocities will be added to the edge features ($v_i-v_j$)

[matteobettini]

In a game setting, where different teams should have different critics and policies, is the GNN network accounting for all the interactions among agents? For instance, with a proximity-based rule to generate the graph, would we have a graph per team with edges between members of the same team only, or a game graph accounting for both intra and inter-team edges?

Teams (groups in benchmarl) are separate (different loss class, buffer, models, etc). All the gnn stuff , parameter sharing, and topology is within a group.
See this issue if you are interested in using different algos and models for different teams #156

[matteobettini]

I understand that the position key when passed in has to be a key in the observation dictionary and the pos_features is the shape of the tensor value associated with the position_key key. I think the position_key field would have to be added in the vmas library's code, right? We will need to have this field be passed in from the observation spec, but the vmas library does not pass the positions of all the agents in the observation spec.

You are right. If you want to use the position key and the velocity key in the GNN model, your observations need to be a dictionary and those need to be keys in it.

In vmas you need an observation() function like this:

def observation(self, agent: Agent):
    return {"obs": you_other_obs, "pos": agent.state.pos, "vel": agent.state.vel}

and then set

• position_key: pos
• pos_features: 2
• velocity_key: vel
• vel_features: 2

You are right that normal vmas scnearios do not do this so you need to either (1) edit their observation function or (2) make your own

What should the shape of this tensor be?

normal shape of an observations just with the last dimension being equal to pos_features. E.g., in the vmas obs function positions are of shape [n_envs, 2] for each agent. after benchmarl stacks agents they will become [n_envs, n_agents,2].

in general, don't overthink it, just return it as any other obs in your simulator .

Should we pass in the relative position vectors or the original position vectors for the agents? My understanding tells me it should be the original position vectors, but I want to make sure I am not missing something.

original positions (and velocities), benchmarl will automatically compute relatives

Should the pos_features be a tuple or a single integer? (Tuple because the shape could be (num_agents, 2))

int, just the number of features of your position (1 in 1D, 2 in 2D, 3 in 3D, and so on)

(configurations in benchmarl are hard-typed, so you can always check the type of what you need to pass in the respective dataclass, e.g., https://benchmarl.readthedocs.io/en/latest/generated/benchmarl.models.GnnConfig.html#benchmarl.models.GnnConfig)

EDIT: Here is an example of how to do it in VMAS #161