No reward in Meltingpot tasks

[simonpedrogonzalez]

Hi! Thanks for the amazing work. I'm having trouble running meltingpot tasks, for example:

HYDRA_FULL_ERROR=1 uv run thirdparty/BenchMARL/benchmarl/run.py -m \
algorithm=mappo \
task=meltingpot/collaborative_cooking__crowded \
model=sequence \
"model.intermediate_sizes=[256]" \
"model/layers@model.layers.l1=cnn" \
"model/layers@model.layers.l2=mlp" \
model@critic_model=sequence \
"critic_model.intermediate_sizes=[256]" \
"model/layers@critic_model.layers.l1=cnn" \
"model/layers@critic_model.layers.l2=mlp" \
seed=0 \
experiment.save_folder=./outputs \
experiment.checkpoint_interval=60000 \
experiment.checkpoint_at_end=True \
experiment.sampling_device="cpu" \
experiment.train_device="cpu" \
experiment.buffer_device="cpu" \
experiment.loggers=[csv] 

But the rewards from the environment seems to be stuck at 0 (even after training several hours):

I get rewards for other tasks (vmas, etc), I bet something is wrong with my command, maybe the cnn setup, but I can't figure out what it is. Any help would be greatly appreciated ;)

[matteobettini]

Hello, yes it will require a lot of fine tuning.

I started doing some here, maybe try these params #78 (I was testing them w ppo)