How to use Hydra Sweeper with a list parameter?

[alex-bronze-vision]

Hi,

I am struggling to get the Hydra Sweeper to work in the following scenario.

Among some other configuration files, I have one that is as the following:

my_config:
  dvc_versions:
    - v1.1.0

dvc_versions basically is a list that can contain as desired versions with the same v.X.Y.Z. This works nice and the code is adapted for this.

However, I want to create a Hydra Sweeper to try different combinations of this versions. Something like:

hydra:
  sweeper:
    params:
      dataset.segmentation.dvc_versions: [v1.1.0], [v2.0.0]

Is this possible? I have been trying all possible sintax combinations and none of them work. The ones that pass through the Hydra parser just create a single job instead of two.

Thanks!

[mainrs]

If dvc_versions is of time list, than you can do ["v1.1.0", "v2.0.0"]

[alex-bronze-vision]

@mainrs thanks for the help, really appreciated. However, if I do that and set the sweeper as:

hydra:
  sweeper:
    params:
      dataset.segmentation.dvc_versions: ["v3.1.0", "v3.2.0"]

What I endup having is this:

[2024-07-24 10:15:40,152][HYDRA] Launching 1 jobs locally
[2024-07-24 10:15:40,152][HYDRA] 	#0 : dataset.segmentation.dvc_versions=['v3.1.0','v3.2.0']

and then my loaded config is basically a list with both versions...

basically the sweeper does not get that I want to sweep over two different lists as I expect:

[2024-07-24 10:15:40,152][HYDRA] Launching 2 jobs locally
[2024-07-24 10:15:40,152][HYDRA] 	#0 : dataset.segmentation.dvc_versions=['v3.1.0']
[2024-07-24 10:20:40,152][HYDRA] 	#1 : dataset.segmentation.dvc_versions=['v3.2.0']

[omry]

hydra.sweeper.params is a simple wrapper on top of the Compose API that takes a list of key=value strings (similar to what you pass in the command line).
Since YAML got its own idea of what is a list, you need to quote the string to avoid YAML parsing it if you want to pass a list there.

Try this.

hydra:
  sweeper:
    params:
      dataset.segmentation.dvc_versions: "[v1.1.0], [v2.0.0]"

[alex-bronze-vision]

hydra.sweeper.params is a simple wrapper on top of the Compose API that takes a list of key=value strings (similar to what you pass in the command line). Since YAML got its own idea of what is a list, you need to quote the string to avoid YAML parsing it if you want to pass a list there.

Try this.

hydra:
  sweeper:
    params:
      dataset.segmentation.dvc_versions: "[v1.1.0], [v2.0.0]"

That's it! Thank yoy @omry with the quotes it is working as expected. :D