Skip to content

Multiple model parameter with same value result in multiple evaluationsΒ #220

@Munsio

Description

@Munsio

When running the evaluation and specifying the same model multiple times currently the evaluation is run X amount of times for the model.

Example:

eval-dev-quality evaluate --runtime docker --result-path ./docker-test --runs 5 --model symflower/symbolic-execution --model symflower/symbolic-execution --model symflower/symbolic-execution --repository golang/plain

This runs the symflower/symbolic-execution 3 times with 5 runs as the model was 3 times specified as parameter.

Question:
Do we want this behavior or should we unique the list of models after parsing?

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingquestionFurther information is requested

Type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions