Skip to content

Conversation

TLSDC
Copy link
Collaborator

@TLSDC TLSDC commented Apr 29, 2025

Benchmark tutorial

This PR serves as a template for benchmark creation. It involves the following steps:

  • Creating a task object to integrate your tasks to BrowserGym
  • Registering said tasks to expose them through the gym.make interface. (see loop.py )
  • Make your benchmark pip installable along with browsergym
  • Define an action set for your benchmark
  • Define a Benchmark object to use your benchmark in the AgentLab experiment setting
  • Add basic Tests (see existing tests for inspiration)
  • Update ReadMe

@ServiceNow ServiceNow deleted a comment from korbit-ai bot Apr 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants