Skip to content

Conversation

creatorrr
Copy link
Contributor

@creatorrr creatorrr commented May 20, 2025

User description

Summary

  • add slim compose for agents-api & friends
  • support dev-up in poe tasks
  • enforce OpenAPI codegen freshness via pre-commit and CI
  • document pre-commit in contributing guide

Testing

  • pre-commit run --files openapi.yaml (fails: command not found)

PR Type

Enhancement, Documentation


Description

  • Introduce lightweight Docker Compose file for dev iteration

    • Aggregates multiple service compose files for easier startup
    • Defines profiles for selective service activation
  • Add dev-up task to poe_tasks.toml for simplified development workflow

  • Enforce OpenAPI code generation freshness in CI and pre-commit

    • Adds pre-commit hook and CI step to check codegen status
  • Update contributing guide to document pre-commit setup


Changes walkthrough 📝

Relevant files
Enhancement
docker-compose.dev.yaml
Add lightweight Docker Compose file for development           

deploy/docker-compose.dev.yaml

  • Adds a new lightweight Docker Compose file for development
  • Includes multiple service compose files for unified dev startup
  • Sets up profiles for each service for flexible activation
  • +33/-0   
    poe_tasks.toml
    Add dev-up task for simplified development startup             

    poe_tasks.toml

    • Adds a dev-up task to run the new dev compose file
    +2/-0     
    generate-openapi-code-from-typespec.yml
    Enforce OpenAPI codegen freshness in CI workflow                 

    .github/workflows/generate-openapi-code-from-typespec.yml

  • Updates workflow to check OpenAPI codegen freshness
  • Fails CI if generated code is out of date
  • +2/-1     
    .pre-commit-config.yaml
    Add pre-commit hook for OpenAPI codegen freshness               

    .pre-commit-config.yaml

  • Adds pre-commit hook to check OpenAPI codegen freshness
  • Runs codegen script and fails if changes are detected
  • +7/-0     
    Documentation
    CONTRIBUTING.md
    Document pre-commit setup in contributing guide                   

    CONTRIBUTING.md

    • Documents pre-commit installation and hook enabling in setup steps
    +1/-0     

    Need help?
  • Type /help how to ... in the comments thread for any questions about Qodo Merge usage.
  • Check out the documentation for more information.
  • @creatorrr creatorrr marked this pull request as ready for review May 20, 2025 19:04
    Copy link
    Contributor

    qodo-merge-for-open-source bot commented May 20, 2025

    CI Feedback 🧐

    (Feedback updated until commit b9a4639)

    A test triggered by this PR failed. Here is an AI-generated analysis of the failure:

    Action: Typecheck

    Failed stage: Ensure openapi code is fresh [❌]

    Failure summary:

    The action failed during the code generation and formatting process. The main issues are:

    1. Multiple warnings about duplicated field name kind_ in various Task classes (Main,
    ErrorWorkflowStep, SwitchStep, ToolCallStep, WaitForInputStep, YieldStep)

    2. The code generation tool (datamodel-codegen) had issues with several integer formats not being
    understood (uint32, uint16, int16, int8, uint8, uint64)

    3. There were formatting and linting issues that were fixed automatically, but the process still
    exited with code 1, suggesting that some validation or check after the fixes still failed

    Relevant error logs:
    1:  ##[group]Operating System
    2:  Ubuntu
    ...
    
    155:  prune-cache: true
    156:  ignore-nothing-to-cache: false
    157:  ##[endgroup]
    158:  Downloading uv from "https://github.com/astral-sh/uv/releases/download/0.7.12/uv-x86_64-unknown-linux-gnu.tar.gz" ...
    159:  [command]/usr/bin/tar xz --warning=no-unknown-keyword --overwrite -C /home/runner/work/_temp/e5725e77-03a1-4d80-a6cb-68f4a0c89613 -f /home/runner/work/_temp/3deb4e8e-6e7e-40b1-9e61-e84de10e8bb4
    160:  Added /opt/hostedtoolcache/uv/0.7.12/x86_64 to the path
    161:  Added /home/runner/.local/bin to the path
    162:  Set UV_CACHE_DIR to /home/runner/work/_temp/setup-uv-cache
    163:  Successfully installed uv version 0.7.12
    164:  Searching files using cache dependency glob: **/uv.lock
    165:  /home/runner/work/julep/julep/agents-api/uv.lock
    166:  /home/runner/work/julep/julep/cli/uv.lock
    167:  /home/runner/work/julep/julep/integrations-service/uv.lock
    168:  Found 3 files to hash.
    169:  Trying to restore uv cache from GitHub Actions cache with key: setup-uv-1-x86_64-unknown-linux-gnu-0.7.12-d2fb266502043ea806ace17d24fc614bc58c57b06c32d25d4a98237661d5d333
    170:  ##[warning]Failed to restore: getCacheEntry failed: Cache service responded with 503
    171:  No GitHub Actions cache found for key: setup-uv-1-x86_64-unknown-linux-gnu-0.7.12-d2fb266502043ea806ace17d24fc614bc58c57b06c32d25d4a98237661d5d333
    ...
    
    230:  �[37mPoe =>�[0m �[94mdatamodel-codegen --input ../openapi.yaml --input-file-type openapi --output agents_api/autogen/ --output-model-type pydantic_v2.BaseModel --strict-types bool --strict-nullable --allow-population-by-field-name --field-include-all-keys --reuse-model --snake-case-field --enum-field-as-literal all --field-constraints --use-operation-id-as-name --use-schema-description --use-field-description --use-annotated --use-default --use-unique-items-as-set --use-subclass-enum --use-union-operator --use-one-literal-as-default --use-double-quotes --use-exact-imports --use-standard-collections --use-non-positive-negative-number-constrained-types --target-python-version 3.12 --treat-dot-as-module --use-title-as-name --collapse-root-models --output-datetime-class AwareDatetime --openapi-scopes schemas --keep-model-order --disable-timestamp�[0m
    231:  /home/runner/work/_temp/setup-uv-cache/archive-v0/54TdhygtUcuLHkVNPvP4S/lib/python3.12/site-packages/datamodel_code_generator/parser/jsonschema.py:612: UserWarning: format of 'uint32' not understood for 'integer' - using default
    232:  _get_type(type_, format__),
    233:  /home/runner/work/_temp/setup-uv-cache/archive-v0/54TdhygtUcuLHkVNPvP4S/lib/python3.12/site-packages/datamodel_code_generator/parser/jsonschema.py:612: UserWarning: format of 'uint16' not understood for 'integer' - using default
    234:  _get_type(type_, format__),
    235:  /home/runner/work/_temp/setup-uv-cache/archive-v0/54TdhygtUcuLHkVNPvP4S/lib/python3.12/site-packages/datamodel_code_generator/parser/jsonschema.py:612: UserWarning: format of 'int16' not understood for 'integer' - using default
    236:  _get_type(type_, format__),
    237:  /home/runner/work/_temp/setup-uv-cache/archive-v0/54TdhygtUcuLHkVNPvP4S/lib/python3.12/site-packages/datamodel_code_generator/parser/jsonschema.py:612: UserWarning: format of 'int8' not understood for 'integer' - using default
    238:  _get_type(type_, format__),
    239:  /home/runner/work/_temp/setup-uv-cache/archive-v0/54TdhygtUcuLHkVNPvP4S/lib/python3.12/site-packages/datamodel_code_generator/parser/jsonschema.py:612: UserWarning: format of 'uint8' not understood for 'integer' - using default
    240:  _get_type(type_, format__),
    241:  /home/runner/work/_temp/setup-uv-cache/archive-v0/54TdhygtUcuLHkVNPvP4S/lib/python3.12/site-packages/datamodel_code_generator/parser/jsonschema.py:612: UserWarning: format of 'uint64' not understood for 'integer' - using default
    242:  _get_type(type_, format__),
    243:  /home/runner/work/_temp/setup-uv-cache/archive-v0/54TdhygtUcuLHkVNPvP4S/lib/python3.12/site-packages/datamodel_code_generator/model/base.py:307: UserWarning: Field name `kind_` is duplicated on Tasks.Main
    244:  self.fields = self._validate_fields(fields) if fields else []
    245:  /home/runner/work/_temp/setup-uv-cache/archive-v0/54TdhygtUcuLHkVNPvP4S/lib/python3.12/site-packages/datamodel_code_generator/model/base.py:307: UserWarning: Field name `kind_` is duplicated on Tasks.ErrorWorkflowStep
    246:  self.fields = self._validate_fields(fields) if fields else []
    ...
    
    294:  �[37mPoe =>�[0m �[94mdatamodel-codegen --input ../openapi.yaml --input-file-type openapi --output integrations/autogen/ --output-model-type pydantic_v2.BaseModel --strict-types bool --strict-nullable --allow-population-by-field-name --field-include-all-keys --reuse-model --snake-case-field --enum-field-as-literal all --field-constraints --use-operation-id-as-name --use-schema-description --use-field-description --use-annotated --use-default --use-unique-items-as-set --use-subclass-enum --use-union-operator --use-one-literal-as-default --use-double-quotes --use-exact-imports --use-standard-collections --use-non-positive-negative-number-constrained-types --target-python-version 3.12 --treat-dot-as-module --use-title-as-name --collapse-root-models --output-datetime-class AwareDatetime --openapi-scopes schemas --keep-model-order --disable-timestamp�[0m
    295:  /home/runner/work/_temp/setup-uv-cache/archive-v0/54TdhygtUcuLHkVNPvP4S/lib/python3.12/site-packages/datamodel_code_generator/parser/jsonschema.py:612: UserWarning: format of 'uint32' not understood for 'integer' - using default
    296:  _get_type(type_, format__),
    297:  /home/runner/work/_temp/setup-uv-cache/archive-v0/54TdhygtUcuLHkVNPvP4S/lib/python3.12/site-packages/datamodel_code_generator/parser/jsonschema.py:612: UserWarning: format of 'uint16' not understood for 'integer' - using default
    298:  _get_type(type_, format__),
    299:  /home/runner/work/_temp/setup-uv-cache/archive-v0/54TdhygtUcuLHkVNPvP4S/lib/python3.12/site-packages/datamodel_code_generator/parser/jsonschema.py:612: UserWarning: format of 'int16' not understood for 'integer' - using default
    300:  _get_type(type_, format__),
    301:  /home/runner/work/_temp/setup-uv-cache/archive-v0/54TdhygtUcuLHkVNPvP4S/lib/python3.12/site-packages/datamodel_code_generator/parser/jsonschema.py:612: UserWarning: format of 'int8' not understood for 'integer' - using default
    302:  _get_type(type_, format__),
    303:  /home/runner/work/_temp/setup-uv-cache/archive-v0/54TdhygtUcuLHkVNPvP4S/lib/python3.12/site-packages/datamodel_code_generator/parser/jsonschema.py:612: UserWarning: format of 'uint8' not understood for 'integer' - using default
    304:  _get_type(type_, format__),
    305:  /home/runner/work/_temp/setup-uv-cache/archive-v0/54TdhygtUcuLHkVNPvP4S/lib/python3.12/site-packages/datamodel_code_generator/parser/jsonschema.py:612: UserWarning: format of 'uint64' not understood for 'integer' - using default
    306:  _get_type(type_, format__),
    307:  /home/runner/work/_temp/setup-uv-cache/archive-v0/54TdhygtUcuLHkVNPvP4S/lib/python3.12/site-packages/datamodel_code_generator/model/base.py:307: UserWarning: Field name `kind_` is duplicated on Tasks.Main
    308:  self.fields = self._validate_fields(fields) if fields else []
    309:  /home/runner/work/_temp/setup-uv-cache/archive-v0/54TdhygtUcuLHkVNPvP4S/lib/python3.12/site-packages/datamodel_code_generator/model/base.py:307: UserWarning: Field name `kind_` is duplicated on Tasks.ErrorWorkflowStep
    310:  self.fields = self._validate_fields(fields) if fields else []
    ...
    
    335:  /home/runner/work/_temp/setup-uv-cache/archive-v0/54TdhygtUcuLHkVNPvP4S/lib/python3.12/site-packages/datamodel_code_generator/model/base.py:307: UserWarning: Field name `kind_` is duplicated on Tasks.SwitchStep
    336:  self.fields = self._validate_fields(fields) if fields else []
    337:  /home/runner/work/_temp/setup-uv-cache/archive-v0/54TdhygtUcuLHkVNPvP4S/lib/python3.12/site-packages/datamodel_code_generator/model/base.py:307: UserWarning: Field name `kind_` is duplicated on Tasks.ToolCallStep
    338:  self.fields = self._validate_fields(fields) if fields else []
    339:  /home/runner/work/_temp/setup-uv-cache/archive-v0/54TdhygtUcuLHkVNPvP4S/lib/python3.12/site-packages/datamodel_code_generator/model/base.py:307: UserWarning: Field name `kind_` is duplicated on Tasks.WaitForInputStep
    340:  self.fields = self._validate_fields(fields) if fields else []
    341:  /home/runner/work/_temp/setup-uv-cache/archive-v0/54TdhygtUcuLHkVNPvP4S/lib/python3.12/site-packages/datamodel_code_generator/model/base.py:307: UserWarning: Field name `kind_` is duplicated on Tasks.YieldStep
    342:  self.fields = self._validate_fields(fields) if fields else []
    343:  + uv_run 'poe format'
    344:  + uvx --with ruff --with datamodel-code-generator --from poethepoet poe format
    345:  �[37mPoe =>�[0m �[94mruff format�[0m
    346:  7 files reformatted, 69 files left unchanged
    347:  + uv_run 'poe lint'
    348:  + uvx --with ruff --with datamodel-code-generator --from poethepoet poe lint
    349:  �[37mPoe =>�[0m �[94mruff check�[0m
    350:  Fixed 3 errors:
    351:  - integrations/autogen/Responses.py:
    352:  1 × W291 (trailing-whitespace)
    353:  - integrations/autogen/Tasks.py:
    354:  1 × F401 (unused-import)
    355:  - integrations/autogen/Tools.py:
    356:  1 × F401 (unused-import)
    357:  Found 3 errors (3 fixed, 0 remaining).
    358:  + cd -
    ...
    
    738:  The steps to run for each iteration
    739:  """
    740:  @@ -266,9 +270,9 @@ class ElseModel(BaseModel):
    741:  model_config = ConfigDict(
    742:  populate_by_name=True,
    743:  )
    744:  -    label: Annotated[str | None, Field(max_length=120, pattern="^[^0-9]|^[0-9]+[^0-9].*$")] = (
    745:  -        None
    746:  -    )
    747:  +    label: Annotated[
    748:  +        str | None, Field(max_length=120, pattern="^[^0-9]|^[0-9]+[^0-9].*$")
    749:  +    ] = None
    750:  """
    751:  The label of this step for referencing it from other steps
    752:  """
    753:  @@ -312,13 +316,15 @@ class ErrorWorkflowStep(BaseModel):
    754:  model_config = ConfigDict(
    755:  populate_by_name=True,
    756:  )
    757:  -    kind_: Annotated[Literal["error"], Field(json_schema_extra={"readOnly": True})] = "error"
    758:  +    kind_: Annotated[Literal["error"], Field(json_schema_extra={"readOnly": True})] = (
    759:  +        "error"
    760:  +    )
    ...
    
    2585:  +            {
    2586:  +              "maxLength": 255,
    2587:  +              "minLength": 1,
    2588:  +              "pattern": "^[a-zA-Z][a-zA-Z0-9_]*$",
    2589:  +              "type": "string"
    2590:  +            },
    2591:  +            {
    2592:  +              "type": "null"
    2593:  +            }
    2594:  +          ],
    2595:  +          "title": "Project"
    2596:  +        },
    2597:  "updated_at": {
    2598:  "format": "date-time",
    2599:  "readOnly": true,
    2600:  ##[error]Process completed with exit code 1.
    2601:  Post job cleanup.
    

    Copy link
    Contributor

    PR Reviewer Guide 🔍

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
    🧪 No relevant tests
    🔒 No security concerns identified
    ⚡ Recommended focus areas for review

    Service Dependencies

    The temporal service has explicit dependencies defined, but other services might also have dependencies that aren't explicitly declared. This could lead to startup order issues.

    temporal:
      profiles: [""]
      depends_on:
        temporal-db:
          condition: service_started
    File Filtering

    The pre-commit hook runs the full codegen script regardless of which files changed. Consider adding a files pattern to only trigger when openapi.yaml or related files change.

    - id: codegen-freshness
      language: system
      entry: bash -c "scripts/generate_openapi_code.sh && git diff --exit-code"

    Copy link
    Contributor

    qodo-merge-for-open-source bot commented May 20, 2025

    PR Code Suggestions ✨

    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Impact
    General
    Enable profile-based service activation

    The command doesn't specify which profiles to activate, so it will start all
    services. Add a --profile flag to enable selective service activation, which is
    the main benefit of the new compose setup.

    poe_tasks.toml [1-2]

     [tasks.dev-up]
    -shell = "docker compose -f deploy/docker-compose.dev.yaml --env-file .env up --build"
    +shell = "docker compose -f deploy/docker-compose.dev.yaml --env-file .env --profile default up --build"
    • Apply / Chat
    Suggestion importance[1-10]: 7

    __

    Why: This suggestion enhances the utility of the new compose setup by enabling selective service activation via profiles, making the development workflow more flexible and aligned with the intended Docker Compose configuration.

    Medium
    Use meaningful profile names

    The empty string in profiles ([""]) will cause all services to be started by
    default, which defeats the purpose of using profiles for flexible service
    activation. Consider using a meaningful profile name like "default" or "core"
    instead of an empty string.

    deploy/docker-compose.dev.yaml [8-18]

     services:
       agents-api:
    -    profiles: [""]
    +    profiles: ["default"]
       worker:
    -    profiles: [""]
    +    profiles: ["default"]
       memory-store:
    -    profiles: [""]
    +    profiles: ["default"]
       vectorizer-worker:
    -    profiles: [""]
    +    profiles: ["default"]
       integrations:
    -    profiles: [""]
    +    profiles: ["default"]
    • Apply / Chat
    Suggestion importance[1-10]: 6

    __

    Why: The suggestion improves maintainability and clarity by recommending meaningful profile names instead of empty strings, which helps future contributors understand and use Docker Compose profiles as intended, but it does not fix a critical bug.

    Low
    • Update

    Copy link
    Contributor

    @ellipsis-dev ellipsis-dev bot left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Important

    Looks good to me! 👍

    Reviewed everything up to a56f63a in 1 minute and 21 seconds. Click for details.
    • Reviewed 90 lines of code in 5 files
    • Skipped 0 files when reviewing.
    • Skipped posting 5 draft comments. View those below.
    • Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
    1. .github/workflows/generate-openapi-code-from-typespec.yml:34
    • Draft comment:
      Good use of 'git diff --exit-code' to enforce freshness. Consider adding 'set -eo pipefail' at the start of the run block to catch any errors immediately.
    • Reason this comment was not posted:
      Comment was on unchanged code.
    2. .pre-commit-config.yaml:6
    • Draft comment:
      Ensure that the generate_openapi_code.sh script is executable to avoid hook failures on some systems.
    • Reason this comment was not posted:
      Confidence changes required: 33% <= threshold 50% None
    3. CONTRIBUTING.md:163
    • Draft comment:
      Great addition of pre-commit installation instructions to ease onboarding for new contributors.
    • Reason this comment was not posted:
      Confidence changes required: 0% <= threshold 50% None
    4. deploy/docker-compose.dev.yaml:2
    • Draft comment:
      The use of the 'include' key and empty string profiles is non-standard; ensure your Docker Compose version or tooling supports this syntax.
    • Reason this comment was not posted:
      Comment did not seem useful. Confidence is useful = 0% <= threshold 50% The comment is asking the author to ensure compatibility with their Docker Compose version or tooling, which violates the rule against asking the author to ensure behavior is intended or tested. It doesn't provide a specific suggestion or ask for a specific change.
    5. poe_tasks.toml:2
    • Draft comment:
      The dev-up task is clear; ensure the referenced .env file exists and has all required environment variables for successful startup.
    • Reason this comment was not posted:
      Confidence changes required: 33% <= threshold 50% None

    Workflow ID: wflow_hWpL3Vjzct37DMty

    You can customize Ellipsis by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.

    Copy link

    gitguardian bot commented Jun 7, 2025

    ⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

    Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

    🔎 Detected hardcoded secret in your pull request
    GitGuardian id GitGuardian status Secret Commit Filename
    17693055 Triggered JSON Web Token a601325 cli/tests/test_auth.py View secret
    🛠 Guidelines to remediate hardcoded secrets
    1. Understand the implications of revoking this secret by investigating where it is used in your code.
    2. Replace and store your secret safely. Learn here the best practices.
    3. Revoke and rotate this secret.
    4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

    To avoid such incidents in the future consider


    🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

    @creatorrr creatorrr marked this pull request as draft June 7, 2025 08:02
    @Ahmad-mtos
    Copy link
    Contributor

    What is this for? are those services enough? what about production where we're not using memory-store service?

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    2 participants