Skip to content

Prototype to generate Infrahub Schema from Pydantic models #306

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 5 commits into
base: develop
Choose a base branch
from
Draft

Prototype to generate Infrahub Schema from Pydantic models #306

wants to merge 5 commits into from

Conversation

dgarros
Copy link
Contributor

@dgarros dgarros commented Mar 19, 2025 •
edited
Loading

This is an initial Prototype to explore how we could better integrate Pydantic within the SDK

The end goal is to leverage Pydantic to define the schema and to support all CRUD operations, similar to how SQLmodel is working for a Relational Database.

In the current prototype, it's possible to: Define the schema and read objects, the other operations (Create, Update, Delete) aren't available yet

Define Infrahub Schema using Pydantic Models

Below a simple example with 2 models connected together with a relationship (One to Many)

# This snippet is a simplified version of the example script in `docs/docs/python-sdk/examples/schema_pydantic.py`

from pydantic import Field
from infrahub_sdk.schema import (
     InfrahubAttributeParam as AttrParam,  
     InfrahubRelationshipParam as RelParam, 
    AttributeKind,
    from_pydantic,
    NodeModel
)

class TestCar(NodeModel):
    name: str = Field(description="The name of the car")
    owner: Annotated[Person, RelParam(identifier="car__person")]

class Person(NodeModel):
    model_config = ConfigDict(
        node_schema=NodeSchema(name="Person", namespace="Test", human_readable_fields=["name__value"])
    )

    name: str
    cars: Annotated[list[TestCar] | None, RelParam(identifier="car__person")] = None
    
async def main():
    client = InfrahubClient()
    schema = from_pydantic(models=[Person, TestCar])
    response = await client.schema.load(schemas=[schema.to_schema_dict()], wait_until_converged=True)

The integration is based on the following principles

  • All models must inherit from either NodeModel or GenericModel
  • Node specific parameters must be provided using model_config
  • Attribute or Relationship specific parameters that are not supported by Pydantic, must be provided via an Annotation (InfrahubAttributeParam and InfrahubRelationshipParam)
  • Attribute or Relationship parameters that are natively supported by Pydantic (default value, type, regex etc ..) can be provided using pydantic Field

In the example below, the annotation is used on cars to define the identifier for this relationship

class Person(NodeModel):
    name: str
    cars: Annotated[list[TestCar] | None, RelParam(identifier="car__person")] = None

This implementation is slightly different than SQLModel that is using a more Integrated approach with a custom Field method, would be good to understand which approach is preferable.

Query data using Pydantic

The main methods to query objects get, all & filters have been updated to access a Pydantic Models

In the example, below the model Site that was used to define the schema can also be used to query data.
The client.all() will automatically format the data returned by the API using the model Site, in Pydantic format

class Site(NodeModel):
    model_config = ConfigDict(
        node_schema=NodeSchema(
            name="Site", namespace="Infra", human_friendly_id=["name__value"], display_labels=["name__value"]
        )
    )

    name: Annotated[str, AttrParam(unique=True)] = Field(description="The name of the site")

async def main() -> None:
    client = InfrahubClient()
    sites = await client.all(kind=Site)
    rprint(sites)

Create / Update objects

This part hasn't been implemented yet but here is an overview of what it could look like following something similar to SQLModel.

async def main() -> None:
    client = InfrahubClient()
    jfk = Site(name="JFK")

    # objects to create or update will be added to the client and later, the changes will be `saved` / `committed` 
    client.add(jfk)

    await client.sync() # Could be client.save() or client.commit() too

@github-actions github-actions bot added the type/documentation Improvements or additions to documentation label Mar 19, 2025
@codecov Codecov
Copy link

codecov bot commented Mar 19, 2025 •
edited
Loading

Codecov Report

❌ Patch coverage is 1.03093% with 192 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
infrahub_sdk/schema/pydantic_utils.py 0.00% 160 Missing ⚠️
infrahub_sdk/client.py 3.57% 26 Missing and 1 partial ⚠️
infrahub_sdk/schema/__init__.py 0.00% 4 Missing and 1 partial ⚠️

❗ There is a different number of reports uploaded between BASE (545956a) and HEAD (41db077). Click for more details.

HEAD has 6 uploads less than BASE
Flag BASE (545956a) HEAD (41db077)
python-3.9 1 0
python-3.10 1 0
python-3.11 1 0
python-3.12 1 0
python-filler-3.12 1 0
python-3.13 1 0 
@@             Coverage Diff              @@
##           develop     #306       +/-   ##
============================================
- Coverage    75.74%   33.97%   -41.77%     
============================================
  Files          100      101        +1     
  Lines         8846     9034      +188     
  Branches      1732     1693       -39     
============================================
- Hits          6700     3069     -3631     
- Misses        1670     5688     +4018     
+ Partials       476      277      -199
Flag Coverage Δ
integration-tests 33.97% <1.03%> (-0.69%) ⬇️
python-3.10 ?
python-3.11 ?
python-3.12 ?
python-3.13 ?
python-3.9 ?
python-filler-3.12 ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
infrahub_sdk/schema/main.py 10.90% <100.00%> (-78.55%) ⬇️
infrahub_sdk/schema/__init__.py 12.73% <0.00%> (-55.92%) ⬇️
infrahub_sdk/client.py 24.16% <3.57%> (-45.33%) ⬇️
infrahub_sdk/schema/pydantic_utils.py 0.00% <0.00%> (ø)

... and 74 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@dgarros dgarros force-pushed the dga-2020319-prototype-pydantic branch from 36158c8 to 8a8a768 Compare April 1, 2025 16:32
@dgarros dgarros force-pushed the dga-2020319-prototype-pydantic branch from 8a8a768 to 41db077 Compare August 24, 2025 13:42
@cloudflare-workers-and-pages Cloudflare Workers and Pages
Copy link

cloudflare-workers-and-pages bot commented Aug 24, 2025 •
edited
Loading

Deploying infrahub-sdk-python with  Cloudflare Pages  Cloudflare Pages

Latest commit: fded2fd
Status: ✅  Deploy successful!
Preview URL: https://37dd3ce0.infrahub-sdk-python.pages.dev
Branch Preview URL: https://dga-2020319-prototype-pydant.infrahub-sdk-python.pages.dev

View logs

@coderabbitai coderabbitai
Copy link

coderabbitai bot commented Aug 24, 2025

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing Touches
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch dga-2020319-prototype-pydantic

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary or Summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

type/documentation Improvements or additions to documentation

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@dgarros