Skip to content

Conversation

jb3rndt
Copy link
Collaborator

@jb3rndt jb3rndt commented Oct 11, 2025

This replaces the table DDLs and insert statements with sqlalchemy.

The ability to set the table name through the config makes handling the models a bit cumbersome, raising the question whether we really need this?

(Does not include migrations using Alembic yet)

@jb3rndt jb3rndt requested review from Phimanu and lisehr October 13, 2025 12:58
@lisehr lisehr self-assigned this Oct 18, 2025
Copy link
Collaborator

@lisehr lisehr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original thought was that other people (external researchers) might store the dq results in their own table with customized names. For now, it should work thought but the goal of Metis is to keep it generic and reusable in different settings.

sqlalchemy: great move, thanks. only one suggestion: could you rename the models.py to something more specific? Models in Metis could be anything (ML models, data models, etc.) - please make clear that these are the models required by sqlalchemy.

@Copilot Copilot AI review requested due to automatic review settings October 19, 2025 18:49
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR migrates database operations from raw SQL to SQLAlchemy ORM, eliminating manual DDL statements and parameterized queries in favor of declarative models and ORM sessions. The refactor consolidates common database logic into a base DatabaseWriter class while maintaining database-specific implementations for SQLite and PostgreSQL.

  • Replaces raw SQL CREATE TABLE and INSERT statements with SQLAlchemy declarative models
  • Introduces a new DatabaseWriter base class to centralize ORM operations
  • Implements dynamic table name configuration through a model registration pattern

Reviewed Changes

Copilot reviewed 4 out of 7 changed files in this pull request and generated 2 comments.

File Description
metis/writer/sqlite_writer.py Refactored to extend DatabaseWriter, implements SQLite-specific engine creation
metis/writer/postgres_writer.py Refactored to extend DatabaseWriter, implements PostgreSQL-specific engine creation
metis/writer/database_writer.py New base class implementing shared ORM logic for database writers
metis/database_models.py New file defining SQLAlchemy declarative models with dynamic table name support

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@jb3rndt
Copy link
Collaborator Author

jb3rndt commented Oct 19, 2025

Thank you, good point. I've changed the file from models.py -> database_models.py.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants