SQLAlchemy Session Store for Production Databases

[habema]

The current SQLiteSession is excellent for development and simple use cases. For production environments, it would be highly beneficial to support more robust databases like PostgreSQL or MySQL for storing conversation history.

Would the project be open to a contribution that adds a new SQLAlchemySession class?

This would implement the existing Session protocol, allowing users to connect to any database supported by SQLAlchemy. It would be an optional feature, and sqlalchemy would be an extra dependency.

If this sounds like a welcome addition, I'm happy to contribute a PR.

[seratch]

This sounds like a great idea to me. Some people may want to use a different ORM like Django ORM but many devs should want to use SQLAlchemy for connecting to various databases.

When I was working at Slack, I implemented local file system, SQLite3, SQLAlchemy, Amazon S3, and Django ORM. Supporting multiple options worked very well:

• https://github.com/slackapi/python-slack-sdk/tree/main/slack_sdk/oauth/installation_store
• https://github.com/slackapi/bolt-python/blob/main/examples/django/oauth_app/slack_datastores.py

Going back to the sessions enhancement, we already have two PRs to add PostgreSQL and Redis:

• Initial pass at PostgreSQL session management #1163
• Feature: Redis Session Management #1247

I just checked both again and they propose different module structures. When it comes to the module structure, I think the one by the PostgreSQL PR would be more scalable. Meaning placing PostgreSQL, Redis, and SQLAlchemy in the extensions directory.

@rm-openai Do you have any thoughts on adding SQLAlchemy and the module structure for additional sessions storage layer?

[rm-openai]

@seratch I like it. I think what makes the most sense is to have a separate package (in this repo) of session implementations. Extensions is fine too, but I do like the structure of the JS repo with a few smaller packages in a monorepo.

[seratch]

@rm-openai Cons of having multiple packages are that the release process could become a bit more complex. We probably won't have more than 10 implementations for memory, so I feel we may not need a separate package for that. Also, while JavaScript/TypeScript communities prefer packages to be as small as possible, I don't think that's exactly the case for Python.

For these reasons, my recommendation for Python is to start with the extensions.memory (or extensions.sessions?) directory. Having many optional dependencies shouldn't be an issue for users. If we decide to split the package into several smaller ones later, we can do so with a major or minor version bump.

If you’d like to handle the two PRs, I’ll leave it up to you. If we reach a consensus here, I’m happy to work on it as well!

[artificial-aidan]

Because of Python's native "extras" support, I vote for not making separate packages. By capturing the dependencies in the extra dependencies group, we negate dependency bloat for the average user, and the additional code is not of meaningful size to cause issues.

In my PR I did src/agents/extensions/memory but happy to change to sessions if needed.

Additionally, I think this PR should be updated to have redis be in extensions as well.