Skip to content

feat: add Elasticsearch as data storage backend#1229

Closed
zhangzhenghao wants to merge 1 commit intogorse-io:masterfrom
zhangzhenghao:add-elasticsearch-storage
Closed

feat: add Elasticsearch as data storage backend#1229
zhangzhenghao wants to merge 1 commit intogorse-io:masterfrom
zhangzhenghao:add-elasticsearch-storage

Conversation

@zhangzhenghao
Copy link
Copy Markdown
Contributor

Summary

Add Elasticsearch as a data storage backend option for Gorse.

Changes

  • storage/data/elasticsearch.go: New file implementing the Database interface (1715 lines)
  • storage/scheme.go: Add and URL prefixes
  • config/config.go: Add Elasticsearch prefix validation for data_store and cache_store
  • go.mod: Add github.com/elastic/go-elasticsearch/v8 dependency
  • storage/data/elasticsearch_test.go: Basic test suite

Usage

[database]
data_store = "elastic://user:password@localhost:9200/gorse"
# or with SSL:
data_store = "elastics://user:password@localhost:9200/gorse"

Implementation Details

  • Implements all Database interface methods (CRUD for users, items, feedback)
  • Supports batch operations and stream-based iteration
  • Uses cursor-based pagination with search_after for efficient large dataset handling
  • Proper index mappings for optimal query performance

Testing

  • Basic test suite added (elasticsearch_test.go)
  • Requires running Elasticsearch instance (set ELASTIC_URI env var for tests)

- Add Elasticsearch implementation of Database interface (1715 lines)
- Support elastic:// and elastics:// URL prefixes
- Implement all CRUD operations for users, items, and feedback
- Add batch operations and stream-based iteration
- Add configuration validation for Elasticsearch URLs
- Add basic test suite for Elasticsearch

Usage:
  data_store = "elastic://user:password@host:9200/database"
  data_store = "elastics://user:password@host:9200/database" (SSL)
@zhenghaoz zhenghaoz closed this Apr 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants