Dapr state store clickhouse

[middt]

Description

This PR adds a new state store component for ClickHouse, a column-oriented database management system. The ClickHouse state store component provides the following features:

• Basic CRUD operations for state management
• Support for TTL (Time-To-Live)
• ETag support for optimistic concurrency
• Bulk operations support
• Username/password authentication
• Uses ReplacingMergeTree engine for efficient updates

Key implementation details:

• Uses the official ClickHouse Go driver
• Implements the full state.Store interface
• Handles connection management and cleanup
• Includes comprehensive integration tests
• Follows Dapr's component development guidelines

Implementation Details

The component includes:

1. State store implementation (clickhouse.go)

   • Full CRUD operations
   • Bulk operations
   • TTL support
   • ETag support
   • Authentication support

2. Tests (clickhouse_test.go)

   • Integration tests for CRUD operations
   • Metadata validation tests
   • Authentication tests
   • Cleanup handling

3. Configuration options:

   apiVersion: dapr.io/v1alpha1
   kind: Component
   metadata:
     name: statestore
     namespace: default
   spec:
     type: state.clickhouse
     version: v1
     metadata:
     - name: clickhouseURL
       value: "tcp://localhost:9000"
     - name: databaseName
       value: "dapr_state"
     - name: tableName
       value: "state_store"
     - name: username
       value: "default"
     - name: password
       value: ""

Checklist

• Code compiles correctly
• Created/updated tests
• Extended the documentation / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]

Testing Done

1. Unit tests for metadata validation
2. Integration tests for:
   • Basic CRUD operations
   • TTL functionality
   • ETag support
   • Bulk operations
   • Authentication
   • Error handling

Tests were run against ClickHouse v23.8 using the official Go driver.

Additional Notes

• The component uses ReplacingMergeTree engine for better update performance
• Includes proper cleanup in tests to avoid test pollution
• Follows ClickHouse best practices for table design
• Implements proper connection management and cleanup

[JoshVanL]

Thanks @middt, implementation generally looks good to me!

Would it be possible to add conformance tests for this too? It seems like we should be able to spin up a click house docker compose.

[middt]

Thank you for the feedback! @JoshVanL I've implemented the conformance tests for the ClickHouse state store in this PR:

1. Created a Docker Compose setup in /tests/certification/state/clickhouse/docker-compose.yml that spins up a ClickHouse server with proper configuration
2. Added a conformance test file in /tests/certification/state/clickhouse/clickhouse_test.go that verifies all state store operations
3. Created configuration files in /tests/config/state/clickhouse/clickhouse.yml for the test environment
4. Added ClickHouse to the state store tests configuration in /tests/config/state/tests.yml

The tests verify all the key functionality including:

• Basic CRUD operations
• ETag support for optimistic concurrency
• Error handling for empty and non-existent keys
• Feature reporting

I've also addressed authentication issues by properly configuring username and password in both the Docker Compose setup and the state store implementation.

All unit tests are now passing, confirming that the implementation works correctly with the ClickHouse server.

[JoshVanL]

Thanks @middt, I think the only thing left is to do a go mod tidy and it should be good to go 🙂

Appreciate the work on this!

[cicoyle]

Also, mind checking the linter locally? It appears to fail in CI

[middt]

Also, mind checking the linter locally? It appears to fail in CI

Also, mind checking the linter locally? It appears to fail in CI

Thanks for the feedback! @cicoyle I've already fixed all linter issues locally (import formatting, gosec warnings, variable shadowing, etc.) and the latest commits should pass CI.

[middt]

@middt thanks for the PR 🎉

mind opening a dapr/docs PR targeting the 1.16 branch documenting this PR's details?

Absolutely! I've created a corresponding PR targeting the v1.16 branch with the same ClickHouse state store documentation.
dapr/docs#4697

[nelson-parente]

@middt can you please run go mod tidy?

[middt]

@middt can you please run go mod tidy?

✅ Done! :)