Skip to content

Commit e14f4a5

Browse files
authored
docs: update client-examples README with common design principles (#54)
- Add Common Design Principles section highlighting shared characteristics - Emphasize use of community-recommended clients and AutoMQ best practices - Document Docker Compose readiness and built-in metrics - Clarify production-ready patterns across all examples
1 parent 818615a commit e14f4a5

File tree

1 file changed

+177
-1
lines changed

1 file changed

+177
-1
lines changed

client-examples/README.md

Lines changed: 177 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,177 @@
1-
# Placeholder
1+
# AutoMQ Client Examples
2+
3+
This directory contains comprehensive client examples for interacting with AutoMQ using different programming languages. Each example demonstrates both basic and advanced Kafka operations with performance metrics and best practices optimized for AutoMQ's object storage architecture.
4+
5+
## Common Design Principles
6+
7+
All client examples in this repository share the following characteristics:
8+
9+
🎯 **Community-Recommended Clients**: Each example uses the client library recommended by both AutoMQ and the Apache Kafka community for optimal compatibility and performance.
10+
11+
⚙️ **AutoMQ Best Practices**: All clients are pre-configured with parameters following AutoMQ's recommended [best practices](https://www.automq.com/docs/automq-cloud/getting-started/client-sdk-guide), including optimized batch sizes, linger times, and timeout settings for object storage architecture.
12+
13+
🐳 **Docker Compose Ready**: Every example includes Docker Compose configuration for quick deployment and can be seamlessly integrated with AutoMQ cluster Docker Compose setups.
14+
15+
📊 **Built-in Metrics**: All examples automatically print comprehensive performance metrics including latency measurements, throughput statistics, and operational insights.
16+
17+
🚀 **Production-Ready**: Examples demonstrate real-world patterns with proper error handling, resource management, and concurrent processing capabilities.
18+
19+
## Available Client Examples
20+
21+
### 🔥 [Java Examples](./java/)
22+
23+
Java examples using the Apache Kafka client library with Maven build system.
24+
25+
**Available Examples:**
26+
- **SimpleMessageExample**: Basic producer/consumer operations
27+
- **TransactionalMessageExample**: ACID-compliant transactional messaging
28+
- **HubSpotContactDemo**: Real-world integration example
29+
- Performance metrics and Docker support included
30+
31+
### 🚀 [Go Examples](./go/)
32+
33+
Go examples using the franz-go client library.
34+
35+
**Available Examples:**
36+
- **SimpleMessageExample**: Basic producer/consumer operations
37+
- **TransactionalMessageExample**: Transactional operations with ACID guarantees
38+
- Performance metrics and Docker support included
39+
40+
### 🐍 [Python Examples](./python/)
41+
42+
Python examples using the kafka-python library.
43+
44+
**Available Examples:**
45+
- **SimpleMessageExample**: Basic producer/consumer operations
46+
- Performance metrics and Docker support included
47+
- Multiple execution methods (direct Python, Docker, shell script)
48+
49+
### [C++ Examples](./cpp/)
50+
51+
*Note: C++ examples are currently TODO. Implementation is planned for future releases.*
52+
53+
### 🌐 JavaScript Examples
54+
55+
*Note: JavaScript examples are currently TODO. Implementation is planned for future releases.*
56+
57+
### 🦀 Rust Examples
58+
59+
*Note: Rust examples are currently TODO. Implementation is planned for future releases.*
60+
61+
## Common Features Across All Examples
62+
63+
### 🎯 **Performance Metrics**
64+
All examples provide detailed performance analytics:
65+
- **Total Messages**: Configured message count
66+
- **Messages Sent/Received**: Actual successful operations
67+
- **Total Execution Time**: End-to-end processing time
68+
- **Average Produce Latency**: Time from send() to broker acknowledgment
69+
- **Average End-to-End Latency**: Time from message creation to consumption
70+
- **Consume Time**: Duration of the consumption phase
71+
72+
### 🔧 **AutoMQ Optimizations**
73+
All clients are configured with AutoMQ-specific optimizations:
74+
- **Batch Size**: 1MB for optimal object storage performance
75+
- **Linger Time**: 100ms for efficient message batching
76+
- **Max Request Size**: 16MB for large message batches
77+
- **Metadata Refresh**: 60 seconds for balanced performance
78+
- **Acknowledgments**: "all" for durability guarantees
79+
80+
### 🐳 **Docker Support**
81+
Every example includes:
82+
- Dockerfile with multi-stage builds
83+
- Docker Compose configuration
84+
- Environment variable configuration
85+
- Network integration with AutoMQ clusters
86+
- Security best practices (non-root execution)
87+
88+
### 🔒 **Production-Ready Features**
89+
- Comprehensive error handling and retry logic
90+
- Proper resource management and cleanup
91+
- Configurable timeouts and connection settings
92+
- Logging with appropriate levels
93+
- Thread-safe operations and concurrency handling
94+
95+
## Quick Start
96+
97+
### Prerequisites
98+
- Running AutoMQ cluster
99+
- Docker and Docker Compose (for containerized deployment)
100+
- Language-specific requirements (see individual README files)
101+
102+
### Running Examples
103+
104+
Each language directory contains detailed instructions, but here's a quick overview:
105+
106+
```bash
107+
# Java
108+
cd java && mvn clean package && docker compose up --build
109+
110+
# Go
111+
cd go && go build -o kafka-examples . && ./kafka-examples
112+
113+
# Python
114+
cd python && ./run.sh
115+
116+
# Or use Docker for any language
117+
cd <language> && docker compose up --build
118+
```
119+
120+
### Configuration
121+
122+
All examples support environment variable configuration:
123+
- `BOOTSTRAP_SERVERS`: AutoMQ broker addresses (default: localhost:9092)
124+
- Additional language-specific variables (see individual READMEs)
125+
126+
## Architecture Overview
127+
128+
```
129+
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
130+
│ Producer │ │ AutoMQ │ │ Consumer │
131+
│ │───▶│ Cluster │───▶│ │
132+
│ • Batching │ │ │ │ • Group Mgmt │
133+
│ • Compression │ │ • Object Store │ │ • Offset Mgmt │
134+
│ • Retries │ │ • Partitioning │ │ • Rebalancing │
135+
└─────────────────┘ └─────────────────┘ └─────────────────┘
136+
```
137+
138+
## Performance Tuning
139+
140+
For optimal performance with AutoMQ:
141+
142+
1. **Batch Configuration**: Use larger batch sizes (1MB) to reduce object storage operations
143+
2. **Linger Time**: Set appropriate linger time (100ms) for batching efficiency
144+
3. **Compression**: Enable compression for better network utilization
145+
4. **Connection Pooling**: Reuse connections and clients when possible
146+
5. **Monitoring**: Use the built-in metrics for performance analysis
147+
148+
## Troubleshooting
149+
150+
### Common Issues
151+
152+
1. **Connection Refused**: Verify AutoMQ cluster is running and accessible
153+
2. **Topic Not Found**: Ensure topics exist or enable auto-creation
154+
3. **Permission Denied**: Check Kafka ACLs and authentication settings
155+
4. **High Latency**: Review batch size and linger time configuration
156+
5. **Memory Issues**: Adjust JVM settings or container memory limits
157+
158+
### Getting Help
159+
160+
- Check individual language README files for specific guidance
161+
- Review AutoMQ documentation: https://www.automq.com/docs
162+
- Examine log files in the `logs/` directory of each example
163+
- Use Docker logs: `docker compose logs -f`
164+
165+
## Contributing
166+
167+
We welcome contributions to improve these examples:
168+
169+
1. Fork the repository
170+
2. Create a feature branch
171+
3. Add tests for new functionality
172+
4. Update documentation
173+
5. Submit a pull request
174+
175+
## License
176+
177+
These examples are provided under the same license as the AutoMQ project.

0 commit comments

Comments
 (0)