Skip to content

Commit 5f9acf8

Browse files
Teodor Ciuraruclaude
Teodor Ciuraru
and
claude
committed
test(flutter): add attachment stress tests with memory monitoring
Add comprehensive stress testing for Ditto attachments: Test Suite: - Small attachment test: 50 tasks with 1KB attachments - Large attachment test: 10 tasks with 100KB attachments - Rapid create/delete cycles: 20 cycles of 5 tasks each - Memory tracking placeholder for future implementation Features: - Performance metrics for create/fetch/delete operations - Configurable test parameters - Helper script with optional memory monitoring (macOS) - Comprehensive documentation Files: - test/attachment_stress_test.dart: Main test suite - run_stress_test.sh: Convenience script for running tests - STRESS_TEST.md: Documentation and usage guide Usage: cd flutter_app ./run_stress_test.sh # Run tests ./run_stress_test.sh --watch # Monitor memory The memory tracking test is currently a placeholder. True memory profiling will require platform-specific implementation using native channels or Flutter DevTools. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
1 parent 9ce000f commit 5f9acf8

File tree

3 files changed

+692
-0
lines changed

3 files changed

+692
-0
lines changed

‎flutter_app/STRESS_TEST.md‎

Lines changed: 251 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,251 @@
1+
# Ditto Attachment Stress Testing
2+
3+
This document describes the attachment stress testing setup for the Flutter Ditto quickstart app.
4+
5+
## Quick Start
6+
7+
```bash
8+
# Run basic stress tests
9+
cd flutter_app
10+
./run_stress_test.sh
11+
12+
# Run with memory monitoring (macOS only)
13+
./run_stress_test.sh --watch
14+
15+
# View help
16+
./run_stress_test.sh --help
17+
```
18+
19+
## Test Suite Overview
20+
21+
The stress tests are located in `test/attachment_stress_test.dart` and include:
22+
23+
### 1. Small Attachment Test
24+
- **Purpose**: Test rapid creation and deletion of many small attachments
25+
- **Configuration**: 50 tasks with 1KB attachments each
26+
- **Metrics**:
27+
- Time to create all tasks with attachments
28+
- Time to fetch all attachments
29+
- Time to delete/evict all tasks
30+
- Average time per operation
31+
32+
### 2. Large Attachment Test
33+
- **Purpose**: Test handling of larger attachment payloads
34+
- **Configuration**: 10 tasks with 100KB attachments each
35+
- **Metrics**:
36+
- Time to create large attachments
37+
- Time to delete large attachments
38+
- Average time per operation
39+
40+
### 3. Rapid Create/Delete Cycle
41+
- **Purpose**: Simulate rapid churn of attachments
42+
- **Configuration**: 20 cycles × 5 tasks (100 create + 100 delete operations)
43+
- **Metrics**:
44+
- Total cycle time
45+
- Average operation time
46+
- Cleanup verification
47+
48+
### 4. Memory Tracking (Placeholder)
49+
- **Purpose**: Monitor memory usage during attachment operations
50+
- **Status**: Not yet implemented
51+
- **Future Implementation**: See [Memory Monitoring](#memory-monitoring) section
52+
53+
## Running Tests
54+
55+
### Basic Test Run
56+
```bash
57+
cd flutter_app
58+
flutter test test/attachment_stress_test.dart
59+
```
60+
61+
### Using the Helper Script
62+
```bash
63+
# Run all stress tests
64+
./run_stress_test.sh
65+
66+
# Run with memory monitoring
67+
./run_stress_test.sh --watch
68+
```
69+
70+
### Run Specific Test
71+
```bash
72+
flutter test test/attachment_stress_test.dart --name "Small Attachment"
73+
```
74+
75+
## Memory Monitoring
76+
77+
### Current Limitations
78+
The current memory tracking is a placeholder. True memory profiling requires platform-specific implementation.
79+
80+
### Future Implementation Options
81+
82+
#### Option 1: Flutter DevTools
83+
```bash
84+
# Install DevTools
85+
flutter pub global activate devtools
86+
87+
# Run DevTools
88+
flutter pub global run devtools
89+
90+
# Run app in profile mode
91+
flutter run --profile
92+
93+
# Use DevTools UI to monitor memory
94+
```
95+
96+
#### Option 2: Native Platform Channels
97+
Create platform channels to access native memory APIs:
98+
99+
**iOS (Objective-C/Swift)**:
100+
```swift
101+
// Get memory usage
102+
let info = mach_task_basic_info()
103+
// Access resident_size, virtual_size
104+
```
105+
106+
**Android (Kotlin)**:
107+
```kotlin
108+
val runtime = Runtime.getRuntime()
109+
val usedMemory = runtime.totalMemory() - runtime.freeMemory()
110+
```
111+
112+
#### Option 3: macOS Activity Monitor
113+
```bash
114+
# Watch dart process memory in real-time
115+
./run_stress_test.sh --watch
116+
```
117+
118+
### Manual Memory Profiling
119+
120+
1. **Baseline**: Record memory before test
121+
2. **Peak**: Monitor memory during attachment creation
122+
3. **Post-Operation**: Check memory after operations
123+
4. **Cleanup**: Verify memory is released after eviction
124+
5. **Cycles**: Run multiple times to detect leaks
125+
126+
## Performance Metrics
127+
128+
### Expected Benchmarks
129+
130+
These are approximate targets; actual performance varies by platform and device:
131+
132+
| Operation | Target | Notes |
133+
|-----------|--------|-------|
134+
| Create 1KB attachment | < 50ms | Per attachment creation |
135+
| Create 100KB attachment | < 200ms | Larger payloads |
136+
| Fetch 1KB attachment | < 100ms | Network dependent |
137+
| Evict all tasks | < 500ms | Batch operation |
138+
139+
### Analyzing Results
140+
141+
Look for:
142+
- **Linear scaling**: Time should scale linearly with attachment count/size
143+
- **Consistent performance**: Similar operations should take similar time
144+
- **Memory cleanup**: Memory should return to baseline after eviction
145+
- **No crashes**: All tests should complete without errors
146+
147+
## Customizing Tests
148+
149+
### Adjust Test Parameters
150+
151+
Edit `test/attachment_stress_test.dart`:
152+
153+
```dart
154+
// Change number of tasks
155+
const numTasks = 100; // Increase for more stress
156+
157+
// Change attachment size
158+
const attachmentSize = 1024 * 1024; // 1MB
159+
160+
// Change number of cycles
161+
const cycles = 50;
162+
```
163+
164+
### Add Custom Tests
165+
166+
```dart
167+
test('Custom stress test', () async {
168+
// Your custom test logic
169+
// - Create attachments with specific patterns
170+
// - Test concurrent operations
171+
// - Test error handling
172+
// - Test edge cases
173+
});
174+
```
175+
176+
## Troubleshooting
177+
178+
### Test Timeout
179+
If tests timeout, increase the timeout:
180+
```dart
181+
test('Long running test', () async {
182+
// Test code
183+
}, timeout: Timeout(Duration(minutes: 5)));
184+
```
185+
186+
### Out of Memory
187+
Reduce test parameters:
188+
- Decrease `numTasks`
189+
- Decrease `attachmentSize`
190+
- Add delays between operations
191+
192+
### Connection Issues
193+
Ensure Ditto credentials are correct in `.env`:
194+
```bash
195+
DITTO_APP_ID=your_app_id
196+
DITTO_PLAYGROUND_TOKEN=your_token
197+
DITTO_WEBSOCKET_URL=your_websocket_url
198+
```
199+
200+
## Platform-Specific Considerations
201+
202+
### iOS
203+
- Test on physical devices for accurate memory profiling
204+
- Use Instruments for detailed performance analysis
205+
- Monitor memory warnings
206+
207+
### Android
208+
- Use Android Studio Profiler for memory tracking
209+
- Test on various devices (different RAM configurations)
210+
- Monitor GC activity
211+
212+
### macOS
213+
- Use Activity Monitor for process memory
214+
- Check Console.app for Ditto logs
215+
- Test both Intel and Apple Silicon if possible
216+
217+
### Web
218+
- Use browser DevTools for memory profiling
219+
- Note: P2P sync is limited on web platform
220+
- Attachments may have different performance characteristics
221+
222+
## Future Enhancements
223+
224+
### Short Term
225+
- [ ] Implement native memory tracking
226+
- [ ] Add concurrent operation tests
227+
- [ ] Add error injection tests
228+
- [ ] Add network latency simulation
229+
230+
### Long Term
231+
- [ ] Automated CI/CD performance regression testing
232+
- [ ] Historical performance tracking
233+
- [ ] Cross-platform performance comparison
234+
- [ ] Memory leak detection automation
235+
- [ ] Load testing with multiple devices
236+
237+
## Resources
238+
239+
- [Flutter DevTools](https://docs.flutter.dev/tools/devtools)
240+
- [Ditto Attachments Documentation](https://docs.ditto.live)
241+
- [Flutter Performance Best Practices](https://docs.flutter.dev/perf/best-practices)
242+
- [Dart VM Performance](https://dart.dev/guides/language/performance)
243+
244+
## Contributing
245+
246+
When adding new stress tests:
247+
1. Document the test purpose and configuration
248+
2. Include relevant metrics
249+
3. Update this README
250+
4. Consider platform differences
251+
5. Add cleanup to avoid test pollution

‎flutter_app/run_stress_test.sh‎

Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
#!/bin/bash
2+
3+
# Ditto Attachment Stress Test Runner
4+
#
5+
# This script runs the attachment stress tests and optionally profiles memory usage.
6+
#
7+
# Usage:
8+
# ./run_stress_test.sh # Run tests normally
9+
# ./run_stress_test.sh --profile # Run with performance profiling
10+
# ./run_stress_test.sh --watch # Run and watch memory (macOS only)
11+
12+
set -e
13+
14+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
15+
cd "$SCRIPT_DIR"
16+
17+
echo "🧪 Ditto Attachment Stress Test Runner"
18+
echo "========================================"
19+
20+
# Check if .env file exists
21+
if [ ! -f "../.env" ]; then
22+
echo "❌ Error: .env file not found in parent directory"
23+
echo "Please create a .env file with your Ditto credentials"
24+
exit 1
25+
fi
26+
27+
# Parse arguments
28+
PROFILE=false
29+
WATCH_MEMORY=false
30+
31+
while [[ $# -gt 0 ]]; do
32+
case $1 in
33+
--profile)
34+
PROFILE=true
35+
shift
36+
;;
37+
--watch)
38+
WATCH_MEMORY=true
39+
shift
40+
;;
41+
--help)
42+
echo ""
43+
echo "Usage: $0 [OPTIONS]"
44+
echo ""
45+
echo "Options:"
46+
echo " --profile Enable Dart VM performance profiling"
47+
echo " --watch Watch memory usage during test (macOS only)"
48+
echo " --help Show this help message"
49+
echo ""
50+
exit 0
51+
;;
52+
*)
53+
echo "Unknown option: $1"
54+
echo "Run with --help for usage information"
55+
exit 1
56+
;;
57+
esac
58+
done
59+
60+
# Check dependencies
61+
if ! command -v flutter &> /dev/null; then
62+
echo "❌ Flutter not found. Please install Flutter first."
63+
exit 1
64+
fi
65+
66+
echo ""
67+
echo "📋 Test Configuration:"
68+
echo " Profile Mode: $PROFILE"
69+
echo " Watch Memory: $WATCH_MEMORY"
70+
echo ""
71+
72+
# Function to monitor memory usage (macOS only)
73+
monitor_memory() {
74+
if [[ "$OSTYPE" == "darwin"* ]] && [ "$WATCH_MEMORY" = true ]; then
75+
echo "📊 Memory monitoring enabled (checking every 2s)..."
76+
echo ""
77+
78+
# Start memory monitoring in background
79+
(
80+
while true; do
81+
# Get dart process memory
82+
DART_PROCS=$(ps aux | grep "[d]art" | grep -v grep)
83+
if [ -n "$DART_PROCS" ]; then
84+
echo "[$(date '+%H:%M:%S')] Dart Process Memory:"
85+
echo "$DART_PROCS" | awk '{printf " PID: %-8s RSS: %-8s VSZ: %-10s CMD: %s\n", $2, $6/1024"MB", $5/1024"MB", $11}'
86+
fi
87+
sleep 2
88+
done
89+
) &
90+
MONITOR_PID=$!
91+
92+
# Cleanup function
93+
cleanup() {
94+
if [ -n "$MONITOR_PID" ]; then
95+
kill $MONITOR_PID 2>/dev/null || true
96+
fi
97+
}
98+
trap cleanup EXIT
99+
fi
100+
}
101+
102+
# Start memory monitoring if requested
103+
monitor_memory
104+
105+
echo "🚀 Running stress tests..."
106+
echo ""
107+
108+
# Build test command
109+
TEST_CMD="flutter test test/attachment_stress_test.dart"
110+
111+
if [ "$PROFILE" = true ]; then
112+
echo "📈 Performance profiling enabled"
113+
# Note: Flutter test doesn't support --profile directly
114+
# For profiling, you'd need to run in profile mode on a device
115+
echo "⚠️ Note: For detailed profiling, run the app in profile mode on a device"
116+
echo " Use: flutter run --profile and trigger stress tests manually"
117+
echo ""
118+
fi
119+
120+
# Run the tests
121+
$TEST_CMD
122+
123+
echo ""
124+
echo "✅ Stress tests completed!"
125+
echo ""
126+
127+
# Show summary
128+
echo "📊 Test Summary:"
129+
echo " - Small attachments (50 x 1KB)"
130+
echo " - Large attachments (10 x 100KB)"
131+
echo " - Rapid create/delete cycles (20 cycles x 5 tasks)"
132+
echo " - Memory tracking placeholder"
133+
echo ""
134+
135+
if [ "$WATCH_MEMORY" = true ]; then
136+
echo "💡 Tip: Check the memory logs above for memory usage patterns"
137+
echo ""
138+
fi
139+
140+
echo "💡 Next Steps:"
141+
echo " - Review test output for performance metrics"
142+
echo " - Check for memory leaks by running multiple times"
143+
echo " - Run on physical devices for accurate memory profiling"
144+
echo " - Use Flutter DevTools for detailed profiling:"
145+
echo " flutter pub global activate devtools"
146+
echo " flutter pub global run devtools"
147+
echo ""

0 commit comments

Comments
 (0)