Skip to content

Commit 0231b72

Browse files
MickLeskMickLesk
committed
Add telemetry data service and dashboard revamp
Introduce a telemetry data microservice under misc/data: add Dockerfile, entrypoint, migration tools, README, LICENSE and a .gitignore. Increase Docker CACHE_TTL_SECONDS to 300s. Implement extensive dashboard and analytics updates in dashboard.go: add total_all_time and sample_size, return total item counts from fetchRecords (with page/limit handling and a maxRecords guard), raise top-N limits, add a minimum-installs threshold for failed-apps, and numerous UI/style/layout improvements in the embedded DashboardHTML. Minor formatting tweak to misc/api.func.
1 parent e4a8ee8 commit 0231b72

File tree

16 files changed

+3088
-720
lines changed

16 files changed

+3088
-720
lines changed

‎misc/api.func‎

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -87,17 +87,17 @@ detect_repo_source() {
8787

8888
# Map detected owner/repo to canonical repo_source value
8989
case "$owner_repo" in
90-
community-scripts/ProxmoxVE) REPO_SOURCE="ProxmoxVE" ;;
91-
community-scripts/ProxmoxVED) REPO_SOURCE="ProxmoxVED" ;;
92-
"")
93-
# No URL detected — use hardcoded fallback
94-
# CI sed transforms this on promotion: ProxmoxVED → ProxmoxVE
95-
REPO_SOURCE="ProxmoxVED"
96-
;;
97-
*)
98-
# Fork or unknown repo
99-
REPO_SOURCE="external"
100-
;;
90+
community-scripts/ProxmoxVE) REPO_SOURCE="ProxmoxVE" ;;
91+
community-scripts/ProxmoxVED) REPO_SOURCE="ProxmoxVED" ;;
92+
"")
93+
# No URL detected — use hardcoded fallback
94+
# CI sed transforms this on promotion: ProxmoxVED → ProxmoxVE
95+
REPO_SOURCE="ProxmoxVED"
96+
;;
97+
*)
98+
# Fork or unknown repo
99+
REPO_SOURCE="external"
100+
;;
101101
esac
102102

103103
export REPO_SOURCE

‎misc/data/.gitignore‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# If you prefer the allow list template instead of the deny list, see community template:
2+
# https://github.com/github/gitignore/blob/main/community/Golang/Go.AllowList.gitignore
3+
#
4+
# Binaries for programs and plugins
5+
*.exe
6+
*.exe~
7+
*.dll
8+
*.so
9+
*.dylib
10+
telemetry-service
11+
migration/migrate
12+
13+
# Test binary, built with `go test -c`
14+
*.test
15+
16+
# Code coverage profiles and other test artifacts
17+
*.out
18+
coverage.*
19+
*.coverprofile
20+
profile.cov
21+
22+
# Dependency directories (remove the comment below to include it)
23+
# vendor/
24+
25+
# Go workspace file
26+
go.work
27+
go.work.sum
28+
29+
# env file
30+
.env
31+
32+
# Editor/IDE
33+
# .idea/
34+
# .vscode/

‎misc/data/Dockerfile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ ENV ENABLE_REQUEST_LOGGING="false"
2424

2525
# Cache config (optional)
2626
ENV ENABLE_CACHE="true"
27-
ENV CACHE_TTL_SECONDS="60"
27+
ENV CACHE_TTL_SECONDS="300"
2828
ENV ENABLE_REDIS="false"
2929
# ENV REDIS_URL="redis://localhost:6379"
3030

‎misc/data/LICENSE‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2026 Community Scripts
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎misc/data/README.md‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# Telemetry Service
2+
3+
A standalone Go microservice that collects anonymous telemetry data from [ProxmoxVE](https://github.com/community-scripts/ProxmoxVE) and [ProxmoxVED](https://github.com/community-scripts/ProxmoxVED) script installations.
4+
5+
## Overview
6+
7+
This service acts as a telemetry ingestion layer between the bash installation scripts and a PocketBase backend. When users run scripts from the ProxmoxVE/ProxmoxVED repositories, optional anonymous usage data is sent here for aggregation and analysis.
8+
9+
**What gets collected:**
10+
- Script name and installation status (success/failed)
11+
- Container/VM type and resource allocation (CPU, RAM, disk)
12+
- OS type and version
13+
- Proxmox VE version
14+
- Anonymous session ID (randomly generated UUID)
15+
16+
**What is NOT collected:**
17+
- IP addresses (not logged, not stored)
18+
- Hostnames or domain names
19+
- User credentials or personal information
20+
- Hardware identifiers (MAC addresses, serial numbers)
21+
- Network configuration or internal IPs
22+
- Any data that could identify a person or system
23+
24+
**What this enables:**
25+
- Understanding which scripts are most popular
26+
- Identifying scripts with high failure rates
27+
- Tracking resource allocation trends
28+
- Improving script quality based on real-world data
29+
30+
## Features
31+
32+
- **Telemetry Ingestion** - Receives and validates telemetry data from bash scripts
33+
- **PocketBase Integration** - Stores data in PocketBase collections
34+
- **Rate Limiting** - Configurable per-IP rate limiting to prevent abuse
35+
- **Caching** - In-memory or Redis-backed caching support
36+
- **Email Alerts** - SMTP-based alerts when failure rates exceed thresholds
37+
- **Dashboard** - Built-in HTML dashboard for telemetry visualization
38+
- **Migration Tool** - Migrate data from external sources to PocketBase
39+
40+
## Architecture
41+
42+
```
43+
┌─────────────────┐ ┌───────────────────┐ ┌────────────┐
44+
│ Bash Scripts │────▶│ Telemetry Service │────▶│ PocketBase │
45+
│ (ProxmoxVE/VED) │ │ (this repo) │ │ Database │
46+
└─────────────────┘ └───────────────────┘ └────────────┘
47+
```
48+
49+
## Project Structure
50+
51+
```
52+
├── service.go # Main service, HTTP handlers, rate limiting
53+
├── cache.go # In-memory and Redis caching
54+
├── alerts.go # SMTP alert system
55+
├── dashboard.go # Dashboard HTML generation
56+
├── migration/
57+
│ ├── migrate.go # Data migration tool
58+
│ └── migrate.sh # Migration shell script
59+
├── Dockerfile # Container build
60+
├── entrypoint.sh # Container entrypoint with migration support
61+
└── go.mod # Go module definition
62+
```
63+
64+
## Related Projects
65+
66+
- [ProxmoxVE](https://github.com/community-scripts/ProxmoxVE) - Proxmox VE Helper Scripts
67+
- [ProxmoxVED](https://github.com/community-scripts/ProxmoxVED) - Proxmox VE Helper Scripts (Dev)
68+
69+
## Privacy & Compliance
70+
71+
This service is designed with privacy in mind and is **GDPR/DSGVO compliant**:
72+
73+
- ✅ **No personal data** - Only anonymous technical metrics are collected
74+
- ✅ **No IP logging** - Request logging is disabled by default, IPs are never stored
75+
- ✅ **Transparent** - All collected fields are documented and the code is open source
76+
- ✅ **No tracking** - Session IDs are randomly generated and cannot be linked to users
77+
- ✅ **No third parties** - Data is only stored in our self-hosted PocketBase instance
78+
79+
## License
80+
81+
MIT License - see [LICENSE](LICENSE) file.

0 commit comments

Comments
 (0)