Automated database changelog digest — scrapes release notes, engineering blogs, and GitHub releases, summarises them with Claude AI, and delivers a structured digest + podcast-style audio to Slack.
Currently supported: MongoDB
Planned: PostgreSQL, MySQL, Redis, Cassandra, Elasticsearch
┌──────────┐ ┌─────────────────────┐ ┌───────────┐ ┌──────────────┐ ┌───────────────┐
│ Scraper │──▶│ Storage (dedup) │──▶│ Summarizer│──▶│ Renderer │──▶│ Notifier │
│ RSS │ │ SQLite (local dev) │ │ Claude AI │ │ Markdown │ │ Slack (text) │
│ GitHub │ │ GCS (Cloud Run) │ │ Two-pass │ │ Podcast .md │ │ Slack (audio) │
│ Releases │ └─────────────────────┘ │ JSON + │ │ MP3 via GCP │ │ Email │
└──────────┘ │ plain text│ │ TTS │ └───────────────┘
└───────────┘ └──────────────┘
Each run:
- Scrapes new articles from RSS feeds and GitHub Releases
- Deduplicates against seen URLs (SQLite locally, GCS JSON on Cloud Run)
- Summarises with Claude — two API calls per database to avoid JSON/prose conflicts
- Renders a structured
.mddigest + a podcast-style transcript - Synthesises the transcript to MP3 via Google Cloud TTS (Neural2, free tier)
- Delivers text to Slack via webhook + MP3 as an inline Slack audio player via bot token
- Go 1.22+ with CGO (needed for SQLite)
gcc/musl-dev- An Anthropic API key
git clone https://github.com/your-org/dbdigest && cd dbdigest
# Required
export ANTHROPIC_API_KEY="sk-ant-..."
# Optional — needed for Slack notifications
export SLACK_WEBHOOK="https://hooks.slack.com/services/..."
export SLACK_TOKEN="xoxb-..." # bot token for audio upload
export SLACK_CHANNEL_ID="C0123ABC" # right-click channel → Copy Channel ID
# Dry run first — scrapes sources, prints what was found, no Claude API calls
make dry-run
# Full run
make run
# Resend the last generated digest without re-scraping
make resendGenerated files land in ./digests/:
digests/
├── dbdigest_2026-04-11_090000.md # structured digest
├── dbdigest_2026-04-11_090000_podcast.md # podcast transcript
└── dbdigest_2026-04-11_090000_podcast.mp3 # audio (if TTS enabled)
One-time infrastructure provisioning via Terraform, then CI/CD handles every deploy automatically.
- GCP project with billing enabled
terraform>= 1.6gcloudCLI, authenticated (gcloud auth login && gcloud auth application-default login)- A GitHub repository for this code
cd terraform
cp terraform.tfvars.example terraform.tfvars
# Edit terraform.tfvars — set project_id, gcs_bucket, github_repo at minimummake tf-init
make tf-plan # review what will be created
make tf-applyThis creates:
| Resource | Purpose |
|---|---|
| Artifact Registry repo | Stores Docker images |
| GCS bucket | Digest state, markdown files, MP3 audio |
| Cloud Run Job | Runs the digest pipeline |
| Cloud Scheduler | Triggers the job on a cron schedule (default: Mon 9 AM UTC) |
| 4 Secret Manager secrets | API keys / webhook URLs |
| 3 Service accounts | Runner, Scheduler, CI — minimal permissions each |
| Workload Identity pool | GitHub Actions auth without long-lived keys |
After apply, the output will show the exact commands for next steps.
echo -n "sk-ant-..." | gcloud secrets versions add dbdigest-anthropic-api-key --data-file=-
echo -n "https://hooks.slack.com/..." | gcloud secrets versions add dbdigest-slack-webhook --data-file=-
echo -n "xoxb-..." | gcloud secrets versions add dbdigest-slack-token --data-file=-
echo -n "ghp_..." | gcloud secrets versions add dbdigest-github-token --data-file=- # optionalgcloud auth configure-docker ${REGION}-docker.pkg.dev
make docker-pushIn your repo: Settings → Secrets and variables → Variables → New repository variable
| Variable | Value (from terraform output) |
|---|---|
GCP_PROJECT_ID |
your GCP project ID |
GCP_REGION |
e.g. us-central1 |
GCP_WIF_PROVIDER |
output of terraform output wif_provider |
GCP_CI_SA |
output of terraform output ci_service_account |
After this, every push to master automatically builds a new image and updates the Cloud Run Job.
gcloud run jobs execute dbdigest --region us-central1 --waitAll configuration lives in config.yaml. Sensitive values are read from environment variables (${VAR_NAME} syntax).
storage:
type: "sqlite" # local dev
# type: "gcs" # Cloud Run — uses GCS for state + digest files
# bucket: "${GCS_BUCKET}"
# digests_prefix: "digests/"summarizer:
provider: "anthropic"
api_key: "${ANTHROPIC_API_KEY}"
model: "claude-sonnet-4-20250514"
max_tokens: 8096output:
format: "podcast" # generates both structured markdown and podcast transcript
dir: "./digests"tts:
enabled: false # set to true to generate MP3
voice: "en-US-Neural2-D" # Neural2 voices = free tier (1M chars/month)
language_code: "en-US"Available Neural2 voices:
| Voice | Character |
|---|---|
en-US-Neural2-D |
Male, clear and neutral |
en-US-Neural2-F |
Female, natural pace |
en-US-Neural2-J |
Male, slightly warmer |
notifier:
type: "slack"
slack_webhook: "${SLACK_WEBHOOK}" # for text digest
slack_token: "${SLACK_TOKEN}" # xoxb-... bot token for audio upload
slack_channel_id: "${SLACK_CHANNEL_ID}" # C0123ABCRequired Slack app scopes for audio upload: files:write, chat:write
databases:
mongodb:
enabled: true
sources:
- name: "MongoDB Official Blog"
type: "rss"
url: "https://www.mongodb.com/blog/rss"
tags: ["official"]
- name: "MongoDB Server GitHub Releases"
type: "github-releases"
url: "https://api.github.com/repos/mongodb/mongo/releases"
github_token: "${GITHUB_TOKEN}" # optional; raises rate limit 60→5000/hr
lookback_days: 90
# Add a new database:
# postgresql:
# enabled: true
# sources:
# - name: "PostgreSQL News"
# type: "rss"
# url: "https://www.postgresql.org/news.rss"./dbdigest [flags]
--config string Path to config file (default "config.yaml")
--dry-run Scrape only — no Claude, no writes, no notifications
--resend Resend the last digest without re-scraping
--channel string Override notification channel for --resend (slack, email)
--verbose Enable debug logging
--version Show version
make run Scrape → summarize → notify (full pipeline)
make dry-run Scrape only — no Claude calls, no writes, no notifications
make resend Resend the last digest + audio via configured channel
make resend-slack Resend the last digest to Slack specifically
make verbose Full pipeline with debug logging
make build Compile the binary
make clean Remove binary, local DB, and generated digests
make docker-build Build the Docker image
make docker-push Build and push to Artifact Registry
make docker-run Run locally in Docker using SQLite
make tf-init Initialise Terraform (run once)
make tf-plan Preview infrastructure changes
make tf-apply Provision / update GCP infrastructure
make tf-destroy Tear down all infrastructure
dbdigest/
├── main.go # Entry point, CLI flags, storage factory
├── config.yaml # Configuration (edit this)
├── Dockerfile # Multi-stage build for Cloud Run Jobs
├── Makefile
│
├── internal/
│ ├── config/config.go # YAML config loader + validation
│ ├── scraper/scraper.go # RSS, web, GitHub Releases scrapers
│ ├── digest/
│ │ ├── models.go # Shared data types (Article, Digest, etc.)
│ │ ├── summarizer.go # Claude API: two-pass JSON + plain-text
│ │ └── render.go # Markdown + podcast transcript renderer
│ ├── tts/tts.go # Google Cloud TTS — chunked MP3 synthesis
│ ├── storage/
│ │ ├── store.go # Store interface + SQLite implementation
│ │ └── gcs.go # GCS implementation (Cloud Run)
│ ├── notify/
│ │ ├── notify.go # Channel interface + AudioSender interface
│ │ ├── slack.go # Slack: webhook (text) + Files v2 API (audio)
│ │ ├── email.go # SMTP email
│ │ └── noop.go # Discard (type: none)
│ └── pipeline/pipeline.go # Orchestration: scrape→summarize→render→notify
│
└── terraform/
├── main.tf # GCP resources
├── variables.tf
├── outputs.tf
├── versions.tf
└── terraform.tfvars.example
Add a database — add an entry to databases: in config.yaml. No code changes needed for RSS or GitHub Releases sources.
Add a notifier (Discord, Teams, WhatsApp) — implement notify.Channel in a new file under internal/notify/, add a case in New().
Add a scraper type — add a case in internal/scraper/scraper.go → scrapeSource().
MIT