📰 NewsBlocks

NewsBlocks is a simple, visual way to see global news. It gathers headlines from major publishers, groups them into related stories using AI, and displays them as a treemap.

How it Works

The app runs on an automated pipeline that updates every 8 hours:

1. Gather: Scrapes massive RSS feeds from trusted elite sources (BBC, Reuters, FT) and Google Topic Aggregation for maximum data density.
2. Filter: Removes non-news content like podcasts, editorial guides, and pricing alerts.
3. Group: Uses AI embeddings to cluster similar headlines into a single "story."
4. Score: Gemini AI analyzes each story for sentiment (positive/negative) and relevance.
5. Clean: If a story only has one source, it's dropped unless it comes from an elite publisher or has a high relevance score.
6. Deploy: Updates the dashboard and generates static search-engine-friendly pages for every story.

The News Pipeline

graph TD
    A[RSS Feeds] -->|Scrape| B(Filter out noise)
    B -->|Clean Headlines| C(Group stories with AI)
    C -->|Story Clusters| D(Score Sentiment & Relevance)
    D --> E{Smart Signal Gate}
    
    E -->|Elite Source| F[✅ Keep]
    E -->|Multiple Sources| F
    E -->|High Importance| F
    E -->|Single Source Noise| G[❌ Drop]
    
    F --> H[Update Dashboard]
    F --> I[Generate SEO Pages]

Loading

Setup & Running Locally

1. Requirements

• Node.js (v18+)
• A Gemini API Key

2. Install

git clone https://github.com/prasadabhishek/newsblocks.git
cd newsblocks
npm install

3. Configure

Create a .env file in the root directory:

GEMINI_API_KEY=your_key_here

4. Run

• Development Server: npm run dev (View at http://localhost:5173)
• Data Update: node scripts/gather-news.js (Manually run the news scraper)
• Tests: npm run test (Run the unit tests)

Built With

• Frontend: React + D3.js (Responsive Treemap & Swipeable Mobile UI)
• AI: Google Gemini (Sentiment & Relevance Rankings)
• Aggregator: Massive Source Density (Google News, Reuters, BBC, TechCrunch)
• Persistence: Local JSON caching (improves speed and saves API costs)
• Hosting: Cloudflare Pages + GitHub Actions

---

Local Runner (Mac Mini / Always-On Setup)

For free AI processing, run the pipeline on a local Mac Mini with Ollama instead of GitHub Actions.

1. Requirements

• Ollama installed: ollama serve
• Node.js v18+
• GitHub Personal Access Token (for auto-push)

2. Install

# Clone repo to a shared location
sudo mkdir -p /Users/Shared/news-map
sudo chown $(whoami) /Users/Shared/news-map
git clone https://github.com/prasadabhishek/newsblocks.git /Users/Shared/news-map
cd /Users/Shared/news-map
npm install

# Create logs directory
mkdir -p /Users/Shared/news-map/logs

3. Configure

# Create .env with GitHub token
cat > /Users/Shared/news-map/.env << 'EOF'
GITHUB_TOKEN=ghp_your_token_here
RUN_INTERVAL_H=4
AI_PROVIDER=ollama
OLLAMA_HOST=http://localhost:11434
EOF

4. Install LaunchAgent (auto-start on boot)

# Copy plist template
cp scripts/com.newsblocks.runner.plist ~/Library/LaunchAgents/

# Edit the plist to set correct paths
# The path in ProgramArguments should match your install location

# Load the service
launchctl load ~/Library/LaunchAgents/com.newsblocks.runner.plist

# View logs
tail -f /Users/Shared/news-map/logs/runner.log

5. Manual Commands

# Start/stop the service
launchctl start com.newsblocks.runner
launchctl stop com.newsblocks.runner

# Restart after changes
launchctl unload ~/Library/LaunchAgents/com.newsblocks.runner.plist
launchctl load ~/Library/LaunchAgents/com.newsblocks.runner.plist

# Run once manually
cd /Users/Shared/news-map
node news-runner.js

---