Skip to content

XiaokunDuan/palo-wire

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

19 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Palo Wire

Palo Wire is an agent-native source layer for Silicon Valley intelligence. It continuously refreshes a curated set of high-signal public sources and exposes lightweight JSON documents that downstream agents can read, rank, summarize, and synthesize.

Live Site · Documents API · Source Registry · Quickstart · Roadmap

TypeScript Cloudflare Workers Workers KV JSON API Scheduled ingestion

Why Palo Wire

Most trend-tracking products are built for people to browse.

Palo Wire is built for systems that need a clean, repeatable source layer. Instead of asking every downstream agent to rediscover the same feeds, pages, podcasts, and launch sources, Palo Wire keeps a small curated graph of public signals fresh and machine-readable.

That makes it useful for workflows such as:

  • daily Silicon Valley signal monitoring
  • source-grounded agent research
  • lightweight startup, product, and VC briefing pipelines
  • downstream ranking, summarization, and synthesis tasks

Product Shape

Palo Wire is not a traditional media product.

It is a continuously refreshed source node that:

  1. reads a curated source registry
  2. ingests sources using the most stable available public interface
  3. normalizes results into lightweight documents
  4. stores the latest snapshot per source
  5. exposes minimal APIs for downstream agents

In one line:

Palo Wire = a rolling, high-signal document layer for AI systems watching Silicon Valley.

Live Access

If you are a human, start with the landing page.

If you are an agent, start with:

  • /api/sources
  • /api/documents
  • /api/sources/:id/documents
  • /api/runs/latest

Design Principles

  • Agent-first: optimize for downstream model consumption
  • Official-entry-first: prefer feeds, sitemaps, APIs, and structured endpoints over brittle scraping
  • High-signal only: fewer sources, better sources
  • Minimal schema: keep documents cheap to ingest downstream
  • Rolling freshness: preserve the last good snapshot for low-frequency publishers
  • Operational restraint: fit inside the practical limits of Workers and KV

Coverage

Tech

  • Hacker News
  • Product Hunt
  • Y Combinator Launches
  • TechCrunch
  • Lenny's Newsletter
  • Lenny's Podcast
  • Latent Space
  • First Round Review
  • Y Combinator Blog

VC

  • Crunchbase News
  • a16z
  • a16z Podcast Network
  • Sequoia
  • Lightspeed
  • NFX
  • Benchmark

Ingestion Strategy

Palo Wire does not treat every source the same.

Each source is assigned an explicit ingestion mode:

  • rss_feed for stable public feeds
  • xml_sitemap for publisher-managed sitemap indexes
  • official_api when a vendor API is more stable than HTML
  • page_data for application shells with structured payloads
  • manual_curated for sources that need explicit curation

Public Interface

Root

  • GET /

Human-facing landing page.

Source Registry

  • GET /api/sources
  • GET /api/sources/:id

Source metadata plus latest snapshot summary.

Documents

  • GET /api/documents
  • GET /api/sources/:id/documents

Normalized article and podcast documents.

Links

  • GET /api/sources/:id/links

Outbound links extracted from the latest source snapshot.

Operations

  • GET /api/runs/latest
  • POST /api/sync

Operational status and manual sync entry point.

Repository Layout

src/                    Worker entry point, routing, sync logic
sources/                Curated source registry
docs/                   API and architecture notes
scripts/                Local support scripts
wrangler.jsonc          Cloudflare Worker configuration

Quickstart

Install dependencies:

npm install

Run locally:

npm run dev

Type-check:

npx tsc --noEmit

Deploy:

npm run deploy

Runtime Requirements

  • Cloudflare Workers
  • Workers KV
  • scheduled triggers
  • source registry configuration
  • optional authenticated sync trigger for POST /api/sync

Documentation

Roadmap

  • Add more source-specific ingest handlers where structured endpoints exist
  • Improve normalized signal extraction on top of raw snapshots
  • Expand operational visibility for crawl failures and source freshness
  • Add clearer public changelog and source coverage notes
  • Tighten downstream agent examples and starter clients

About

Agent-native source layer for tracking high-signal Silicon Valley product, startup, and VC signals.

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages