Integration Scaffold — Project Specs

Overview

This repository is a reusable scaffold for rapidly bootstrapping HubSpot public app integrations. It provides a pre-configured HubSpot developer project (platform version 2025.2) with all core component types stubbed out, and a consistent pattern for standing up new integrations against external third-party APIs.

The goal is to eliminate the repetitive setup work each time a new HubSpot integration is built — auth flow, component directory structure, hsmeta configurations, UI extension boilerplate, webhook subscriptions, and workflow action schemas are all ready to clone and configure.

---

Repository Layout

Integrations/
├── hubspot.config.yml              # HubSpot CLI authentication config (shared across all integrations)
├── specs.md                        # This file — top-level project specs
│
├── integration Scaffold/           # Reusable template — copy this to start a new integration
│   ├── specs.md → (this file)
│   │
│   ├── hubspot/                    # HubSpot developer project
│   │   ├── hsproject.json          # Project config (name, srcDir, platformVersion)
│   │   ├── CLAUDE.md               # AI agent guidance for HubSpot development
│   │   ├── AGENTS.md               # Agent rules (mirrors CLAUDE.md)
│   │   ├── HUBSPOT_PROJECTS.md     # HubSpot projects reference
│   │   └── src/app/
│   │       ├── app-hsmeta.json             # App config: OAuth, scopes, permittedUrls
│   │       ├── cards/                      # CRM UI extension cards
│   │       ├── settings/                   # App settings page
│   │       ├── pages/                      # Full-page app UI
│   │       ├── webhooks/                   # HubSpot webhook subscriptions
│   │       ├── workflow-actions/           # Custom workflow actions
│   │       ├── app-objects/                # Custom CRM object definitions
│   │       └── app-object-associations/    # Object association definitions
│   │
│   └── aws/                        # AWS backend
│       ├── infra/                  # Terraform infrastructure-as-code
│       │   └── specs.md            # Infra specs (resources, modules, variables)
│       └── src/                    # Lambda source code
│           ├── specs.md            # Source overview (shared utilities, build, env vars)
│           ├── shared/             # Shared modules (HubSpot client, DynamoDB, Secrets, logger)
│           ├── poller/             # Scheduled polling Lambda
│           │   └── specs.md        # Poller specs (flow, error handling, field mapping)
│           ├── oauth/              # OAuth install/callback/uninstall Lambda
│           │   └── specs.md        # OAuth specs (flows, security, routes)
│           └── webhooks/           # HubSpot + external webhook receiver Lambda
│               └── specs.md        # Webhook specs (flows, idempotency, security)
│
└── <IntegrationName>/              # One directory per live integration (e.g., RunSignUp/)
    ├── hubspot/                    # Copied + configured from scaffold/hubspot/
    └── aws/                        # Copied + configured from scaffold/aws/

---

Scaffold Components

App (app-hsmeta.json)

• Distribution: marketplace (public app, requires OAuth)
• Auth type: oauth with configurable redirect URLs and scopes
• permittedUrls.fetch: list of external API base URLs the app is allowed to call via hubspot.fetch
• Placeholder support contact info to be replaced per integration

Card (cards/)

• CRM record tab extension using @hubspot/ui-extensions
• Renders on contact records by default; objectTypes is configurable
• Entry point: NewCard.tsx — stub component using hubspot.extend<'crm.record.tab'>
• Uses hubspot.fetch (not window.fetch) for all HTTP calls

Settings Page (settings/)

• App settings UI rendered via hubspot.extend<'settings'>
• Used for per-installation configuration (API keys, preferences, etc.)
• Stub component in SettingsPage.tsx

Pages (pages/)

• Full-page app UI via hubspot.extend for home/dashboard views
• Stub in Home.tsx

Webhooks (webhooks/webhooks-hsmeta.json)

• Pre-configured with example CRM object subscriptions (contact creation, property changes, deletion, privacy deletion)
• All subscriptions default to active: false — enable per integration need
• targetUrl must be updated to the integration's backend endpoint

Workflow Actions (workflow-actions/workflow-actions-hsmeta.json)

• Custom action stub with example input fields (string textarea, enumeration select)
• actionUrl must be updated to the integration's backend endpoint
• objectTypes defaults to CONTACT

App Objects & Associations (app-objects/, app-object-associations/)

• Stubs for defining custom CRM object schemas and their associations
• Only needed if the integration requires proprietary data objects in HubSpot CRM

---

Creating a New Integration

Step 1 — Copy the scaffold

Duplicate integration Scaffold/ and rename it to the integration name (e.g., RunSignUp/). Both hubspot/ and aws/ subdirectories come along — configure each independently.

Step 2 — Configure the HubSpot project (hubspot/)

1. Update hsproject.json — set name to the integration name
2. In src/app/app-hsmeta.json:
   • Update uid (e.g., app-runSignUp)
   • Set config.name
   • Add required OAuth scopes for the HubSpot APIs needed
   • Add the external API's base URL(s) to config.permittedUrls.fetch
   • Update support contact info
3. In webhooks/webhooks-hsmeta.json — set targetUrl to the API Gateway webhook endpoint; enable relevant subscriptions
4. In workflow-actions/workflow-actions-hsmeta.json — set actionUrl; define input fields for this integration
5. Remove component directories not needed (e.g., delete app-objects/ if no custom CRM objects)
6. Implement cards/NewCard.tsx and settings/SettingsPage.tsx
7. Deploy: hs project upload && hs project deploy

Step 3 — Provision AWS infrastructure (aws/infra/)

1. Copy terraform.tfvars.example → terraform.tfvars; fill in integration_name, environment, region, SNS topic ARN
2. terraform init && terraform apply
3. Manually populate the app-level secret in Secrets Manager (/integrations/{name}/app)

Step 4 — Deploy Lambda functions (aws/src/)

1. Implement poller/mapper.js — the integration-specific field mapping from external record → HubSpot properties
2. Implement oauth/callback.js additions — any integration-specific steps during install (e.g., external OAuth flow)
3. Implement webhooks/external.js if the external system supports outbound webhooks
4. Build and upload each Lambda artifact to S3
5. terraform apply to update Lambda with the new artifact

Step 5 — Wire up OAuth redirect URL

• Set the API Gateway callback URL in hubspot/src/app/app-hsmeta.json → auth.redirectUrls
• Set the webhook receiver URL in webhooks/webhooks-hsmeta.json → config.settings.targetUrl
• Redeploy the HubSpot project

---

OAuth / Auth Flow

• All integrations use HubSpot OAuth (auth.type: "oauth") since they are public/marketplace apps
• redirectUrls must include the integration backend's OAuth callback endpoint
• requiredScopes should be the minimum set needed — follow least-privilege
• The backend is responsible for handling the OAuth token exchange and storing tokens per HubSpot portal

---

Local Development

• Run hs project dev to start a local dev server with hot reload for cards and settings
• Use local.json to proxy hubspot.fetch calls to a local backend:

  {
    "proxy": {
      "https://your-api.example.com": "http://localhost:3000"
    }
  }

• Inject CLIENT_SECRET for testing request signing:

  CLIENT_SECRET="your-secret" hs project dev
  

---

Technical Constraints

• Platform version: 2025.2
• UI components: only @hubspot/ui-extensions exports are valid in cards and settings
• hubspot.fetch requires fully-qualified HTTPS URLs — no relative paths, no localhost
• All external URLs called via hubspot.fetch must be listed in app-hsmeta.json under permittedUrls.fetch
• Component UIDs must be unique within the project
• Components must live in their designated directories (no nested subdirectories)
• The app component's distribution: "marketplace" means auth.type must be oauth

---

Conventions

• One integration = one HubSpot project directory at the repo root
• All integration projects share the same hubspot.config.yml portal config
• Each integration backend is external to this repo; this repo contains only the HubSpot project (frontend + config)
• Name UIDs with the pattern: <componentType>-<integrationName> (e.g., card-runSignUp, webhooks-runSignUp)
• Keep unused scaffold components deleted — don't leave stubs in production projects

---

Polling Architecture

The standard integration pattern is a polling loop: the backend periodically queries the external system for changes and syncs them into HubSpot. This is preferable to full data replication — only IDs, references, and sync state are stored, not a copy of the source database.

How It Works

EventBridge Scheduler (every 3 min)
        │
        ▼
  Lambda Poller
        │
        ├─► For each active portal installation:
        │       │
        │       ├─► Read last cursor/timestamp from DynamoDB (sync state)
        │       ├─► Fetch changes from External API (since last cursor)
        │       ├─► Map external records → HubSpot properties
        │       ├─► Upsert records via HubSpot CRM API
        │       └─► Write new cursor/timestamp back to DynamoDB
        │
        └─► Emit metrics & logs to CloudWatch

Key Design Decisions

• Pull, don't copy: The poller fetches only what changed since the last run using a cursor, timestamp, or changelog endpoint. No full data dumps.
• Least data principle: Only the external record's ID, its corresponding HubSpot record ID, and sync metadata are persisted. PII and source record content are never stored.
• Per-portal isolation: Each installed HubSpot portal is processed independently. A failure or rate limit on one portal does not block others.
• Idempotent upserts: Records are upserted by external ID, so re-running the poller on the same data is safe.
• Cursor-based pagination: If the external API supports cursors or since parameters, use them. Fallback: store a lastPolledAt timestamp and filter on the external side.

---

AWS Infrastructure

Architecture Diagram

┌──────────────────────────────────────────────────────────────────┐
│                           AWS Account                            │
│                                                                  │
│  ┌─────────────────┐    ┌──────────────────────────────────┐    │
│  │ EventBridge     │    │  Lambda — Poller                  │    │
│  │ Scheduler       │───►│  Runtime: Node.js 22.x            │    │
│  │ rate(3 minutes) │    │  Timeout: 5 min                   │    │
│  └─────────────────┘    │  Memory: 256 MB                   │    │
│                         └──────────┬─────────────────────────    │
│                                    │                             │
│              ┌─────────────────────┼──────────────────┐         │
│              │                     │                   │         │
│              ▼                     ▼                   ▼         │
│  ┌─────────────────────┐  ┌──────────────────┐  ┌───────────┐  │
│  │ DynamoDB            │  │ Secrets Manager  │  │CloudWatch │  │
│  │                     │  │ /integrations/   │  │ Logs      │  │
│  │ [global]            │  │ {name}/app       │  │ Metrics   │  │
│  │  PortalRegistry     │  │                  │  │ Alarms    │  │
│  │                     │  │ /integrations/   │  └───────────┘  │
│  │ [per install]       │  │ {name}/{portal}/ │                  │
│  │  {portal}-{obj}-    │  │  hubspot         │                  │
│  │    SyncState        │  │  external        │                  │
│  │    IdMap            │  └──────────────────┘                  │
│  │    FieldMap         │                                         │
│  └─────────────────────┘                                         │
└──────────────────────────────────────────────────────────────────┘

---

Compute — AWS Lambda

Property	Value
Runtime	Node.js 22.x
Trigger	EventBridge Scheduler — rate(3 minutes)
Timeout	5 minutes (leave headroom under 15 min hard limit)
Memory	256 MB (increase if processing large page sizes)
Concurrency	Reserved concurrency = 1 per integration (prevent overlapping runs)
IAM Role	Least-privilege: DynamoDB read/write, Secrets Manager read, CloudWatch put metrics

Poller Lambda responsibilities:

1. Load the portal registry from DynamoDB to get all active portal installations
2. For each portal, read its sync state (last cursor/timestamp)
3. Fetch credentials from Secrets Manager for that portal
4. Call the external API for changes since last cursor
5. Upsert records in HubSpot via the CRM API using the portal's stored OAuth token
6. Refresh the HubSpot OAuth token if expired (write new token back to Secrets Manager)
7. Write the updated cursor/timestamp to DynamoDB sync state
8. Emit success/failure metrics to CloudWatch

Error handling:

• Catch and log per-portal errors without failing the entire invocation
• On HubSpot 429 (rate limit): back off and skip remaining records for that portal in this run; they will be caught on the next poll
• On external API errors: log and skip; preserve the previous cursor so records aren't lost
• Lambda itself is retried by EventBridge on invocation failure (configure retry = 0 for scheduled triggers to avoid duplicate runs)

---

Storage — DynamoDB

Design: One Database Per Portal Installation

Each HubSpot portal installation gets its own isolated set of DynamoDB tables. This is a deliberate security decision: a breach of one portal's data cannot expose any other portal's data, and a full uninstall is a clean table deletion with no residual data risk.

There is one global table shared across all portals, and three per-installation tables per synced object type created at install time and destroyed at uninstall.

Table Naming Convention

Global:
  {integrationName}-portal-registry

Per installation, per object type:
  {integrationName}-{portalId}-{objectType}-sync-state
  {integrationName}-{portalId}-{objectType}-id-map
  {integrationName}-{portalId}-{objectType}-field-map

Example (RunSignUp, portal 23786096, syncing contacts):
  runsignup-portal-registry
  runsignup-23786096-contacts-sync-state
  runsignup-23786096-contacts-id-map
  runsignup-23786096-contacts-field-map

Provisioning Responsibility

• PortalRegistry — created by Terraform at infra setup time. Never deleted by application code.
• Per-portal tables — created by the OAuth Lambda at install time. Deleted by the OAuth Lambda at uninstall time. Terraform is not involved in their lifecycle; instead, Terraform grants the Lambda IAM permission to create and delete tables matching the {integrationName}-* name pattern.

Table Limit Planning

DynamoDB has a soft limit of 2,500 tables per AWS account (raisable to 10,000 by request). With 3 tables per object type per portal, capacity planning is:

Object types synced	Tables per portal	Portals at soft limit	Portals at 10k limit
1	3	~833	~3,333
2	6	~416	~1,666
3	9	~277	~1,111

Request a limit increase before launch if significant growth is expected.

---

Global Table: {integrationName}-portal-registry

Tracks every HubSpot portal that has installed the app. One row per installation.

Attribute	Type	Description
portalId (PK)	String	HubSpot portal/hub ID
installedAt	String	ISO timestamp of OAuth install
active	Boolean	Whether this portal should be polled
externalAccountId	String	ID of the account in the external system — not credentials
syncedObjectTypes	List<String>	Object type slugs being synced (e.g., ["contacts"]) — determines which per-portal tables exist

---

Per-Portal Table: *-sync-state

One table per synced object type per portal. Tracks polling progress so the poller can resume from where it left off without re-processing records.

Attribute	Type	Description
objectType (PK)	String	Object type this row tracks (matches the table's object type — single-row table)
lastCursor	String	Cursor or ISO timestamp from the last successful poll
lastPolledAt	String	ISO timestamp of last poll attempt
lastSuccessAt	String	ISO timestamp of last fully successful poll
consecutiveErrors	Number	Count of consecutive failed polls (used for alerting)

Per-Portal Table: *-id-map

One table per synced object type per portal. Maps external system record IDs to HubSpot object IDs. This is the core of the least-data principle — IDs only, no record content.

Attribute	Type	Description
externalId (PK)	String	Record ID from the external system
hubspotObjectId	String	Corresponding HubSpot CRM object ID
syncedAt	String	ISO timestamp of last sync for this record

Per-Portal Table: *-field-map

One table per synced object type per portal. Stores the field mapping configuration for this installation — which external fields map to which HubSpot properties. Allows per-portal customization of the mapping without code changes.

Attribute	Type	Description
externalField (PK)	String	Field name from the external system
hubspotProperty	String	Target HubSpot property internal name
transformType	String	Optional transform to apply (none, uppercase, dateToTimestamp, etc.)
enabled	Boolean	Whether this mapping is active
updatedAt	String	ISO timestamp of last configuration change

Default rows are written at install time from the integration's default mapping config. A portal admin can update rows via the settings UI without redeploying code.

What is NOT stored in any table:

• Source record field values or content from the external system
• Copies of HubSpot record properties
• PII (names, emails, phone numbers) — always fetched live from the authoritative system at query time

---

Key Storage — AWS Secrets Manager

All credentials are stored in Secrets Manager using a consistent path scheme. No credentials ever touch DynamoDB or CloudWatch logs.

Secret Path Convention

/integrations/{integrationName}/{portalId}/hubspot
/integrations/{integrationName}/{portalId}/external
/integrations/{integrationName}/app

Secret: HubSpot OAuth Tokens (per portal)

Path: /integrations/{integrationName}/{portalId}/hubspot

{
  "accessToken": "...",
  "refreshToken": "...",
  "expiresAt": "2026-03-16T21:00:00.000Z"
}

• Updated in-place by the poller whenever a token refresh occurs
• Access is scoped to the Lambda IAM role; no other service or human should have read access in production

Secret: External API Credentials (per portal)

Path: /integrations/{integrationName}/{portalId}/external

{
  "apiKey": "...",
  "accountId": "..."
}

• Populated during the OAuth/onboarding flow when the user connects their external account
• If the external system uses OAuth, stores access + refresh tokens in the same shape as HubSpot tokens above

Secret: App-Level Secrets

Path: /integrations/{integrationName}/app

{
  "hubspotClientId": "...",
  "hubspotClientSecret": "...",
  "externalApiKey": "..."
}

• Shared across all portals — contains the app's own credentials, not per-installation credentials
• Rotated manually or via Secrets Manager rotation Lambda

---

Observability — CloudWatch

Log Groups

Log Group	Contents
/aws/lambda/{integrationName}-poller	Structured JSON logs from the poller Lambda

Structured log format (per poll run):

{
  "level": "INFO",
  "portalId": "12345678",
  "event": "sync_complete",
  "recordsProcessed": 42,
  "recordsFailed": 0,
  "durationMs": 8200,
  "newCursor": "2026-03-16T21:03:00.000Z"
}

Custom Metrics (emitted via PutMetricData)

Namespace: Integrations/{IntegrationName}

Metric	Unit	Description
RecordsSynced	Count	Records successfully upserted to HubSpot per run
RecordsFailed	Count	Records that failed to sync per run
PollDuration	Milliseconds	Total time for a full poll cycle
PortalsPolled	Count	Number of portals processed in a run
TokenRefreshes	Count	Number of OAuth token refreshes performed
ExternalApiErrors	Count	Errors calling the external system
HubSpotApiErrors	Count	Errors calling the HubSpot API

All metrics include a PortalId dimension where applicable to allow per-portal filtering.

Alarms

Alarm	Condition	Action
PollerErrors	Lambda errors > 0 in any 5-min window	SNS → email/PagerDuty
SyncLag	consecutiveErrors > 3 for any portal	SNS alert
RecordsFailed	> 0 records failed in a run	SNS alert
PollerNotRunning	No PollDuration metric for > 10 minutes	SNS alert (scheduler failure)
HubSpotApiErrors	> 10 errors in 15 minutes	SNS alert (rate limit or outage)

---

Multi-Portal Support

A single Lambda deployment serves all installed portals. Each portal's data is fully isolated in its own set of DynamoDB tables. The poller:

1. Scans PortalRegistry for all portals where active = true
2. For each portal, reads syncedObjectTypes to know which per-portal tables exist
3. For each object type, reads the portal's {portalId}-{objectType}-sync-state table for the cursor
4. Fetches credentials from Secrets Manager using the portal's ID
5. Processes records and writes results back to the portal's isolated tables
6. Emits metrics and logs tagged with portalId and objectType

Adding a new portal requires no infrastructure changes — the OAuth Lambda creates the tables at install time and the poller picks up the new PortalRegistry row on its next run.

---

Future Considerations

• Shared component library: common UI patterns (auth status banner, settings form fields, data table) extracted as shared utilities
• CLI scaffolding script: automate the copy-and-configure flow (rename UIDs, update URLs, set scopes) based on a config input
• Backend scaffold: a companion template for the Node.js/Express OAuth handler and webhook receiver that pairs with this HubSpot project scaffold
• IaC: Terraform or AWS CDK module that provisions the full Lambda + DynamoDB + Secrets Manager + CloudWatch stack from a single config file per integration
• Dead-letter queue: SQS DLQ on the EventBridge target to catch and inspect failed Lambda invocations
• Secrets rotation: automated Secrets Manager rotation Lambda for external API keys that support rotation