feat(docs): lambda-ify load with url #4124
Conversation
coltondotio
had a problem deploying
to
Fern Dev
GitHub Actions
Failure
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
💡 Enable Vercel Agent with $100 free credit for automated AI reviews |
![seer-by-sentry[bot]](https://avatars.githubusercontent.com/in/801464?s=60&v=4){kind=small}
coltondotio
had a problem deploying
to
Fern Dev
GitHub Actions
Failure
🚀 FDR Lambda Preview DeployedYour Lambda function has been deployed to a preview environment! 🔗 Preview URL: https://jb5voktrj5.execute-api.us-east-1.amazonaws.com/preview-4124 📝 Available Endpoints:
📋 Example Usage: # Test default endpoint
curl "https://jb5voktrj5.execute-api.us-east-1.amazonaws.com/preview-4124"
# Test metadata endpoint (public - no auth required)
curl -X POST "https://jb5voktrj5.execute-api.us-east-1.amazonaws.com/preview-4124/metadata-for-url" \
-H "Content-Type: application/json" \
-d '{"url":"https://docs.buildwithfern.com"}'
# Test load docs endpoint (requires Fern token)
curl -X POST "https://jb5voktrj5.execute-api.us-east-1.amazonaws.com/preview-4124/load-docs-for-url" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $FERN_TOKEN" \
-d '{"url":"https://docs.buildwithfern.com"}'🏷️ Stack Name: ℹ️ Note: This preview will be automatically destroyed when the PR is closed or merged. |
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
| try { | ||
| return JSON.parse(raw); | ||
| } catch (e) { | ||
| console.error(`Failed to parse buffer: ${raw}`); |
There was a problem hiding this comment.
| console.error(`Failed to parse buffer: ${raw}`); | |
| console.error('Failed to parse buffer: invalid JSON format'); |
The error logging exposes potentially sensitive buffer contents when JSON parsing fails.
View Details
Analysis
Information leakage through error logging in readBuffer() exposes sensitive data
What fails: readBuffer() in servers/fdr-lambda/src/utils/serde.ts logs complete buffer contents via console.error(\Failed to parse buffer:
How to reproduce:
// Buffer with sensitive data that causes JSON parse error
const sensitiveBuffer = Buffer.from('{"apiKey": "sk-secret123", "dbUrl": "postgresql://user:pass@host/db", invalid}');
readBuffer(sensitiveBuffer); // Logs: Failed to parse buffer: {"apiKey": "sk-secret123", "dbUrl": "postgresql://user:pass@host/db", invalid}Result: Sensitive data (API keys, database credentials, PII) from malformed JSON is logged in plaintext to CloudWatch Logs, accessible to users with logs:PutLogEvents permissions
Expected: Error logging should not expose raw buffer contents per AWS CloudWatch Logs security guidance which recommends protecting credentials and PII in logs
| docsDbDefinition: DocsV1Db.DocsDefinitionDb.V3, | ||
| usesPublicS3: boolean | ||
| ): Promise<Record<DocsV1Read.FileId, DocsV1Read.File_>> { | ||
| const promisedFiles = Object.entries(docsDbDefinition.files).map( |
There was a problem hiding this comment.
| const promisedFiles = Object.entries(docsDbDefinition.files).map( | |
| const promisedFiles = Object.entries(docsDbDefinition.files ?? {}).map( |
The code calls Object.entries(docsDbDefinition.files) without null checking, which could cause a runtime error if files is null or undefined.
View Details
Analysis
Object.entries() call without null check in getFilesV2() causes TypeError
What fails: getFilesV2() function in servers/fdr-lambda/src/services/getDocsForUrl.ts calls Object.entries(docsDbDefinition.files) on line 163 without null checking
How to reproduce:
// If docsDbDefinition.files is null or undefined:
Object.entries(null); // TypeError: Cannot convert undefined or null to object
Object.entries(undefined); // TypeError: Cannot convert undefined or null to objectResult: Runtime crash with TypeError: Cannot convert undefined or null to object, causing entire request to fail
Expected: Should handle null/undefined gracefully like the defensive pattern used at line 224: docsDbDefinition.files ?? {}
Evidence: MDN Object.entries documentation confirms null/undefined throws TypeError. Same file already uses defensive pattern docsDbDefinition.files ?? {} at line 224, indicating this is a known concern.
coltondotio
had a problem deploying
to
Fern Dev
GitHub Actions
Failure
coltondotio
force-pushed
the
cberry/lambda-docs-loader
branch
from
8ace930 to
727851f
Compare
coltondotio
had a problem deploying
to
Fern Dev
GitHub Actions
Failure
coltondotio
force-pushed
the
cberry/lambda-docs-loader
branch
from
b264e8b to
1b51978
Compare
Short description of the changes made
Implements getDocsForUrl endpoint in fdr-lambda to fetch complete docs definitions from the database, following the same pattern as the FDR service but without Redis caching. Also refactors getMetadataForUrl
into a separate service file for better organization.
What was the motivation & context behind this PR?
The FDR Lambda is being enhanced to support serving full documentation definitions directly from the database. This enables:
The implementation closely mirrors the logic in servers/fdr/src/services/docs-cache/DocsDefinitionCache.ts:217-269, specifically the getDocsForUrlFromDatabase method, ensuring consistency across services.
Key features: