Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .eslintrc.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
},
"extends": [
"airbnb-base",
"prettier/@typescript-eslint",
"plugin:prettier/recommended"
],
"globals": {
Expand Down
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
.vscode
.idea
node_modules
node_modules
exhaustive-test.js
16 changes: 13 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,21 @@
# Changelog

## [1.8.3](https://github.com/ScrapingBee/scrapingbee-node/compare/v1.8.2...v1.8.3) (2026-06-30)
## [2.0.0](https://github.com/ScrapingBee/scrapingbee-node/compare/v1.8.2...v2.0.0) (2026-08-05)

### Features

- Added Auto-Mode support for the HTML API: pass `mode: 'auto'` (GET only) to let ScrapingBee pick the cheapest scraping config that succeeds, charged only for the winning config. Optional `max_cost` (integer ≥ 1) caps the credits a request may cost. The credits charged are returned in the `Spb-auto-cost` response header.
- Added `mode` and `max_cost` to the `HtmlApiParams` type for autocomplete.
- Added `fastSearch()` method for Fast Search API
- Added `amazonPricing()` method for Amazon Pricing API
- Added `gemini()` method for Gemini API
- Added `youtubeSubtitles()` method for YouTube Subtitles API (replaces the removed `youtubeTranscript()`)
- Expanded param type hints for all endpoints to match the documented API contract (e.g. HTML API `mode`/`max_cost`, Google `pages`/`date_range`/geo/`sort_by`/price filters, Amazon Search `autoselect_variant`, Walmart Search `start_page`)
- Added `tag` optional param (request label) to every endpoint's params type except Usage, including a new `YouTubeMetadataParams` type so `youtubeMetadata()` now accepts a `params` object

### Removed

- Removed deprecated `get()` and `post()` methods (use `htmlApi()` with `method: 'GET'`/`'POST'` instead)
- Removed `youtubeTranscript()` method (endpoint no longer supported; use `youtubeSubtitles()` instead)
- Removed `youtubeTrainability()` method (no longer supported)

## [1.8.2](https://github.com/ScrapingBee/scrapingbee-node/compare/v1.8.0...v1.8.2) (2026-01-22)

Expand Down
117 changes: 82 additions & 35 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,12 @@ Signup to ScrapingBee to [get your API key](https://app.scrapingbee.com/account/

- [HTML API](#html-api)
- [Google Search API](#google-search-api)
- [Fast Search API](#fast-search-api)
- [Amazon API](#amazon-api)
- [Walmart API](#walmart-api)
- [YouTube API](#youtube-api)
- [ChatGPT API](#chatgpt-api)
- [Gemini API](#gemini-api)
- [Usage API](#usage-api)

---
Expand Down Expand Up @@ -110,7 +112,7 @@ async function post(url) {
const response = await client.htmlApi({
url: url,
method: 'POST',
'username=user&password=pass',
data: 'username=user&password=pass',
params: {
render_js: false,
},
Expand All @@ -127,7 +129,7 @@ async function post(url) {
console.log(text);
}

post('https://httpbin.org/post');
post('https://httpbin.scrapingbee.com/post');
```

### Screenshot
Expand Down Expand Up @@ -186,6 +188,33 @@ googleSearch('web scraping tools');

---

## Fast Search API

Retrieve a lightweight Google result set optimized for sub-second responses.

```javascript
const { ScrapingBeeClient } = require('scrapingbee');

async function fastSearch(query) {
const client = new ScrapingBeeClient('YOUR-API-KEY');
const response = await client.fastSearch({
search: query,
params: {
page: 1,
country_code: 'us',
language: 'en',
tag: 'my-request',
}
});

console.log(response.data);
}

fastSearch('web scraping tools');
```

---

## Amazon API

Scrape Amazon search results and product details.
Expand Down Expand Up @@ -245,6 +274,32 @@ async function amazonProduct(asin) {
amazonProduct('B0D2Q9397Y');
```

### Amazon Pricing

```javascript
const { ScrapingBeeClient } = require('scrapingbee');

async function amazonPricing(asin) {
const client = new ScrapingBeeClient('YOUR-API-KEY');
const response = await client.amazonPricing({
asin: asin,
params: {
domain: 'com',
language: 'en',
zip_code: '10001',
currency: 'USD',
device: 'desktop',
light_request: true,
add_html: false,
}
});

console.log(response.data);
}

amazonPricing('B0D2Q9397Y');
```

---

## Walmart API
Expand Down Expand Up @@ -307,7 +362,7 @@ walmartProduct('123456789');

## YouTube API

Scrape YouTube search results, video metadata, transcripts, and trainability data.
Scrape YouTube search results, video metadata, and subtitles.

### YouTube Search

Expand Down Expand Up @@ -353,59 +408,67 @@ async function youtubeMetadata(videoId) {
youtubeMetadata('dQw4w9WgXcQ');
```

### YouTube Transcript
### YouTube Subtitles

```javascript
const { ScrapingBeeClient } = require('scrapingbee');

async function youtubeTranscript(videoId) {
async function youtubeSubtitles(videoId) {
const client = new ScrapingBeeClient('YOUR-API-KEY');
const response = await client.youtubeTranscript({
const response = await client.youtubeSubtitles({
video_id: videoId,
params: {
language: 'en',
transcript_origin: 'auto_generated',
subtitle_origin: 'auto_generated',
}
});

console.log(response.data);
}

youtubeTranscript('dQw4w9WgXcQ');
youtubeSubtitles('dQw4w9WgXcQ');
```

### YouTube Trainability
---

## ChatGPT API

Use ChatGPT with optional web search capabilities.

```javascript
const { ScrapingBeeClient } = require('scrapingbee');

async function youtubeTrainability(videoId) {
async function askChatGPT(prompt) {
const client = new ScrapingBeeClient('YOUR-API-KEY');
const response = await client.youtubeTrainability({
video_id: videoId,
const response = await client.chatGPT({
prompt: prompt,
params: {
search: true,
country_code: 'us',
add_html: false,
}
});

console.log(response.data);
}

youtubeTrainability('dQw4w9WgXcQ');
askChatGPT('What are the latest web scraping trends?');
```

---

## ChatGPT API
## Gemini API

Use ChatGPT with optional web search capabilities.
Ask Gemini through ScrapingBee and receive citation objects when available.

```javascript
const { ScrapingBeeClient } = require('scrapingbee');

async function askChatGPT(prompt) {
async function askGemini(prompt) {
const client = new ScrapingBeeClient('YOUR-API-KEY');
const response = await client.chatGPT({
const response = await client.gemini({
prompt: prompt,
params: {
search: true,
country_code: 'us',
add_html: false,
}
Expand All @@ -414,7 +477,7 @@ async function askChatGPT(prompt) {
console.log(response.data);
}

askChatGPT('What are the latest web scraping trends?');
askGemini('What are the latest web scraping trends?');
```

---
Expand Down Expand Up @@ -475,22 +538,6 @@ client.googleSearch({ search: 'test' })

---

## Legacy Methods (Deprecated)

The `get()` and `post()` methods are deprecated and will be removed in a future version. Please use `htmlApi()` instead.

```javascript
// Deprecated
await client.get({ url: '...' });
await client.post({ url: '...' });

// Use instead
await client.htmlApi({ url: '...', method: 'GET' });
await client.htmlApi({ url: '...', method: 'POST' });
```

---

## Documentation

For more details on all available parameters, visit [ScrapingBee's documentation](https://www.scrapingbee.com/documentation/).
Loading