|
4 | 4 | [](https://www.python.org/downloads/release/python-3100/) |
5 | 5 | [](https://smithery.ai/server/@ScrapeGraphAI/scrapegraph-mcp) |
6 | 6 |
|
7 | | -A [Model Context Protocol](https://modelcontextprotocol.io/introduction) (MCP) server that provides access to the [ScapeGraph AI](https://scrapegraphai.com) API. It allows language models to use AI-powered web scraping capabilities. |
| 7 | +A production-ready [Model Context Protocol](https://modelcontextprotocol.io/introduction) (MCP) server that provides seamless integration with the [ScapeGraph AI](https://scrapegraphai.com) API. This server enables language models to leverage advanced AI-powered web scraping capabilities with enterprise-grade reliability. |
8 | 8 |
|
9 | 9 | ## Available Tools |
10 | 10 |
|
11 | | -The server exposes the following tools: |
| 11 | +The server provides the following enterprise-ready tools: |
12 | 12 |
|
13 | | -- `markdownify(website_url: str)`: Convert any webpage into clean, formatted markdown |
14 | | -- `smartscraper(user_prompt: str, website_url: str)`: Extract structured data from any webpage using AI |
15 | | -- `searchscraper(user_prompt: str)`: Perform AI-powered web searches with structured results |
| 13 | +- `markdownify(website_url: str)`: Transform any webpage into clean, structured markdown format |
| 14 | +- `smartscraper(user_prompt: str, website_url: str)`: Leverage AI to extract structured data from any webpage |
| 15 | +- `searchscraper(user_prompt: str)`: Execute AI-powered web searches with structured, actionable results |
16 | 16 |
|
17 | | -## Usage |
| 17 | +## Setup Instructions |
18 | 18 |
|
19 | | -You'll need a ScapeGraph API key to use this server. You can obtain one by: |
| 19 | +To utilize this server, you'll need a ScapeGraph API key. Follow these steps to obtain one: |
20 | 20 |
|
21 | | -1. Going to the [ScapeGraph Dashboard](https://dashboard.scrapegraphai.com) |
22 | | -2. Creating an account and obtaining an API key |
| 21 | +1. Navigate to the [ScapeGraph Dashboard](https://dashboard.scrapegraphai.com) |
| 22 | +2. Create an account and generate your API key |
23 | 23 |
|
24 | | -### Installing via Smithery |
| 24 | +### Automated Installation via Smithery |
25 | 25 |
|
26 | | -To install ScrapeGraph API Integration Server for Claude Desktop automatically via [Smithery](https://smithery.ai/server/@ScrapeGraphAI/scrapegraph-mcp): |
| 26 | +For automated installation of the ScrapeGraph API Integration Server using [Smithery](https://smithery.ai/server/@ScrapeGraphAI/scrapegraph-mcp): |
27 | 27 |
|
28 | 28 | ```bash |
29 | 29 | npx -y @smithery/cli install @ScrapeGraphAI/scrapegraph-mcp --client claude |
30 | 30 | ``` |
31 | 31 |
|
32 | | -### Claude for Desktop |
| 32 | +### Claude Desktop Configuration |
33 | 33 |
|
34 | | -Update your `claude_desktop_config.json` (located in `~/Library/Application\ Support/Claude/claude_desktop_config.json` on macOS and `%APPDATA%/Claude/claude_desktop_config.json` on Windows) to include the following: |
| 34 | +Update your Claude Desktop configuration file with the following settings: |
35 | 35 |
|
36 | 36 | ```json |
37 | 37 | { |
38 | 38 | "mcpServers": { |
39 | | - "scrapegraph": { |
40 | | - "command": "uvx", |
| 39 | + "@ScrapeGraphAI-scrapegraph-mcp": { |
| 40 | + "command": "npx", |
41 | 41 | "args": [ |
42 | | - "scrapegraph_mcp" |
43 | | - ], |
44 | | - "env": { |
45 | | - "SGAI_API_KEY": "YOUR_SCRAPEGRAPH_API_KEY" |
46 | | - } |
| 42 | + "-y", |
| 43 | + "@smithery/cli@latest", |
| 44 | + "run", |
| 45 | + "@ScrapeGraphAI/scrapegraph-mcp", |
| 46 | + "--config", |
| 47 | + "\"{\\\"scrapegraphApiKey\\\":\\\"sgai-123456\\\"}\"" |
| 48 | + ] |
47 | 49 | } |
48 | 50 | } |
49 | 51 | } |
50 | 52 | ``` |
51 | 53 |
|
52 | | -## Example Queries |
| 54 | +The configuration file is located at: |
| 55 | +- Windows: `%APPDATA%/Claude/claude_desktop_config.json` |
| 56 | +- macOS: `~/Library/Application\ Support/Claude/claude_desktop_config.json` |
53 | 57 |
|
54 | | -Once connected, you can ask Claude questions like: |
| 58 | +## Example Use Cases |
55 | 59 |
|
56 | | -- "What are the main features of the ScapeGraph API?" |
57 | | -- "Convert the ScapeGraph homepage into markdown" |
58 | | -- "Extract the pricing information from the ScapeGraph website" |
59 | | -- "Find information about the latest advancements in AI-powered web scraping" |
60 | | -- "Summarize the content of the Python documentation website" |
| 60 | +The server enables sophisticated queries such as: |
| 61 | + |
| 62 | +- "Analyze and extract the main features of the ScapeGraph API" |
| 63 | +- "Generate a structured markdown version of the ScapeGraph homepage" |
| 64 | +- "Extract and analyze pricing information from the ScapeGraph website" |
| 65 | +- "Research and summarize recent developments in AI-powered web scraping" |
| 66 | +- "Create a comprehensive summary of the Python documentation website" |
61 | 67 |
|
62 | 68 | ## Error Handling |
63 | 69 |
|
64 | | -The server provides human-readable error messages for common issues: |
| 70 | +The server implements robust error handling with detailed, actionable error messages for: |
| 71 | + |
| 72 | +- API authentication issues |
| 73 | +- Malformed URL structures |
| 74 | +- Network connectivity failures |
| 75 | +- Rate limiting and quota management |
| 76 | + |
| 77 | +## Common Issues |
65 | 78 |
|
66 | | -- API authentication errors |
67 | | -- Invalid URL formats |
68 | | -- Network connectivity problems |
| 79 | +### Windows-Specific Connection |
| 80 | + |
| 81 | +When running on Windows systems, you may need to use the following command to connect to the MCP server: |
| 82 | + |
| 83 | +```bash |
| 84 | +C:\Windows\System32\cmd.exe /c npx -y @smithery/cli@latest run @ScrapeGraphAI/scrapegraph-mcp --config "{\"scrapegraphApiKey\":\"sgai-123456\"}" |
| 85 | +``` |
| 86 | + |
| 87 | +This ensures proper execution in the Windows environment. |
69 | 88 |
|
70 | 89 | ## License |
71 | 90 |
|
72 | | -This project is licensed under the MIT License - see the LICENSE file for details. |
| 91 | +This project is distributed under the MIT License. For detailed terms and conditions, please refer to the LICENSE file. |
| 92 | + |
| 93 | +Made with ❤️ by ScrapeGraphAI Team |
0 commit comments