Skip to content

Commit 361f4a1

Browse files
PeriniMPeriniM
committed
feat: sgai mcp server
1 parent c402940 commit 361f4a1

File tree

10 files changed

+427
-2
lines changed

10 files changed

+427
-2
lines changed

‎.github/workflows/python-package.yml‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
name: Python Package
2+
3+
on:
4+
push:
5+
branches: [ main ]
6+
pull_request:
7+
branches: [ main ]
8+
9+
jobs:
10+
lint:
11+
runs-on: ubuntu-latest
12+
steps:
13+
- uses: actions/checkout@v3
14+
- name: Set up Python 3.12
15+
uses: actions/setup-python@v4
16+
with:
17+
python-version: "3.12"
18+
- name: Install dependencies
19+
run: |
20+
python -m pip install --upgrade pip
21+
python -m pip install ruff mypy
22+
pip install -e .
23+
- name: Lint with ruff
24+
run: |
25+
ruff check .
26+
- name: Type check with mypy
27+
run: |
28+
mypy src

‎.gitignore‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
# Build artifacts
2+
build/
3+
dist/
4+
wheels/
5+
*.egg-info/
6+
*.egg
7+
8+
# Python artifacts
9+
__pycache__/
10+
*.py[cod]
11+
*$py.class
12+
*.so
13+
.Python
14+
.pytest_cache/
15+
.coverage
16+
htmlcov/
17+
.tox/
18+
.nox/
19+
.hypothesis/
20+
.mypy_cache/
21+
.ruff_cache/
22+
23+
# Virtual environments
24+
.venv/
25+
venv/
26+
ENV/
27+
env/
28+
29+
# IDE files
30+
.idea/
31+
.vscode/
32+
*.swp
33+
*.swo
34+
.DS_Store
35+
36+
# Environment variables
37+
.env
38+
.env.local
39+
40+
# Node-generated files (if any)
41+
node_modules/
42+
43+
# Logs
44+
*.log

‎.python-version‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
3.10

‎Dockerfile‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Generated by https://smithery.ai. See: https://smithery.ai/docs/config#dockerfile
2+
FROM python:3.12-alpine
3+
4+
# Install build dependencies
5+
RUN apk add --no-cache gcc musl-dev libffi-dev
6+
7+
WORKDIR /app
8+
9+
# Copy project files
10+
COPY pyproject.toml .
11+
COPY README.md .
12+
COPY LICENSE .
13+
COPY src/ src/
14+
15+
# Upgrade pip and install the package
16+
RUN pip install --upgrade pip && \
17+
pip install .
18+
19+
# Expose MCP server via the entrypoint command
20+
ENTRYPOINT ["scrapegraph-mcp"]

‎README.md‎

Lines changed: 71 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,71 @@
1-
# scrapegraph-mcp
2-
ScapeGraph MCP Server
1+
# ScapeGraph MCP Server
2+
3+
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
4+
[![Python 3.10](https://img.shields.io/badge/python-3.10-blue.svg)](https://www.python.org/downloads/release/python-3100/)
5+
[![smithery badge](https://smithery.ai/badge/@scrapegraphai/scrapegraph-mcp)](https://smithery.ai/server/@scrapegraphai/scrapegraph-mcp)
6+
A [Model Context Protocol](https://modelcontextprotocol.io/introduction) (MCP) server that provides access to the [ScapeGraph AI](https://scrapegraphai.com) API. It allows language models to use AI-powered web scraping capabilities.
7+
8+
## Available Tools
9+
10+
The server exposes the following tools:
11+
12+
- `markdownify(website_url: str)`: Convert any webpage into clean, formatted markdown
13+
- `smartscraper(user_prompt: str, website_url: str)`: Extract structured data from any webpage using AI
14+
- `searchscraper(user_prompt: str)`: Perform AI-powered web searches with structured results
15+
16+
## Usage
17+
18+
You'll need a ScapeGraph API key to use this server. You can obtain one by:
19+
20+
1. Going to the [ScapeGraph Dashboard](https://dashboard.scrapegraphai.com)
21+
2. Creating an account and obtaining an API key
22+
23+
### Installing via Smithery
24+
25+
To install Oura API Integration Server for Claude Desktop automatically via [Smithery](https://smithery.ai/server/@scrapegraphai/scrapegraph-mcp):
26+
27+
```bash
28+
npx -y @smithery/cli install @scrapegraphai/scrapegraph-mcp --client claude
29+
```
30+
31+
### Claude for Desktop
32+
33+
Update your `claude_desktop_config.json` (located in `~/Library/Application\ Support/Claude/claude_desktop_config.json` on macOS and `%APPDATA%/Claude/claude_desktop_config.json` on Windows) to include the following:
34+
35+
```json
36+
{
37+
"mcpServers": {
38+
"scrapegraph": {
39+
"command": "uvx",
40+
"args": [
41+
"scrapegraph_mcp"
42+
],
43+
"env": {
44+
"SGAI_API_KEY": "YOUR_SCRAPEGRAPH_API_KEY"
45+
}
46+
}
47+
}
48+
}
49+
```
50+
51+
## Example Queries
52+
53+
Once connected, you can ask Claude questions like:
54+
55+
- "What are the main features of the ScapeGraph API?"
56+
- "Convert the ScapeGraph homepage into markdown"
57+
- "Extract the pricing information from the ScapeGraph website"
58+
- "Find information about the latest advancements in AI-powered web scraping"
59+
- "Summarize the content of the Python documentation website"
60+
61+
## Error Handling
62+
63+
The server provides human-readable error messages for common issues:
64+
65+
- API authentication errors
66+
- Invalid URL formats
67+
- Network connectivity problems
68+
69+
## License
70+
71+
This project is licensed under the MIT License - see the LICENSE file for details.

‎pyproject.toml‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
[project]
2+
name = "scrapegraph-mcp"
3+
version = "1.0.0"
4+
description = "MCP server for ScapeGraph API integration"
5+
license = {text = "MIT"}
6+
readme = "README.md"
7+
authors = [
8+
{ name = "Marco Perini", email = "[email protected]" }
9+
]
10+
requires-python = ">=3.10"
11+
dependencies = [
12+
"mcp[cli]>=1.3.0",
13+
"scrapegraph-py>=1.12.0",
14+
]
15+
classifiers = [
16+
"Development Status :: 4 - Beta",
17+
"Intended Audience :: Developers",
18+
"License :: OSI Approved :: MIT License",
19+
"Programming Language :: Python :: 3",
20+
"Programming Language :: Python :: 3.10",
21+
"Topic :: Software Development :: Libraries :: Python Modules",
22+
]
23+
24+
[project.optional-dependencies]
25+
dev = [
26+
"ruff>=0.1.0",
27+
"mypy>=1.0.0",
28+
]
29+
30+
[project.urls]
31+
"Homepage" = "https://github.com/ScrapeGraphAI/scrapegraph-mcp"
32+
"Bug Tracker" = "https://github.com/ScrapeGraphAI/scrapegraph-mcp/issues"
33+
34+
[project.scripts]
35+
scrapegraph-mcp = "scrapegraph_mcp.server:main"
36+
37+
[build-system]
38+
requires = ["hatchling"]
39+
build-backend = "hatchling.build"
40+
41+
[tool.hatch.build.targets.wheel]
42+
packages = ["src/scrapegraph_mcp"]
43+
44+
[tool.hatch.build]
45+
only-packages = true
46+
47+
[tool.ruff]
48+
line-length = 100
49+
target-version = "py312"
50+
select = ["E", "F", "I", "B", "W"]
51+
52+
[tool.mypy]
53+
python_version = "3.12"
54+
warn_return_any = true
55+
warn_unused_configs = true
56+
disallow_untyped_defs = true
57+
disallow_incomplete_defs = true

‎smithery.yaml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# Smithery configuration file: https://smithery.ai/docs/config#smitheryyaml
2+
3+
startCommand:
4+
type: stdio
5+
configSchema:
6+
# JSON Schema defining the configuration options for the MCP.
7+
type: object
8+
required:
9+
- scrapegraphApiKey
10+
properties:
11+
scrapegraphApiKey:
12+
type: string
13+
description: Your Scrapegraph API key
14+
commandFunction:
15+
# A JS function that produces the CLI command based on the given config to start the MCP on stdio.
16+
|-
17+
(config) => ({
18+
command: 'scrapegraph-mcp',
19+
args: [],
20+
env: {
21+
SGAI_API_KEY: config.scrapegraphApiKey
22+
}
23+
})
24+
exampleConfig:
25+
scrapegraphApiKey: sgai-1234567890

‎src/scrapegraph_mcp/__init__.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
"""ScapeGraph MCP Server."""
2+
3+
from .server import main
4+
5+
__all__ = ["main"]

‎src/scrapegraph_mcp/py.typed‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+

0 commit comments

Comments
 (0)