Skip to content

Commit 41e1f0b

Browse files
Chasen-ZhangChasen-Zhang
authored
Merge branch 'main' into add-copy-pages
2 parents 319cf3b + 6cbb07a commit 41e1f0b

File tree

3 files changed

+261
-20
lines changed

3 files changed

+261
-20
lines changed

‎docs/en/guides/51-ai-functions/01-external-functions.md‎

Lines changed: 20 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,26 +1,28 @@
11
---
2-
title: External Functions for Custom AI/ML
2+
title: Custom AI/ML with External Functions
33
---
44

5-
# External Functions for Custom AI/ML
5+
# Custom AI/ML with External Functions
66

7-
For advanced AI/ML scenarios, Databend supports external functions that connect your data with custom AI/ML infrastructure written in languages like Python.
7+
Build powerful AI/ML capabilities by connecting Databend with your own infrastructure. External functions let you deploy custom models, leverage GPU acceleration, and integrate with any ML framework while keeping your data secure.
88

9-
| Feature | Description | Benefits |
10-
|---------|-------------|----------|
11-
| **Model Flexibility** | Use open-source models or your internal AI/ML infrastructure | • Freedom to choose any model<br/>• Leverage existing ML investments<br/>• Stay up-to-date with latest AI advancements |
12-
| **GPU Acceleration** | Deploy external function servers on GPU-equipped machines | • Faster inference for deep learning models<br/>• Handle larger batch sizes<br/>• Support compute-intensive workloads |
13-
| **Custom ML Models** | Deploy and use your own machine learning models | • Proprietary algorithms<br/>• Domain-specific models<br/>• Fine-tuned for your data |
14-
| **Advanced AI Pipelines** | Build complex AI workflows with specialized libraries | • Multi-step processing<br/>• Custom transformations<br/>• Integration with ML frameworks |
15-
| **Scalability** | Handle resource-intensive AI operations outside Databend | • Independent scaling<br/>• Optimized resource allocation<br/>• High-throughput processing |
9+
## Key Capabilities
1610

17-
## Implementation Overview
11+
| Feature | Benefits |
12+
|---------|----------|
13+
| **Custom Models** | Use any open-source or proprietary AI/ML models |
14+
| **GPU Acceleration** | Deploy on GPU-equipped machines for faster inference |
15+
| **Data Privacy** | Keep your data within your infrastructure |
16+
| **Scalability** | Independent scaling and resource optimization |
17+
| **Flexibility** | Support for any programming language and ML framework |
1818

19-
1. Create an external server with your AI/ML code (Python with [databend-udf](https://pypi.org/project/databend-udf))
20-
2. Register the server with Databend using `CREATE FUNCTION`
21-
3. Call your AI/ML functions directly in SQL queries
19+
## How It Works
2220

23-
## Example: Custom AI Model Integration
21+
1. **Create AI Server**: Build your AI/ML server using Python and [databend-udf](https://pypi.org/project/databend-udf)
22+
2. **Register Function**: Connect your server to Databend with `CREATE FUNCTION`
23+
3. **Use in SQL**: Call your custom AI functions directly in SQL queries
24+
25+
## Example: Text Embedding Function
2426

2527
```python
2628
# Simple embedding UDF server demo
@@ -74,7 +76,7 @@ ORDER BY similarity ASC
7476
LIMIT 5;
7577
```
7678

77-
## Next Steps
79+
## Learn More
7880

79-
1. **[External Functions Guide](/guides/query/external-function)** - Detailed setup instructions
80-
2. **[Databend Cloud](https://databend.com)** - Try AI functions with a free trial
81+
- **[External Functions Guide](/guides/query/external-function)** - Complete setup and deployment instructions
82+
- **[Databend Cloud](https://databend.com)** - Try external functions with a free account

‎docs/en/guides/51-ai-functions/02-mcp.md‎

Lines changed: 227 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,227 @@
1+
# MCP Server for Databend
2+
3+
[mcp-databend](https://github.com/databendlabs/mcp-databend) is an MCP (Model Context Protocol) server that enables AI assistants to interact directly with your Databend database using natural language.
4+
5+
## What mcp-databend Can Do
6+
7+
- **execute_sql** - Execute SQL queries with timeout protection
8+
- **show_databases** - List all available databases
9+
- **show_tables** - List tables in a database (with optional filter)
10+
- **describe_table** - Get detailed table schema information
11+
12+
## Build a ChatBI Tool
13+
14+
This tutorial shows you how to build a conversational Business Intelligence tool using mcp-databend and the Agno framework. You'll create a local agent that can answer data questions in natural language.
15+
16+
## Step-by-Step Tutorial
17+
18+
### Step 1: Setup Databend Connection
19+
20+
First, you need a Databend database to connect to:
21+
22+
1. **Sign up for [Databend Cloud](https://app.databend.com)** (free tier available)
23+
2. **Create a warehouse and database**
24+
3. **Get your connection string** from the console
25+
26+
For detailed DSN format and examples, see [Connection String Documentation](https://docs.databend.com/developer/drivers/#connection-string-dsn).
27+
28+
| Deployment | Connection String Example |
29+
|------------|---------------------------|
30+
| **Databend Cloud** | `databend://user:pwd@host:443/database?warehouse=wh` |
31+
| **Self-hosted** | `databend://user:pwd@localhost:8000/database?sslmode=disable` |
32+
33+
### Step 2: Install Dependencies
34+
35+
Create a virtual environment and install the required packages:
36+
37+
```bash
38+
# Create virtual environment
39+
python3 -m venv .venv
40+
source .venv/bin/activate
41+
42+
# Install packages
43+
pip install packaging openai agno openrouter sqlalchemy fastapi mcp-databend
44+
```
45+
46+
### Step 3: Create ChatBI Agent
47+
48+
Now create your ChatBI agent that uses mcp-databend to interact with your database.
49+
50+
Create a file `agent.py`:
51+
52+
```python
53+
from contextlib import asynccontextmanager
54+
import os
55+
import logging
56+
import sys
57+
58+
from agno.agent import Agent
59+
from agno.playground import Playground
60+
from agno.storage.sqlite import SqliteStorage
61+
from agno.tools.mcp import MCPTools
62+
from agno.models.openrouter import OpenRouter
63+
from fastapi import FastAPI
64+
65+
logging.basicConfig(level=logging.INFO)
66+
logger = logging.getLogger(__name__)
67+
68+
def check_env_vars():
69+
"""Check required environment variables"""
70+
required = {
71+
"DATABEND_DSN": "https://docs.databend.com/developer/drivers/#connection-string-dsn",
72+
"OPENROUTER_API_KEY": "https://openrouter.ai/settings/keys"
73+
}
74+
75+
missing = [var for var in required if not os.getenv(var)]
76+
77+
if missing:
78+
print("❌ Missing environment variables:")
79+
for var in missing:
80+
print(f" • {var}: {required[var]}")
81+
print("\nExample: export DATABEND_DSN='...' OPENROUTER_API_KEY='...'")
82+
sys.exit(1)
83+
84+
print("✅ Environment variables OK")
85+
86+
check_env_vars()
87+
88+
class DatabendTool:
89+
def __init__(self):
90+
self.mcp = None
91+
self.dsn = os.getenv("DATABEND_DSN")
92+
93+
def create(self):
94+
env = os.environ.copy()
95+
env["DATABEND_DSN"] = self.dsn
96+
self.mcp = MCPTools(
97+
command="python -m mcp_databend",
98+
env=env,
99+
timeout_seconds=300
100+
)
101+
return self.mcp
102+
103+
async def init(self):
104+
try:
105+
await self.mcp.connect()
106+
logger.info("✓ Connected to Databend")
107+
return True
108+
except Exception as e:
109+
logger.error(f"✗ Databend connection failed: {e}")
110+
return False
111+
112+
databend = DatabendTool()
113+
114+
agent = Agent(
115+
name="ChatBI",
116+
model=OpenRouter(
117+
id=os.getenv("MODEL_ID", "anthropic/claude-sonnet-4"),
118+
api_key=os.getenv("OPENROUTER_API_KEY")
119+
),
120+
tools=[],
121+
instructions=[
122+
"You are ChatBI - a Business Intelligence assistant for Databend.",
123+
"Help users explore and analyze their data using natural language.",
124+
"Always start by exploring available databases and tables.",
125+
"Format query results in clear, readable tables.",
126+
"Provide insights and explanations with your analysis."
127+
],
128+
storage=SqliteStorage(table_name="chatbi", db_file="chatbi.db"),
129+
add_datetime_to_instructions=True,
130+
add_history_to_messages=True,
131+
num_history_responses=5,
132+
markdown=True,
133+
show_tool_calls=True,
134+
)
135+
136+
@asynccontextmanager
137+
async def lifespan(app: FastAPI):
138+
tool = databend.create()
139+
if not await databend.init():
140+
logger.error("Failed to initialize Databend")
141+
raise RuntimeError("Databend connection failed")
142+
143+
agent.tools.append(tool)
144+
logger.info("ChatBI initialized successfully")
145+
146+
yield
147+
148+
if databend.mcp:
149+
await databend.mcp.close()
150+
151+
playground = Playground(
152+
agents=[agent],
153+
name="ChatBI with Databend",
154+
description="Business Intelligence Assistant powered by Databend"
155+
)
156+
157+
app = playground.get_app(lifespan=lifespan)
158+
159+
if __name__ == "__main__":
160+
print("🤖 Starting MCP Server for Databend")
161+
print("Open http://localhost:7777 to start chatting!")
162+
playground.serve(app="agent:app", host="127.0.0.1", port=7777)
163+
```
164+
165+
### Step 4: Configure Environment
166+
167+
Set up your API keys and database connection:
168+
169+
```bash
170+
# Set your OpenRouter API key
171+
export OPENROUTER_API_KEY="your-openrouter-key"
172+
173+
# Set your Databend connection string
174+
export DATABEND_DSN="your-databend-connection-string"
175+
```
176+
177+
### Step 5: Start Your ChatBI Agent
178+
179+
Run your agent to start the local server:
180+
181+
```bash
182+
python agent.py
183+
```
184+
185+
You should see:
186+
```
187+
✅ Environment variables OK
188+
🤖 Starting MCP Server for Databend
189+
Open http://localhost:7777 to start chatting!
190+
INFO Starting playground on http://127.0.0.1:7777
191+
INFO: Started server process [189851]
192+
INFO: Waiting for application startup.
193+
INFO:agent:✓ Connected to Databend
194+
INFO:agent:ChatBI initialized successfully
195+
INFO: Application startup complete.
196+
INFO: Uvicorn running on http://127.0.0.1:7777 (Press CTRL+C to quit)
197+
```
198+
199+
### Step 6: Setup Web Interface
200+
201+
For a better user experience, you can set up Agno's web interface:
202+
203+
```bash
204+
# Create the Agent UI
205+
npx create-agent-ui@latest
206+
207+
# Enter 'y' when prompted, then run:
208+
cd agent-ui && npm run dev
209+
```
210+
211+
**Connect to Your Agent:**
212+
1. Open [http://localhost:3000](http://localhost:3000)
213+
2. Select "localhost:7777" as your endpoint
214+
3. Start asking questions about your data!
215+
216+
**Try These Queries:**
217+
- "Show me all databases"
218+
- "What tables do I have?"
219+
- "Describe the structure of my tables"
220+
- "Run a query to show sample data"
221+
222+
## Resources
223+
224+
- **GitHub Repository**: [databendlabs/mcp-databend](https://github.com/databendlabs/mcp-databend)
225+
- **PyPI Package**: [mcp-databend](https://pypi.org/project/mcp-databend)
226+
- **Agno Framework**: [Agno MCP](https://docs.agno.com/tools/mcp/mcp)
227+
- **Agent UI**: [Agent UI](https://docs.agno.com/agent-ui/introduction)

‎docs/en/guides/51-ai-functions/index.md‎

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
# AI Functions in Databend
1+
# AI & ML Integration
22

3-
Databend provides AI and ML capabilities through external functions, allowing you to integrate custom AI models and leverage advanced ML infrastructure while maintaining data privacy and control.
3+
Databend enables powerful AI and ML capabilities through two complementary approaches: build custom AI functions with your own infrastructure, or create conversational data experiences using natural language.
44

55
## External Functions - The Recommended Approach
66

@@ -14,6 +14,18 @@ External functions enable you to connect your data with custom AI/ML infrastruct
1414
| **Scalability** | Independent scaling and resource optimization |
1515
| **Flexibility** | Support for any programming language and ML framework |
1616

17+
## MCP Server - Natural Language Data Interaction
18+
19+
The Model Context Protocol (MCP) server enables AI assistants to interact with your Databend database using natural language, perfect for building conversational BI tools.
20+
21+
| Feature | Benefits |
22+
|---------|----------|
23+
| **Natural Language** | Query your data using plain English |
24+
| **AI Assistant Integration** | Works with Claude, ChatGPT, and custom agents |
25+
| **Real-time Analysis** | Get instant insights from your data |
26+
1727
## Getting Started
1828

1929
**[External Functions Guide](01-external-functions.md)** - Learn how to create and deploy custom AI functions with practical examples and implementation guidance
30+
31+
**[MCP Server Guide](02-mcp.md)** - Build a conversational BI tool using mcp-databend and natural language queries

0 commit comments

Comments
 (0)