Skip to content

Commit 31c12fd

Browse files
authored
Enhance file upload functionality and system tool integration (#5257)
* Enhance file upload functionality and system tool integration * Add supplementary documents and optimize the upload interface * Refactor file plugin types and update upload configurations * Refactor MinIO configuration variables and clean up API plugin handlers for improved readability and consistency * File name change * Refactor SystemTools component layout * fix i18n * fix * fix * fix
1 parent e0c21a9 commit 31c12fd

File tree

35 files changed

+867
-69
lines changed

35 files changed

+867
-69
lines changed

docSite/assets/imgs/plugins/entry.png

513 KB
Loading

docSite/assets/imgs/plugins/file.png

49.3 KB
Loading
Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,14 @@
11
{
22
"title": "系统插件",
33
"description": "介绍如何使用和提交系统插件,以及各插件的填写说明",
4-
"pages": ["dev_system_tool","how_to_submit_system_plugin","searxng_plugin_guide","google_search_plugin_guide","bing_search_plugin","doc2x_plugin_guide"]
5-
}
4+
"pages": [
5+
"dev_system_tool",
6+
"how_to_submit_system_plugin",
7+
"upload_system_tool",
8+
"searxng_plugin_guide",
9+
"google_search_plugin_guide",
10+
"bing_search_plugin",
11+
"doc2x_plugin_guide",
12+
"deepseek_plugin_guide"
13+
]
14+
}
Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
---
2+
title: 如何在线上传系统工具
3+
description: FastGPT 系统工具在线上传指南
4+
---
5+
6+
> 从 FastGPT 4.10.0 版本开始,系统管理员可以通过 Web 界面直接上传和更新系统工具,无需重新部署服务
7+
8+
## 权限要求
9+
10+
⚠️ **重要提示**:只有 **root 用户** 才能使用在线上传系统工具功能。
11+
12+
- 确保您已使用 `root` 账户登录 FastGPT
13+
- 普通用户无法看到"导入/更新"按钮和删除功能
14+
15+
## 支持的文件格式
16+
17+
- **文件类型**`.js` 文件
18+
- **文件大小**:最大 10MB
19+
- **文件数量**:每次只能上传一个文件
20+
21+
## 上传步骤
22+
23+
### 1. 进入系统工具页面
24+
25+
1. 登录 FastGPT 管理后台
26+
2. 导航到:**工作台****系统工具**
27+
3. 确认页面右上角显示"导入/更新"按钮(只有 root 用户可见)
28+
29+
![](/imgs/plugins/entry.png)
30+
31+
### 2. 准备工具文件
32+
33+
在上传之前,请确保您的 `.js` 文件是从 fastgpt-plugin 项目中通过 `bun run build` 命令打包后的 dist/tools/built-in 文件夹下得到的
34+
35+
![](/imgs/plugins/file.png)
36+
37+
### 3. 执行上传
38+
39+
1. 点击 **"导入/更新"** 按钮
40+
2. 在弹出的对话框中,点击文件选择区域
41+
3. 选择您准备好的 `.js` 工具文件
42+
4. 确认文件信息无误后,点击 **"确认导入"**
43+
44+
### 4. 上传过程
45+
46+
- 上传成功后会显示成功提示
47+
- 页面自动刷新,新工具会出现在工具列表中
48+
49+
## 功能特点
50+
51+
### 工具管理
52+
53+
- **查看工具**:所有用户都可以查看已安装的系统工具
54+
- **上传工具**:仅 root 用户可以上传新工具或更新现有工具
55+
- **删除工具**:仅 root 用户可以删除已上传的工具
56+
57+
### 工具类型识别
58+
59+
系统会根据工具的配置自动识别工具类型:
60+
61+
- 🔧 **工具 (tools)**
62+
- 🔍 **搜索 (search)**
63+
- 🎨 **多模态 (multimodal)**
64+
- 💬 **通讯 (communication)**
65+
- 📦 **其他 (other)**
66+
67+
## 常见问题
68+
69+
### Q: 上传失败,提示"文件内容存在错误"
70+
71+
**可能原因:**
72+
- fastgpt-plugin 项目不是最新的,导致打包的 `.js` 文件缺少正确的内容
73+
- 工具配置格式不正确
74+
75+
**解决方案:**
76+
1. 拉取最新的 fastgpt-plugin 项目重新进行 `bun run build` 获得打包后的 `.js` 文件
77+
2. 检查本地插件运行是否成功
78+
79+
### Q: 无法看到"导入/更新"按钮
80+
81+
**原因:** 当前用户不是 root 用户
82+
83+
**解决方案:** 使用 root 账户重新登录
84+
85+
### Q: 文件上传超时
86+
87+
**可能原因:**
88+
- 文件过大(超过 10MB)
89+
- 网络连接不稳定
90+
91+
**解决方案:**
92+
1. 确认文件大小在限制范围内
93+
2. 检查网络连接
94+
3. 尝试重新上传
95+
96+
## 最佳实践
97+
98+
### 上传前检查
99+
100+
1. **代码测试**:在本地环境测试工具功能
101+
2. **格式验证**:确保符合 FastGPT 工具规范
102+
3. **文件大小**:保持文件在合理大小范围内
103+
104+
### 版本管理
105+
106+
- 建议为工具添加版本号注释
107+
- 更新工具时,先备份原有版本
108+
- 记录更新日志和功能变更
109+
110+
### 安全考虑
111+
112+
- 仅上传来源可信的工具文件
113+
- 避免包含敏感信息或凭据
114+
- 定期审查已安装的工具
115+
116+
### 存储方式
117+
118+
- 工具文件存储在 MinIO 中
119+
- 工具元数据保存在 MongoDB 中
120+
121+
---
122+
123+
通过在线上传功能,您可以快速部署和管理系统工具,提高 FastGPT 的扩展性和灵活性。如遇到问题,请参考上述常见问题或联系技术支持。

packages/global/core/app/plugin/type.d.ts

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,9 @@ export type SystemPluginTemplateItemType = WorkflowTemplateType & {
5555
// Admin config
5656
inputList?: FlowNodeInputItemType['inputList'];
5757
hasSystemSecret?: boolean;
58+
59+
// Plugin source type
60+
toolSource?: 'uploaded' | 'built-in';
5861
};
5962

6063
export type SystemPluginTemplateListItemType = Omit<

packages/global/core/workflow/type/node.d.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -124,6 +124,7 @@ export type NodeTemplateListItemType = {
124124
instructions?: string; // 使用说明
125125
courseUrl?: string; // 教程链接
126126
sourceMember?: SourceMember;
127+
toolSource?: 'uploaded' | 'built-in'; // Plugin source type
127128
};
128129

129130
export type NodeTemplateListType = {
Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
import { mimeTypes } from './utils';
2+
3+
export enum FilePluginTypeEnum {
4+
tool = 'tool'
5+
}
6+
7+
export enum PluginPathEnum {
8+
tools = 'plugin/tools'
9+
}
10+
11+
export const PLUGIN_TYPE_TO_PATH_MAP: Record<FilePluginTypeEnum, PluginPathEnum> = {
12+
[FilePluginTypeEnum.tool]: PluginPathEnum.tools
13+
};
14+
15+
export type UploadFileConfig = {
16+
maxFileSize: number;
17+
allowedExtensions?: string[];
18+
bucket: string;
19+
};
20+
21+
export const defaultUploadConfig: UploadFileConfig = {
22+
maxFileSize: process.env.UPLOAD_MAX_FILE_SIZE
23+
? parseInt(process.env.UPLOAD_MAX_FILE_SIZE)
24+
: 10 * 1024 * 1024,
25+
bucket: process.env.MINIO_UPLOAD_BUCKET || 'fastgpt-uploads',
26+
allowedExtensions: Object.keys(mimeTypes)
27+
};
28+
29+
export type FileMetadata = {
30+
fileId: string;
31+
originalFilename: string;
32+
contentType: string;
33+
size: number;
34+
uploadTime: Date;
35+
accessUrl: string;
36+
};
37+
38+
export type PresignedUrlInput = {
39+
filename: string;
40+
pluginType?: FilePluginTypeEnum;
41+
contentType?: string;
42+
metadata?: Record<string, string>;
43+
maxSize?: number;
44+
};
45+
46+
export type PresignedUrlResponse = {
47+
fileId: string;
48+
objectName: string;
49+
uploadUrl: string;
50+
formData: Record<string, string>;
51+
};
52+
53+
export type FileUploadInput = {
54+
buffer: Buffer;
55+
filename: string;
56+
};
Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
import { randomBytes } from 'crypto';
2+
import {
3+
defaultUploadConfig,
4+
type UploadFileConfig,
5+
type PresignedUrlInput,
6+
type PresignedUrlResponse,
7+
FilePluginTypeEnum,
8+
PLUGIN_TYPE_TO_PATH_MAP
9+
} from './config';
10+
import { addLog } from '../../system/log';
11+
import { ensureBucket } from '../../minio/init';
12+
import { connectionMinio } from '../../minio/index';
13+
import { inferContentType } from './utils';
14+
15+
let globalConfig: UploadFileConfig = defaultUploadConfig;
16+
17+
export const initFileUploadService = async (config?: Partial<UploadFileConfig>) => {
18+
globalConfig = { ...defaultUploadConfig, ...config };
19+
20+
try {
21+
addLog.info(`Initializing upload bucket: ${globalConfig.bucket}`);
22+
await ensureBucket(globalConfig.bucket, true);
23+
addLog.info(`Upload bucket initialized successfully: ${globalConfig.bucket}`);
24+
return true;
25+
} catch (error) {
26+
addLog.error(`Failed to initialize upload bucket: ${globalConfig.bucket}`, error);
27+
throw error;
28+
}
29+
};
30+
31+
const generateFileId = (): string => {
32+
return randomBytes(16).toString('hex');
33+
};
34+
35+
export const generateDownloadUrl = (objectName: string, config: UploadFileConfig): string => {
36+
const pathParts = objectName.split('/');
37+
const encodedParts = pathParts.map((part) => encodeURIComponent(part));
38+
const encodedObjectName = encodedParts.join('/');
39+
return `${config.bucket}/${encodedObjectName}`;
40+
};
41+
42+
//Generate a pre-signed URL for direct file upload
43+
export const generatePresignedUrl = async (
44+
input: PresignedUrlInput
45+
): Promise<PresignedUrlResponse> => {
46+
const currentConfig = { ...globalConfig };
47+
48+
const fileId = generateFileId();
49+
const pluginType = input.pluginType || FilePluginTypeEnum.tool;
50+
const pluginPath = PLUGIN_TYPE_TO_PATH_MAP[pluginType];
51+
const objectName = `${pluginPath}/${fileId}/${input.filename}`;
52+
const contentType = input.contentType || inferContentType(input.filename);
53+
const maxSize = input.maxSize || currentConfig.maxFileSize;
54+
55+
try {
56+
const policy = connectionMinio.newPostPolicy();
57+
58+
policy.setBucket(currentConfig.bucket);
59+
policy.setKey(objectName);
60+
policy.setContentType(contentType);
61+
policy.setContentLengthRange(1, maxSize);
62+
policy.setExpires(new Date(Date.now() + 10 * 60 * 1000));
63+
64+
const metadata = {
65+
'original-filename': encodeURIComponent(input.filename),
66+
'upload-time': new Date().toISOString(),
67+
'file-id': fileId,
68+
...input.metadata
69+
};
70+
policy.setUserMetaData(metadata);
71+
72+
const { postURL, formData } = await connectionMinio.presignedPostPolicy(policy);
73+
74+
const response: PresignedUrlResponse = {
75+
fileId,
76+
objectName,
77+
uploadUrl: postURL,
78+
formData
79+
};
80+
81+
return response;
82+
} catch (error) {
83+
addLog.error('Failed to generate presigned URL', error);
84+
const errorMessage = error instanceof Error ? error.message : 'Unknown error';
85+
return Promise.reject(`Failed to generate presigned URL: ${errorMessage}`);
86+
}
87+
};
88+
89+
//Confirm the presigned URL upload is complete and save the file information to MongoDB
90+
export const confirmPresignedUpload = async (objectName: string, size: string): Promise<string> => {
91+
try {
92+
const currentConfig = { ...globalConfig };
93+
const stat = await connectionMinio.statObject(currentConfig.bucket, objectName);
94+
95+
if (stat.size !== Number(size)) {
96+
addLog.error(`File size mismatch. Expected: ${size}, Actual: ${stat.size}`);
97+
return Promise.reject(`File size mismatch. Expected: ${size}, Actual: ${stat.size}`);
98+
}
99+
100+
const accessUrl = generateDownloadUrl(objectName, currentConfig);
101+
102+
return accessUrl;
103+
} catch (error) {
104+
addLog.error('Failed to confirm presigned upload', error);
105+
return Promise.reject(`Failed to confirm presigned upload: ${error}`);
106+
}
107+
};
Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
import * as path from 'path';
2+
3+
export const mimeTypes: Record<string, string> = {
4+
'.js': 'application/javascript'
5+
};
6+
7+
export const inferContentType = (filename: string): string => {
8+
const ext = path.extname(filename).toLowerCase();
9+
return mimeTypes[ext] || 'application/octet-stream';
10+
};
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
import { Client } from 'minio';
2+
3+
export * from 'minio';
4+
export { Client };
5+
6+
export const S3_ENDPOINT = process.env.S3_ENDPOINT || 'localhost';
7+
export const S3_PORT = process.env.S3_PORT ? parseInt(process.env.S3_PORT) : 9000;
8+
export const S3_USE_SSL = process.env.S3_USE_SSL === 'true';
9+
export const S3_ACCESS_KEY = process.env.S3_ACCESS_KEY || 'minioadmin';
10+
export const S3_SECRET_KEY = process.env.S3_SECRET_KEY || 'minioadmin';
11+
12+
export const connectionMinio = (() => {
13+
if (!global.minioClient) {
14+
global.minioClient = new Client({
15+
endPoint: S3_ENDPOINT,
16+
port: S3_PORT,
17+
useSSL: S3_USE_SSL,
18+
accessKey: S3_ACCESS_KEY,
19+
secretKey: S3_SECRET_KEY
20+
});
21+
}
22+
return global.minioClient;
23+
})();
24+
25+
export const getMinioClient = () => connectionMinio;
26+
27+
export default connectionMinio;

0 commit comments

Comments
 (0)