Skip to content

Commit c962dcf

Browse files
JohnRichard4096JohnRichard4096
authored
Introduce multimodal MessageContent and session-aware chat hooks (#7)
* feat: multimodal and types * fix: types
1 parent 0e07cef commit c962dcf

File tree

12 files changed

+875
-163
lines changed

12 files changed

+875
-163
lines changed

‎docs/guide/getting-started/architecture.md‎

Lines changed: 91 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22

33
## 2.4.1 Architecture Diagram
44

5+
### Core Architecture
6+
57
```mermaid
68
graph TB
79
subgraph "AmritaCore"
@@ -12,7 +14,7 @@ graph TB
1214
E[Memory Model]
1315
F[Agent Core]
1416
end
15-
17+
1618
UserInput[User Input] --> A
1719
A --> B
1820
A --> C
@@ -22,16 +24,51 @@ graph TB
2224
D --> F
2325
E --> F
2426
F --> ResponseStream[Response Stream]
25-
27+
2628
ResponseStream --> UserOutput[User Output]
2729
F --> E
28-
30+
2931
LLM[LLM Provider] <---> F
30-
32+
3133
style AmritaCore fill:#e1f5fe,stroke:#0277bd,stroke-width:2px
3234
style F fill:#fff3e0,stroke:#f57c00,stroke-width:2px
3335
```
3436

37+
### Session and Global Data Container Architecture
38+
39+
#### Global Container and Session Conversation Context
40+
41+
```mermaid
42+
graph TB
43+
subgraph "Global Container"
44+
G_Tools[Global Tools]
45+
G_Presets[Global Presets]
46+
G_Config[Global Configuration]
47+
end
48+
49+
subgraph "SessionsManager"
50+
S1[Session 1<br/>Conversation Context]
51+
S2[Session 2<br/>Conversation Context]
52+
SN[Session N<br/>Conversation Context]
53+
end
54+
55+
subgraph "Single Session Structure"
56+
Mem[Memory Model<br/>Conversation History]
57+
Tools[Tools Manager<br/>Current Session Tools]
58+
Conf[Configuration<br/>Current Session Config]
59+
end
60+
61+
G_Tools -.-> Tools
62+
G_Presets -.-> S1
63+
G_Config -.-> Conf
64+
65+
style Global_Container fill:#e8f5e9,stroke:#2e7d32,stroke-width:2px
66+
style Session_Container fill:#e3f2fd,stroke:#1565c0,stroke-width:2px
67+
style S1 fill:#bbdefb,stroke:#1565c0,stroke-width:1px
68+
style S2 fill:#bbdefb,stroke:#1565c0,stroke-width:1px
69+
style SN fill:#bbdefb,stroke:#1565c0,stroke-width:1px
70+
```
71+
3572
## 2.4.2 Core Component Relationships
3673

3774
- **ChatObject**: The main interaction point that manages a single conversation
@@ -40,31 +77,60 @@ graph TB
4077
- **Tools Manager**: Extends the agent's capabilities with external functions
4178
- **Memory Model**: Maintains conversation context and history
4279
- **Agent Core**: The central processing unit coordinating all components
80+
- **SessionsManager**: Manages multiple isolated sessions, each as an independent conversation context
81+
- **Session (Conversation Context)**: Stores all relevant information for a specific user or specific conversation, including memory model, tools, configurations, etc.
4382

44-
## 2.4.3 Data Flow Explanation
83+
## 2.4.3 Agent Loop and Session Isolation Mechanism
4584

4685
```mermaid
4786
sequenceDiagram
48-
User->>CO: Send input message
49-
CO->>Config: Check configuration
50-
CO->>Events: Trigger input events
51-
CO->>Agent: Initialize processing
52-
Agent->>Memory: Load conversation context
53-
Agent->>Tools: Check for required tools
54-
Agent->>LLM: Format and send request
87+
participant User1 as User1
88+
participant User2 as User2
89+
participant SM as SessionsManager
90+
participant S1 as Session 1<br/>(Conversation Context 1)
91+
participant S2 as Session 2<br/>(Conversation Context 2)
92+
participant Agent as Agent Core
93+
participant LLM as LLM Provider
94+
95+
Note over User1,LLM: Agent Loop Begins
96+
User1->>SM: Request to create Session 1
97+
User2->>SM: Request to create Session 2
98+
SM-->>User1: Return Session ID 1
99+
SM-->>User2: Return Session ID 2
100+
101+
Note over User1,User2: Each user interacts in their respective conversation context
102+
103+
User1->>S1: Send message to Session 1
104+
S1->>S1: Initialize ChatObject
105+
S1->>Agent: Start Agent Loop to process request
106+
107+
User2->>S2: Send message to Session 2
108+
S2->>S2: Initialize ChatObject
109+
S2->>Agent: Start Agent Loop to process request
110+
111+
par Parallel Processing of Two Conversation Contexts
112+
Agent->>S1: Process in Session 1 Context
113+
Agent->>S2: Process in Session 2 Context
114+
115+
Agent->>S1: Update Session 1 Memory Model
116+
Agent->>S2: Update Session 2 Memory Model
117+
end
118+
119+
Agent->>LLM: Send request (from Session 1)
55120
LLM-->>Agent: Return response
56-
Agent->>Memory: Update conversation state
57-
Agent->>Events: Trigger output events
58-
CO-->>User: Stream response
121+
Agent-->>S1: Update Session 1 State
122+
S1-->>User1: Stream response
123+
124+
Agent->>LLM: Send request (from Session 2)
125+
LLM-->>Agent: Return response
126+
Agent-->>S2: Update Session 2 State
127+
S2-->>User2: Stream response
128+
129+
Note over User1,LLM: Each conversation context maintains independent history and state
59130
```
60131

61-
1. User input enters through a ChatObject
62-
2. Configuration determines processing behavior
63-
3. The input triggers various events in the Events System
64-
4. Agent Core loads the conversation context from Memory Model
65-
5. Agent Core checks if any tools need to be called based on the input
66-
6. The processed input is sent to the LLM Provider
67-
7. The response from LLM is handled by Agent Core
68-
8. Memory Model is updated with the new conversation state
69-
9. Output events may intercept and modify the final response
70-
10. The response is streamed back to the user via ChatObject
132+
1. **Session as Conversation Context**: Each Session represents an independent conversation context, storing all relevant information for a specific user or specific conversation
133+
2. **Global Data Container**: SessionsManager manages all active conversation contexts, providing global resource sharing
134+
3. **Agent Loop**: Inside each conversation context, the Agent Core executes the complete processing loop
135+
4. **Context Isolation**: Data between different conversation contexts is completely isolated, ensuring conversation histories don't mix
136+
5. **Global Resource Sharing**: Each conversation context can access resources from the Global container, but maintains its own independent state

‎docs/zh/guide/getting-started/architecture.md‎

Lines changed: 88 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22

33
## 2.4.1 架构图
44

5+
### Core Architecture
6+
57
```mermaid
68
graph TB
79
subgraph "AmritaCore"
@@ -32,6 +34,41 @@ graph TB
3234
style F fill:#fff3e0,stroke:#f57c00,stroke-width:2px
3335
```
3436

37+
### Session 与 Global 数据容器架构
38+
39+
#### Global 全局容器与 Session 对话上下文
40+
41+
```mermaid
42+
graph TB
43+
subgraph "Global 全局容器"
44+
G_Tools[全局工具]
45+
G_Presets[全局预设]
46+
G_Config[全局配置]
47+
end
48+
49+
subgraph "SessionsManager"
50+
S1[Session 1<br/>对话上下文]
51+
S2[Session 2<br/>对话上下文]
52+
SN[Session N<br/>对话上下文]
53+
end
54+
55+
subgraph "单个 Session 结构"
56+
Mem[记忆模型<br/>对话历史]
57+
Tools[工具管理器<br/>当前会话工具]
58+
Conf[配置<br/>当前会话配置]
59+
end
60+
61+
G_Tools -.-> Tools
62+
G_Presets -.-> S1
63+
G_Config -.-> Conf
64+
65+
style Global_Container fill:#e8f5e9,stroke:#2e7d32,stroke-width:2px
66+
style Session_Container fill:#e3f2fd,stroke:#1565c0,stroke-width:2px
67+
style S1 fill:#bbdefb,stroke:#1565c0,stroke-width:1px
68+
style S2 fill:#bbdefb,stroke:#1565c0,stroke-width:1px
69+
style SN fill:#bbdefb,stroke:#1565c0,stroke-width:1px
70+
```
71+
3572
## 2.4.2 核心组件关系
3673

3774
- **ChatObject**: 管理单个对话的主要交互点
@@ -40,31 +77,60 @@ graph TB
4077
- **工具管理器**: 通过外部函数扩展Agent功能
4178
- **记忆模型**: 维护对话上下文和历史记录
4279
- **Agent核心**: 协调所有组件的中央处理单元
80+
- **SessionsManager**: 管理多个独立的会话，每个会话都是一个独立的对话上下文
81+
- **Session（对话上下文）**: 保存特定用户或特定对话的所有相关信息，包括记忆模型、工具、配置等
4382

44-
## 2.4.3 数据流说明
83+
## 2.4.3 Agent 循环与 Session 隔离机制
4584

4685
```mermaid
4786
sequenceDiagram
48-
用户->>ChatObject: 发送输入消息
49-
ChatObject->>配置: 检查配置
50-
ChatObject->>事件系统: 触发输入事件
51-
ChatObject->>Agent核心: 初始化处理
52-
Agent核心->>记忆模型: 加载对话上下文
53-
Agent核心->>工具管理器: 检查所需的工具
54-
Agent核心->>LLM 提供商: 格式化并发送请求
55-
LLM 提供商-->>Agent核心: 返回响应
56-
Agent核心->>记忆模型: 更新对话状态
57-
Agent核心->>事件系统: 触发输出事件
58-
ChatObject-->>用户: 流式传输响应
87+
participant User1 as 用户1
88+
participant User2 as 用户2
89+
participant SM as SessionsManager
90+
participant S1 as Session 1<br/>(对话上下文1)
91+
participant S2 as Session 2<br/>(对话上下文2)
92+
participant Agent as Agent核心
93+
participant LLM as LLM 提供商
94+
95+
Note over User1,LLM: Agent 循环开始
96+
User1->>SM: 请求创建 Session 1
97+
User2->>SM: 请求创建 Session 2
98+
SM-->>User1: 返回 Session ID 1
99+
SM-->>User2: 返回 Session ID 2
100+
101+
Note over User1,User2: 每个用户在各自的对话上下文中交互
102+
103+
User1->>S1: 发送消息到 Session 1
104+
S1->>S1: 初始化 ChatObject
105+
S1->>Agent: 启动 Agent 循环处理请求
106+
107+
User2->>S2: 发送消息到 Session 2
108+
S2->>S2: 初始化 ChatObject
109+
S2->>Agent: 启动 Agent 循环处理请求
110+
111+
par 并行处理两个对话上下文
112+
Agent->>S1: 在 Session 1 上下文中处理
113+
Agent->>S2: 在 Session 2 上下文中处理
114+
115+
Agent->>S1: 更新 Session 1 记忆模型
116+
Agent->>S2: 更新 Session 2 记忆模型
117+
end
118+
119+
Agent->>LLM: 发送请求 (来自 Session 1)
120+
LLM-->>Agent: 返回响应
121+
Agent-->>S1: 更新 Session 1 状态
122+
S1-->>User1: 流式传输响应
123+
124+
Agent->>LLM: 发送请求 (来自 Session 2)
125+
LLM-->>Agent: 返回响应
126+
Agent-->>S2: 更新 Session 2 状态
127+
S2-->>User2: 流式传输响应
128+
129+
Note over User1,LLM: 每个对话上下文保持独立的历史和状态
59130
```
60131

61-
1. 用户输入通过 ChatObject 进入
62-
2. 配置确定处理行为
63-
3. 输入触发事件系统中的各种事件
64-
4. Agent核心从记忆模型加载对话上下文
65-
5. Agent核心根据输入检查是否需要调用任何工具
66-
6. 处理后的输入发送到 LLM 提供商
67-
7. LLM 的响应由Agent核心处理
68-
8. 记忆模型使用新的对话状态进行更新
69-
9. 输出事件可能会拦截并修改最终响应
70-
10. 响应通过 ChatObject 流式传输回用户
132+
1. **Session 作为对话上下文**: 每个 Session 代表一个独立的对话上下文，保存特定用户或特定对话的所有相关信息
133+
2. **Global 数据容器**: SessionsManager 管理所有活动的对话上下文，提供全局资源共享
134+
3. **Agent 循环**: 在每个对话上下文内部，Agent 核心执行完整的处理循环
135+
4. **上下文隔离**: 不同对话上下文之间的数据完全隔离，确保对话历史不混淆
136+
5. **全局资源共享**: 每个对话上下文可以访问 Global 容器中的公共资源，但拥有各自独立的状态

‎pyproject.toml‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,14 @@
11
[project]
22
name = "amrita_core"
3-
version = "0.4.0.post1"
3+
version = "0.4.1"
44
description = "Agent core of Project Amrita"
55
readme = "README.md"
66
requires-python = ">=3.10,<3.14"
77
dependencies = [
8+
"aiofiles>=25.1.0",
9+
"aiohttp>=3.13.3",
810
"fastmcp>=2.14.4",
11+
"filetype>=1.2.0",
912
"jieba>=0.42.1",
1013
"loguru>=0.7.3",
1114
"openai>=2.16.0",

‎src/amrita_core/builtins/adapter.py‎

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,11 @@
1717

1818
from amrita_core.config import AmritaConfig
1919
from amrita_core.logging import debug_log
20-
from amrita_core.protocol import ModelAdapter
20+
from amrita_core.protocol import (
21+
COMPLETION_RETURNING,
22+
ModelAdapter,
23+
StringMessageContent,
24+
)
2125
from amrita_core.tools.models import ToolChoice, ToolFunctionSchema
2226
from amrita_core.types import (
2327
ModelConfig,
@@ -36,7 +40,7 @@ class OpenAIAdapter(ModelAdapter):
3640
@override
3741
async def call_api(
3842
self, messages: Iterable[ChatCompletionMessageParam]
39-
) -> AsyncGenerator[str | UniResponse[str, None], None]:
43+
) -> AsyncGenerator[COMPLETION_RETURNING, None]:
4044
"""Call OpenAI API to get chat responses"""
4145
preset: ModelPreset = self.preset
4246
preset_config: ModelConfig = preset.config
@@ -81,7 +85,7 @@ async def call_api(
8185
)
8286
if chunk.choices[0].delta.content is not None:
8387
response += chunk.choices[0].delta.content
84-
yield chunk.choices[0].delta.content
88+
yield StringMessageContent(response)
8589
debug_log(chunk.choices[0].delta.content)
8690
except IndexError:
8791
break
@@ -93,7 +97,7 @@ async def call_api(
9397
if completion.choices[0].message.content is not None
9498
else ""
9599
)
96-
yield response
100+
yield StringMessageContent(response)
97101
if completion.usage:
98102
uni_usage = UniResponseUsage.model_validate(
99103
completion.usage, from_attributes=True

‎src/amrita_core/builtins/agent.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,6 @@
55
from copy import deepcopy
66
from typing import Any
77

8-
from amrita_core.chatmanager import MessageWithMetadata
98
from amrita_core.config import AmritaConfig, get_config
109
from amrita_core.hook.event import CompletionEvent, PreCompletionEvent
1110
from amrita_core.hook.exception import MatcherException as ProcEXC
@@ -14,6 +13,7 @@
1413
tools_caller,
1514
)
1615
from amrita_core.logging import debug_log, logger
16+
from amrita_core.protocol import MessageWithMetadata
1717
from amrita_core.sessions import SessionsManager
1818
from amrita_core.tools.manager import ToolsManager, on_tools
1919
from amrita_core.tools.models import ToolContext

0 commit comments

Comments
 (0)