Skip to content

Commit 84cd7ff

Browse files
committed
Assistant checkpoint: Move architecture diagrams to dedicated docs file
Assistant generated file changes: - docs/ARCHITECTURE.md: Create dedicated architecture documentation with all diagrams - README.md: Update README to reference architecture documentation --- User prompt: ALso keep those architecture diagrams as well. Replit-Commit-Author: Assistant Replit-Commit-Session-Id: e5c8aa2d-05f7-401c-b338-9e102c33d9ab
1 parent 317c39a commit 84cd7ff

File tree

2 files changed

+329
-147
lines changed

2 files changed

+329
-147
lines changed

README.md

Lines changed: 9 additions & 147 deletions
Original file line numberDiff line numberDiff line change
@@ -131,7 +131,7 @@ import { NodeQMindMap } from 'nodeq-mindmap';
131131

132132
const MindMapComponent = ({ data }) => {
133133
const containerRef = useRef(null);
134-
134+
135135
useEffect(() => {
136136
const mindMap = new NodeQMindMap({
137137
container: containerRef.current,
@@ -140,7 +140,7 @@ const MindMapComponent = ({ data }) => {
140140
});
141141
mindMap.render();
142142
}, [data]);
143-
143+
144144
return <div ref={containerRef}></div>;
145145
};
146146
```
@@ -1061,153 +1061,15 @@ const pipeline = await mindMap.createDataPipeline(
10611061

10621062
## 🏗️ System Architecture
10631063

1064-
### Core Architecture Overview
1065-
1066-
```mermaid
1067-
graph TB
1068-
subgraph "🎨 Visualization Layer"
1069-
A[Interactive Mind Maps]
1070-
B[Pipeline Monitoring]
1071-
C[Performance Dashboards]
1072-
end
1073-
1074-
subgraph "🧠 ML Analysis Engine"
1075-
D[TensorFlow.js Model]
1076-
E[Hugging Face Integration]
1077-
F[OpenAI API Support]
1078-
G[Pattern Recognition]
1079-
H[Field Mapping Intelligence]
1080-
end
1081-
1082-
subgraph "⚙️ Pipeline Engine"
1083-
I[ETL Process Automation]
1084-
J[Stream Processing]
1085-
K[Data Quality Validation]
1086-
L[Error Handling]
1087-
end
1088-
1089-
subgraph "🔌 Data Source Connectors"
1090-
M[IoT Hub]
1091-
N[Kafka]
1092-
O[REST APIs]
1093-
P[WebSockets]
1094-
Q[MQTT]
1095-
R[Databases]
1096-
end
1097-
1098-
A --> D
1099-
B --> I
1100-
C --> J
1101-
D --> I
1102-
E --> G
1103-
F --> H
1104-
G --> I
1105-
H --> J
1106-
M --> I
1107-
N --> J
1108-
O --> I
1109-
P --> J
1110-
Q --> I
1111-
R --> I
1112-
```
1113-
1114-
### ETL Process Flow
1115-
1116-
```mermaid
1117-
graph LR
1118-
subgraph "Data Input"
1119-
A[IoT Sensors]
1120-
B[Kafka Streams]
1121-
C[REST APIs]
1122-
D[Databases]
1123-
end
1124-
1125-
E[Data Extraction] --> F[Auto Schema Detection]
1126-
F --> G[ML Pattern Analysis]
1127-
G --> H[Rule Generation]
1128-
H --> I[Data Transformation]
1129-
I --> J[Quality Validation]
1130-
J --> K[Error Handling]
1131-
K --> L[Output Processing]
1132-
1133-
subgraph "Data Output"
1134-
M[Analytics DB]
1135-
N[Real-time Dashboard]
1136-
O[Alert Systems]
1137-
P[ML Training Data]
1138-
end
1139-
1140-
A --> E
1141-
B --> E
1142-
C --> E
1143-
D --> E
1144-
1145-
L --> M
1146-
L --> N
1147-
L --> O
1148-
L --> P
1149-
```
1064+
NodeQ MindMap features a sophisticated multi-layered architecture designed for scalability and performance. The system combines interactive visualization, ML-powered pattern recognition, and automated ETL processing.
11501065

1151-
### ML Model Interaction Lifecycle
1066+
**[📋 View Complete Architecture Documentation](docs/ARCHITECTURE.md)**
11521067

1153-
```mermaid
1154-
sequenceDiagram
1155-
participant User as User
1156-
participant NodeQ as NodeQ Engine
1157-
participant ML as ML Model
1158-
participant Pipeline as Compiled Pipeline
1159-
participant Data as Input Data
1160-
1161-
Note over User,Data: Pipeline Creation Phase (ML Active)
1162-
User->>NodeQ: Create Pipeline (input/output samples)
1163-
NodeQ->>ML: Initialize Model
1164-
ML->>ML: Analyze transformation patterns
1165-
ML->>NodeQ: Generate transformation rules
1166-
NodeQ->>Pipeline: Compile static execution logic
1167-
NodeQ->>User: Pipeline ready (ML model no longer needed)
1168-
1169-
Note over User,Data: Execution Phase (Static, ML-Free)
1170-
Data->>Pipeline: Input data
1171-
Pipeline->>Pipeline: Execute compiled transformations
1172-
Pipeline->>User: Transformed output (fast, no ML overhead)
1173-
1174-
Note over User,Data: Configuration Update Phase (ML Re-activated)
1175-
User->>NodeQ: Update pipeline config
1176-
NodeQ->>ML: Re-initialize model
1177-
ML->>ML: Re-analyze with new samples
1178-
ML->>NodeQ: Generate updated rules
1179-
NodeQ->>Pipeline: Recompile static logic
1180-
NodeQ->>User: Updated pipeline ready
1181-
```
1182-
1183-
### Static vs Dynamic Execution
1184-
1185-
```mermaid
1186-
graph TB
1187-
subgraph "Pipeline Creation (ML Active)"
1188-
A[Input/Output Samples] --> B[ML Model Analysis]
1189-
B --> C[Transformation Rules Generated]
1190-
C --> D[Static Logic Compiled]
1191-
D --> E[Pipeline Ready]
1192-
end
1193-
1194-
subgraph "Runtime Execution (ML-Free)"
1195-
F[Incoming Data] --> G[Compiled Transform Function]
1196-
G --> H[Output Data]
1197-
1198-
style G fill:#4caf50,color:#fff
1199-
style H fill:#4caf50,color:#fff
1200-
end
1201-
1202-
subgraph "Config Changes (ML Re-activated)"
1203-
I[New Input/Output Format] --> J[ML Re-analysis]
1204-
J --> K[Updated Rules]
1205-
K --> L[Recompiled Logic]
1206-
end
1207-
1208-
E -.->|Static Execution| G
1209-
L -.->|Updated Logic| G
1210-
```
1068+
Key architectural components:
1069+
- **🎨 Visualization Layer**: Interactive mind maps and monitoring dashboards
1070+
- **🧠 ML Analysis Engine**: TensorFlow.js and Hugging Face integration
1071+
- **⚙️ Pipeline Engine**: ETL automation and stream processing
1072+
- **🔌 Data Connectors**: Support for IoT, Kafka, APIs, and databases
12111073

12121074
## 🔄 ETL Process Replacement
12131075

0 commit comments

Comments
 (0)