BioCypher overarching CLI (working title biotope)
#9
slobentanzer
started this conversation in
Ideas
Replies: 1 comment 4 replies
-
|
@jdreo @mbaric758 as discussed, here are two high-level diagrams for the options we have. The CLI would be able to be in command of multiple steps, probably best to manage that using a config and not a huge amount of CLI parameters, but I know how programmers think. ;) Core at end of pipelineflowchart LR
%% User-Facing Schema Layer
SC["Schema Configuration<br>(Controls Adapters<br>→ Biocypher Mapping)"]
%% Main Pipeline
A["User CLI Invocation"]
B["Adapter 1<br>(Generates Node/Edge Stream)"]
B2["Adapter 2<br>(Generates Node/Edge Stream)"]
B3["... Other Adapters ..."]
C["Named Pipe or<br>STDOUT Middleware<br>"]
D["Fusion Step<br>(DuckDB or Map-Reduce)"]
E["Biocypher Core<br>(Tuple Writing or<br>other format)"]
%% Connections
A --> B
A --> B2
A --> B3
SC -.-> B
SC -.-> B2
SC -.-> B3
SC --> E
B -->|Stream| C
B2 -->|Stream| C
B3 -->|Stream| C
C --> D
D --> E
Core after adaptersflowchart LR
SC["Schema Configuration<br>(Controls Adapters to Biocypher)"]
A["User CLI Invocation"]
B["Adapter 1<br>(Generates Node/Edge Data)"]
B2["Adapter 2<br>(Generates Node/Edge Data)"]
B3["... Other Adapters ..."]
E["Biocypher Core<br>(Receives Adapter Data)"]
O["Biocypher Output<br>Adapter Stream"]
M["Named Pipe or<br>STDOUT Middleware"]
F["Fusion Step<br>(DuckDB or Map-Reduce)"]
%% Schema config influences adapters and Biocypher
SC -.-> B
SC -.-> B2
SC -.-> B3
SC --> E
%% CLI triggers adapters
A --> B
A --> B2
A --> B3
%% Adapters feed data into Biocypher
B --> E
B2 --> E
B3 --> E
%% Biocypher outputs a stream for fusion
E --> O
O --> M
M --> F
These may be a bit too high-level though, lacking some of the procedural details. Please modify to add your own view, particularly with respect to ontoweaver and the fusion aspects. |
Beta Was this translation helpful? Give feedback.
4 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Here we discuss CLI-related things like orchestration and data streams
Beta Was this translation helpful? Give feedback.
All reactions