Skip to content

Commit 1f7b08c

Browse files
committed
docs: update README to reflect consolidated script structure
1 parent c5ac3c0 commit 1f7b08c

File tree

1 file changed

+25
-24
lines changed

1 file changed

+25
-24
lines changed

README.md

Lines changed: 25 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -138,15 +138,20 @@ Access via **EOxHub workspace** (single sign-on): [workspace.devseed.hub-eopf-ex
138138

139139
## Pipeline
140140

141-
```
142-
STAC item URL → Extract zarr → Convert (Dask) → S3 → Register STAC + TiTiler → Done (~15-20 min)
143-
```
141+
**Flow:** STAC item URL → Extract zarr → Convert to GeoZarr → Upload S3 → Register STAC item → Add visualization links
144142

145-
**Steps:**
146-
1. **Convert** - Fetch STAC item, extract zarr URL, convert to GeoZarr, upload to S3
147-
2. **Register** - Create STAC item with TiTiler preview links, register to catalog
143+
**Processing:**
144+
1. **convert.py** - Fetch STAC item, extract zarr URL, convert to cloud-optimized GeoZarr, upload to S3
145+
2. **register.py** - Create STAC item with asset hrefs, add projection metadata and TiTiler links, register to catalog
148146

149-
**Stack:** Argo Workflows • [eopf-geozarr](https://github.com/EOPF-Explorer/data-model) • Dask • RabbitMQ • Kustomize
147+
**Runtime:** ~15-20 minutes per item
148+
149+
**Stack:**
150+
- Orchestration: Argo Workflows, Kustomize
151+
- Processing: eopf-geozarr, Dask, Python 3.13
152+
- Storage: S3 (OVH)
153+
- Catalog: pgSTAC, TiTiler
154+
- Events: RabbitMQ
150155

151156
---
152157

@@ -177,27 +182,23 @@ kubectl get wf -n devseed-staging --sort-by=.metadata.creationTimestamp \
177182

178183
---
179184

180-
## Structure
185+
## Repository Structure
181186

182187
```
183-
scripts/ # Workflow steps
184-
├── convert.py # GeoZarr conversion (extract zarr URL, convert, upload)
185-
├── register.py # STAC registration orchestrator
186-
├── register_stac.py # STAC item creation with TiTiler links
187-
├── create_geozarr_item.py # Convert zarr → geozarr
188-
├── augment_stac_item.py # Add visualization links to STAC items
189-
└── get_conversion_params.py # Fetch collection config
190-
191-
workflows/ # Kubernetes manifests (Kustomize)
192-
├── base/ # WorkflowTemplate, EventSource, Sensor, RBAC
193-
└── overlays/ # staging, production configs
194-
195-
docker/Dockerfile # Pipeline image
196-
tools/submit_burst.py # RabbitMQ burst submission tool
188+
scripts/
189+
├── convert.py # Zarr → GeoZarr conversion and S3 upload
190+
└── register.py # STAC item creation and catalog registration
191+
192+
workflows/ # Kubernetes manifests
193+
├── base/ # WorkflowTemplate, EventSource, Sensor, RBAC
194+
└── overlays/staging/ # Environment configuration
195+
/production/
196+
197+
docker/Dockerfile # Container image
198+
tests/unit/ # Unit tests
199+
/integration/ # Integration tests
197200
```
198201

199-
Tests are planned for `tests/` directory (structure exists, test files to be added).
200-
201202
---
202203

203204
## Configuration

0 commit comments

Comments
 (0)