Author and maintainer: Aleksandar Kondinski, Graz University of Technology
Automated PXRD processing, human-in-the-loop phase assignment, crystallinity analysis, and structured data preparation for ZIF biocomposite synthesis–structure–property datasets.
This repository provides an automated analytics workflow for ZIF-based biocomposite datasets. The pipeline was developed to replace previously manual or semi-manual processing of individual diffractograms with a reproducible batch workflow for background correction, amorphous-envelope fitting, peak detection, phase assignment, crystallinity analysis, replicate aggregation, and structured JSON report generation.
The workflow includes a human-in-the-loop validation step for ambiguous cases. When automated rules reach the boundary of reliable phase assignment, selected cases can be reviewed by expert inspection before the pipeline continues. This makes the workflow scalable, reproducible, and scientifically auditable.
The resulting reports provide structured inputs for downstream synthesis–structure–property analysis, including separate visualisation and modelling workflows.
The workflow ingests raw PXRD .txt files, processes 1,080 measurements across three rounds, subtracts physical background, fits an ALS amorphous envelope, detects crystalline peaks, identifies phases against curated reference peaks, estimates crystallinity descriptors, applies documented expert-review corrections where needed, and aggregates results into structured JSON reports for downstream visualisation and modelling.
For a detailed description of the automated processing, reference-annotation comparison, and human-in-the-loop review process, see METHODS_WORKFLOW.md.
# Install dependencies
pip install -r requirements.txt
# Run the full pipeline
python main.py
# Inspect published results
cat results/summary.jsonzif-biocomposite-analytics/
|-- data/ # Input data (gitignored)
| |-- XRD/ # Raw .txt diffractograms + background.txt
| |-- ATR_xy/ # ATR-IR spectra
| |-- EE/ # Encapsulation efficiency CSVs
| `-- LC_two_column.csv # LC% data
|
|-- pxrdpipe/ # Core pipeline package
| |-- analysis.py # Per-file XRD analysis
| |-- peaks.py # Peak detection + interpolation
| |-- phases.py # Phase selection rules
| |-- aggregate.py # Replicate consensus + aggregation
| |-- config.py # Tunable constants
| `-- ... # I/O, baseline, ATR, EE, LC, reporting
|
|-- scripts/ # Diagnostics and validation tools
| |-- validate_phases.py # Accuracy checker against reference annotations
| `-- plot_diagnostics.py # Per-sample diagnostic PNGs
|
|-- results/ # Published pipeline outputs
| |-- summary.json # Redacted dataset: 360 ternary points, phase + crystallinity + ATR-IR
| `-- review_overrides.json # 31 expert phase corrections applied by Aleksandar Kondinski during human-in-the-loop review
|
|-- outputs/ # Generated locally (gitignored)
| |-- xy/ # Background-subtracted .xy files
| |-- png/ # Diagnostic plots
| `-- reports/
| `-- summary.json # Full local output (includes EE and LC, not published)
|
|-- CITATION.cff
|-- NOTICE.md
|-- main.py # Pipeline entry point
`-- requirements.txt
The fully automated pre-review phase-assignment workflow achieved 90.6% agreement with reference annotations. These reference annotations originate from a prior manual or semi-manual phase-analysis workflow and are treated here as comparison annotations rather than absolute ground truth.
Disagreement cases were inspected through a graphical human-in-the-loop review step. The file results/review_overrides.json records 31 expert phase corrections applied by Aleksandar Kondinski during this review process. These documented review decisions were applied to produce the final expert-reviewed output.
After applying the documented expert-review corrections, the post-review output achieved 97.5% agreement with the reference annotations. The remaining 9 cases out of 360 are retained as expert-reviewed differences, where the final transparent pipeline assignment was kept even though it does not coincide with the prior reference annotation. They are not treated as automatic pipeline errors, because the reference annotations are comparison annotations rather than absolute ground truth.
| Metric | Value |
|---|---|
| Agreement with reference annotations, automated pre-review pass | 90.6% (326 / 360) |
| Agreement with reference annotations, post-review output | 97.5% (351 / 360) |
| Expert-review corrections applied by Aleksandar Kondinski | 31 |
| Expert-reviewed differences retained | 9 / 360 |
-
Amorphous fraction estimation. When no reference phase peaks are matched for a replicate, the pipeline assigns zero crystalline fraction and zero crystalline area for that replicate, regardless of the raw ALS baseline split. This keeps phase assignment and crystallinity fractions internally consistent.
-
Mixed-phase fractions. Some reference phase fractions were originally obtained using RIR-based interpretation. In the absence of internal-standard calibration for all biocomposite samples, the automated workflow uses rule-based or equal-fraction approximations for selected mixed-phase cases.
-
Ambiguous phase assignments. Samples containing weak or overlapping signatures from multiple phases may require human-in-the-loop inspection. The pipeline is designed to support this review step rather than hide the ambiguity.
-
Noise and weak features. Weak features on broad amorphous envelopes can occasionally resemble reference peaks. These cases should be interpreted with diagnostic plots and expert review.
Villanova, Hafner, Carraro. CrystEngComm 2022, 24, 7266–7271. DOI: 10.1039/d2ce00073c
Phase reference peaks and selection rules follow Table S3 and Section S6 of the paper.