Deep Supervision and Adaptive Compute for Process-Chain Modeling

Technical research report on shared-weight recursive Transformers, deep supervision, and adaptive inference over Windows process-ancestry chains.

Status: Technical research report developed with academic supervision; not formally peer reviewed.

Read the Technical Research Report

Overview

This project studies whether mechanisms inspired by Tiny Recursive Model-style architectures transfer to masked process-chain modeling over large-scale security telemetry.

The study uses Windows Event ID 4688 process-creation records from the LANL Unified Host dataset. Raw events are reconstructed into temporal root-to-leaf process-ancestry chains and modeled using a shared-weight Transformer trained with masked log-key prediction.

The work focuses on three questions:

1. Which recursive-model mechanisms improve process-chain modeling?
2. Can inference compute be adapted per sample without retraining?
3. Do refinement dynamics expose useful effort or instability signals?

This is not a validated anomaly-detection system. The dataset used in this study does not contain attack labels, so the report evaluates intrinsic masked-prediction quality, compute behavior, and refinement dynamics rather than anomaly-detection performance.

Key Results

• Built a restartable Bronze/Silver data pipeline over approximately 1.2 TB of compressed Windows telemetry.
• Reconstructed approximately 1.4 billion temporal process-ancestry chains in about four hours on consumer Intel i5-class hardware.
• Found that deep supervision was the useful architectural mechanism in the tested recursive model.
• Found that within-step recursion and learned halting did not justify their cost in the tested regime.
• Observed a robust 2–3-segment inference-quality plateau across held-out data and three retraining seeds.
• Selected an early-exit policy using validation data that reduced inference compute by 41.2% relative to a fixed three-segment budget while remaining within a predefined quality tolerance.
• In matched held-out comparisons, LogBERT achieved better intrinsic masked-prediction quality.
• The shared-weight model used 6.89 million trainable parameters, compared with 11.09 million for the matched LogBERT architecture, and supported adaptive inference budgets.

Data Engineering

The report describes a purpose-built processing pipeline with:

• streaming ingestion using PyArrow;
• explicit schemas and filtering for Event ID 4688 telemetry;
• partitioned, ZSTD-compressed Parquet storage;
• temporal PID-reuse-aware ancestry reconstruction;
• bounded-memory eviction;
• exact-state memoization;
• deterministic, host-balanced sampling;
• leaf-time temporal train, validation, and test splits;
• manifest-backed experiment provenance.

Modeling and Evaluation

The model is a shared-weight Transformer-style encoder evaluated using masked log-key prediction.

The study includes:

• structural ablations of deep supervision;
• within-step recursive refinement;
• learned halting;
• fixed inference-budget sweeps;
• validation-selected early-exit policies;
• held-out evaluation across three retraining seeds;
• matched comparison with a LogBERT-style baseline;
• analysis of prediction-instability signals.

Main Conclusion

The tested recursive architecture did not transfer as a complete package.

The useful findings were more specific:

• deep supervision improved iterative refinement;
• inference quality generally plateaued after approximately 2–3 segments;
• additional refinement could degrade quality;
• validation-selected early exit reduced compute while remaining within the predefined quality tolerance;
• prediction-flip instability emerged as a promising signal for future study.

The contribution is therefore not a state-of-the-art anomaly detector. It is an evidence-based analysis of which recursive-computation mechanisms transfer to process-chain modeling, which do not, and how prediction quality changes with inference effort.

Current Status

This repository contains the public technical report only.

The implementation repository, experiment artifacts, and raw data pipeline are currently private.

The report has not been peer reviewed.

The next research phase will use labeled datasets to investigate:

• adaptive computation during inference;
• model effort as a possible anomaly signal;
• anomaly-detection metrics such as PR-AUC and recall at fixed false-positive rates;
• transfer across fresh checkpoints and masking seeds.

The longer-term objective is to develop the follow-up study into a submission for an international peer-reviewed venue.

Suggested Citation

Talha Zencirkıran.
Deep Supervision and Adaptive Compute for Process-Chain Modeling.
Technical Research Report, 2026.

License

The technical report and repository documentation are licensed under the Creative Commons Attribution 4.0 International License.

© 2026 Talha Zencirkıran