Grounded physical-world reasoning over product teardowns.
A multimodal AI architecture that understands why products are built the way they are,
how they fail, and how to design them better — within the hard constraints of physics, manufacturing, and economics.
Architecture · Data Taxonomy · Roadmap · Docs · Research · Discuss
The world produces an enormous amount of unstructured engineering intelligence — teardown videos on YouTube, repair guides on iFixit, 1-star reviews describing failures, Reddit threads diagnosing hardware issues — yet none of it is machine-queryable at the causal physics level.
General-purpose vision-language models (VLMs) can describe what they see in a teardown video. They cannot tell you:
- Why the injection molding gate location causes warpage under thermal cycling
- What the microscopic parting line geometry implies about the draft angle and ejector pin force
- How the 0.3mm graphite thermal pad thickness correlates with GPS-workout overheating complaints across 12,000 Amazon reviews
- Whether replacing unfilled PC with 30% glass-filled PA6 in the button stem will survive 100,000 press cycles at 5 ATM
PTKE is built to answer exactly these questions.
The key innovation is linking perception (vision, audio, text) directly to physical causality (material properties, manufacturing constraints, failure physics) through a Boundary Representation (B-rep) topological layer compatible with ISO 10303 STEP.
"Burns my wrist during runs" (Amazon review)
↓ Sentiment-to-subsystem mapping
Thermal failure mode node in EKG
↓ Graph traversal
0.3mm graphite thermal pad (teardown measurement)
↓ B-rep parametric model (STEP-compatible)
Surface area: 12mm² · Thickness: 0.3mm
↓ Physics constraint engine (TAPS + FEA)
ΔT = Q·R_th → Validated: 8°C junction rise above safe threshold
↓ Counterfactual generation
"Competitor uses 0.8mm vapor chamber; replace TIM or add VC for $2.40/unit"
↓ Evidence attribution
[Video: 08:42] [Review: #142332] [B-rep: step_uuid_4821]
Every assertion is grounded, sourced, and physics-validated.
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
│ RAW INPUT │────▶│ PRE-PROCESSING │────▶│ MODAL EXTRACT │
│ (Video/Img/TXT) │ │ (Transcode/OCR/ │ │ (Vision/Audio/ │
│ │ │ Normalize/Scale)│ │ NLP Sub-models)│
└─────────────────┘ └──────────────────┘ └────────┬────────┘
│
┌───────────────────────────────┼────────────────────────┐
▼ ▼ ▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ VISION ENGINE │ │ AUDIO ENGINE │ │ TEXT ENGINE │
│ SAM2 · B-rep │ │ Event Seg · │ │ NER · Relation │
│ CADFS · B-repLer│ │ FFT · ASR │ │ Sentiment · ABSA│
└────────┬────────┘ └────────┬────────┘ └────────┬────────┘
│ │ │
└──────────────────────────────┼─────────────────────────┘
▼
┌─────────────────────┐
│ CROSS-MODAL │
│ GROUNDING LAYER │
│ Grounded-VideoLLM │
│ Hierarchical Proto │
│ Alignment │
└──────────┬──────────┘
▼
┌─────────────────────┐
│ ENGINEERING │
│ KNOWLEDGE GRAPH │
│ Neo4j · Weaviate │
│ PostgreSQL · MinIO │
└──────────┬──────────┘
▼
┌─────────────────────┐
│ AGENTIC RAG + │
│ PHYSICS ENGINE │
│ LangGraph · TAPS │
│ CalculiX · FreeCAD │
└─────────────────────┘
| Tier | Role | Implementation |
|---|---|---|
| Tier 1: Agentic RAG | Grounded retrieval with proactive reasoning | LangGraph DAG · Routing Agent · ReAct agents · RRF fusion |
| Tier 2: Domain LLM | Engineering causal reasoning + multimodal alignment | Llama 3.1 70B / Qwen2-72B fine-tuned · SigLIP/DINOv2 projector |
| Tier 3: Physics Engine | Hard constraints against physically impossible outputs | TAPS surrogates · CalculiX FEA · SymPy/SciPy solvers |
Node Types: Product · Component · Material · ManufacturingProcess · FailureMode · MediaAsset · Review · MarketSegment · Regulation · PhysicalProperty · BrepTopology
Edge Types: HAS_COMPONENT · MADE_OF · MANUFACTURED_BY · EXHIBITS_FAILURE · MENTIONED_IN · VISUALIZED_IN · COMPETES_WITH · CAUSES · REQUIRES · HAS_BREP
Example traversal — "Why does Apple Watch Series 10 overheat during GPS tracking?"
MATCH (p:Product {name: "Apple Watch S10"})
-[:EXHIBITS_FAILURE]-> (f:FailureMode {type: "ThermalThrottling"})
-[:CAUSES]-> (c:Component {name: "S10 SiP"})
<-[:REQUIRES]- (tim:Component {name: "GraphiteThermalPad"})
-[:VISUALIZED_IN]-> (m:MediaAsset)
-[:HAS_BREP]-> (b:BrepTopology)
WITH p, f, c, tim, m, b
MATCH (p)-[:COMPETES_WITH]->(rival:Product)
-[:HAS_COMPONENT]->(vc:Component {type: "VaporChamber"})
RETURN p, f, c, tim, m, b, rival, vcReturns a causal subgraph with B-rep geometry, video timestamps, bounding boxes, and review sentiment — enabling a fully evidenced, physics-validated answer.
📖 Full architecture documentation →
Four orthogonal dimensions of engineering intelligence extracted from public sources:
| Dimension | Contents | Key Extraction Methods |
|---|---|---|
| P-Data (Perceptual) | Video telemetry, acoustic signatures, spatial/photogrammetric data, thermal/NDE imagery | Grounded-VideoLLM, SAM 2, Whisper V3, YAMNet, COLMAP |
| E-Data (Engineering) | BOM hierarchy, B-rep topology, manufacturing process signatures, PCB components, thermal management, materials | EC-YOLO, CADFS, B-repLer, DyHead, PaddleOCR |
| M-Data (Market) | Pricing, competitive benchmarking, IP ratings, regulatory compliance, supply chain telemetry | E-commerce scrapers, OCR, regulatory DB |
| L-Data (Lifecycle) | Failure mode taxonomy, failure correlates, sentiment-to-subsystem mapping, repair economics | ABSA, iFixit, Reddit/Amazon reviews, temporal analysis |
All geometric extractions target editable, STEP-compatible parametric surfaces (B-rep), not fragile voxel or point-cloud approximations. Uses CADFS (generative CAD as a vision-language task) and B-repLer (language-guided CAD editing) to produce ISO 10303-compatible STEP files from teardown imagery.
Every extracted insight is a structured GroundingToken linking text span → video frame → bounding box → audio event:
{
"insight_id": "uuid",
"assertion": "retaining_clip_sheared",
"modal_evidence": {
"text": { "source": "youtube_transcript", "span": "00:04:23.200–00:04:25.800", "confidence": 0.94 },
"vision": { "source": "video_frame_14291", "bbox": [x1, y1, x2, y2], "mask_rle": "...", "confidence": 0.91 },
"audio": { "source": "spectrogram_event", "class": "plastic_fracture", "confidence": 0.87 }
},
"causal_inference": "shear_failure_due_to_cyclic_fatigue"
}Tensor-decomposition-based A Priori Surrogates (TAPS) coupled with CalculiX/FreeCAD FEA act as a deterministic filter on all generative outputs. If the LLM suggests a material or geometry that violates static equilibrium, material compatibility, or manufacturability — it is blocked before the user sees it.
Six specialized LangGraph agents (DataScout, PerceptionParser, SchemaEngineer, KnowledgeWeaver, ReasoningDev, SafetyGuard) orchestrated by a Meta-Architect agent. Workflows modeled as DAGs with durable execution, deterministic error handling, and human-in-the-loop escalation.
A transient processing clean-room: raw media → factual data extraction → discard. Only non-copyrightable facts (dimensions, temperatures, resistor values, material grades) persist in the EKG. Compliant with India Copyright Act 1957 Section 52 and DPDP Act 2023.
The first target domain — chosen for high teardown density, extreme integration complexity, and rich failure mode diversity.
Target products: Apple Watch Series 10, Garmin Fenix 8, Samsung Galaxy Watch 7, Google Pixel Watch 3
| Phase | Timeline | Goal |
|---|---|---|
| 1. Corpus Assembly | Weeks 1–4 | 500 teardowns · 50,000 reviews ingested |
| 2. Extraction Pipeline | Weeks 5–8 | Structured schemas + B-rep models for 100 products |
| 3. Knowledge Graph | Weeks 9–11 | Neo4j: 100 products · 3,000 components · 500 failure modes |
| 4. Agentic RAG | Weeks 12–14 | Functional Q&A benchmarked vs. human mechanical engineer |
| 5. Validation & Demo | Weeks 15–16 | Predict failure modes of newly launched watch; compare to actual teardown |
- v0.1 — Concept RFC and architecture specification (current)
- v0.2 — Gold dataset: 20 manually annotated smartwatch teardowns
- v0.3 — Extraction pipeline (vision + audio + NLP engines)
- v0.4 — EKG v1: Neo4j + Weaviate schema + 100 products
- v0.5 — Agentic RAG layer with LangGraph orchestration
- v0.6 — TAPS + FEA physics constraint engine
- v1.0 — Full smartwatch vertical demo + public benchmark
- v2.0 — Multi-vertical expansion (TWS earbuds, laptops, industrial sensors)
PTKE is explicitly designed for execution within Bengaluru's deep-tech infrastructure:
Sovereign Cloud:
| Provider | Role |
|---|---|
| Yotta Shakti Cloud | Primary training (H100/B200, InfiniBand, NM-ICPS aligned) |
| Neysa Velocis | Live agentic RAG inference (bare-metal, no noisy-neighbor) |
| E2E Networks | Burst capacity + DPDP Act 2023-compliant government workloads |
Physical Validation:
| Facility | Role |
|---|---|
| IKP EDEN, Koramangala | Primary teardown ground-truth facility; SmartFab manufacturing validation |
| CMRIT Makerspace | Automated turntable scanning for B-rep geometric ground truth |
| Workbench Projects | Community hardware diagnostics and secondary validation |
Talent Pipeline: IISc ARTPARK · CPDM · DESE
📖 Full ecosystem documentation →
PTKE is built on 2025–2026 SOTA model architectures:
| Component | Paper / System |
|---|---|
| Video grounding | Grounded-VideoLLM (dual-stream temporal expert) |
| PCB detection | EC-YOLO (improved YOLOv7) + Balanced-YOLOv3 + WIoU-Soft-NMS |
| B-rep reconstruction | CADFS · B-repLer · VideoCADFormer |
| Cross-modal grounding | Hierarchical Prototype Alignment |
| Physics surrogates | TAPS (Tensor-decomposition A Priori Surrogates) |
| Spatial attention | DyHead (dynamic head for dense detection) |
| Component segmentation | SAM 2.0 (Segment Anything Model) |
📖 Full research bibliography →
This repository is in RFC / Concept status. Contributions welcome:
- 🔬 Research: Propose better model architectures via GitHub Discussions
- 📐 Architecture: Open issues or RFCs to challenge or extend the design
- 🗄️ Ontology: Suggest extensions to the Product Modelling Ontology
- 🛠️ Prototypes: Build proof-of-concept implementations of individual modules
This system is designed from the ground up for India-sovereign legal compliance:
- India Copyright Act 1957, Section 52: Transient processing clean-room — only non-copyrightable factual data persists
- DPDP Act 2023: Data processed and stored exclusively on Indian sovereign cloud (Yotta/E2E)
- Apache 2.0 License: This specification is freely usable, modifiable, and distributable
📖 Legal compliance architecture →
| Category | INR | USD |
|---|---|---|
| Personnel (8.5 FTE × 4 months) | ₹1,06,00,000 | ~$127,000 |
| Infrastructure & Compute | ₹80,50,000 | ~$96,000 |
| Data, APIs & Legal | ₹16,80,000 | ~$20,000 |
| Physical Validation (Makerspaces) | ₹2,50,000 | ~$3,000 |
| Buffer (15%) | ₹30,90,000 | ~$37,000 |
| TOTAL | ₹2,36,70,000 | ~$283,000 |
Bengaluru ecosystem execution reduces effective cost by ~37% vs. US-equivalent resourcing ($450,000).
If you reference this architecture in research or derivative work, please cite:
@misc{ptke2026,
title = {Product Teardown Knowledge Engine (PTKE): Bengaluru-Anchored Multimodal Engineering Intelligence},
author = {Jagath Srujan},
year = {2026},
howpublished = {\url{https://github.com/jagathsrujan/product-teardown-knowledge-engine}},
note = {Apache 2.0 Open Architecture Specification}
}Apache 2.0 Licensed · Bengaluru, India · 2026
An open architecture for grounded physical-world AI reasoning.