Inner Circle Squared

A Compose Multiplatform (Android + iOS + wasmJs) app for streaming live and recorded HLS event streams, built on Media3/ExoPlayer (Android) and AVPlayer (iOS) with a standalone KMP playback SDK (:mediakit).

Live on Google Play: com.livingpresence.inner.circle.squared

It plays live/recorded HLS event streams from a Wowza nDVR server and turns four unadvertised sibling renditions into a genuine client-side ABR ladder.

📐 The full design — ground-truth server measurements and the rationale behind each decision — lives in plan.md.

Features

• Adaptive streaming. Genuine client-side ABR synthesized from four unadvertised sibling renditions, with viewport-aware track selection so the chosen quality matches the surface size.
• Live & recorded playback. Live-vs-VOD is inferred from playlist inspection; live events expose a LIVE badge and jump-to-live, and the seek bar tracks a growing Wowza nDVR window without drift.
• Thumbnail feed with scrub preview. Poster tiles and a scrub-preview bubble are powered by an ExoPlayer frame engine on Android, and native AVAssetImageGenerator on iOS, avoiding the overhead of N full per-tile players.
• Flexible presentation. fit/fill/zoom resize matrix for both horizontal and vertical (9:16) video, rotate-to-fullscreen, auto-hiding controls, buffering/error UX, and a stats overlay + quality menu.
• Background & Picture-in-Picture. Playback is owned by a MediaSession service (surviving config changes), with PiP aspect-clamped for vertical video and background audio constrained to the low-bitrate audio-only tier.
• Offline downloads. Bounded (VOD) events download via WorkManager (Android) and AVAssetDownloadURLSession (iOS) into a cache shared with playback; truly-live events get no download affordance.
• Cross-platform Parity. A seamless unified experience across Android, iOS, and Wasm. The app features native in-app Web navigation, unified UI aesthetics across all targets, hardware-accelerated thumbnail extraction on iOS via AVAssetImageGenerator, and robust native iOS background HLS downloading using AVAssetDownloadURLSession.

Feature Matrix

Feature	Android	iOS	Web (Wasm)
Live STT Captions (Deepgram/Soniox WebSockets)	✅	✅	✅
ABR Ladder Synthesis (from sibling renditions)	✅	✅	✅
Viewport-Aware ABR (auto-caps to screen size)	✅	✅	✅
Manual Quality Override (Auto, 720p, audio, etc)	✅	✅	✅
Scrub Frame Previews (floating thumbnail)	✅	✅	✅
Live Event DVR (drifting seekbar & jump-to-live)	✅	✅	✅
Pillarboxing & Orientation Handling	✅	✅	✅
Background Audio (auto-shifts to audio-only tier)	✅	✅	✅
Picture-in-Picture (PiP)	✅	✅	✅
Offline HLS Downloads	✅	✅	❌
Preview Disk Caching (persisted between sessions)	✅	✅	❌
Memory Governor (OOM prevention during PiP/bg)	✅	❌	❌

Caption provider evaluation

The live-caption provider choice is backed by a reproducible, record/replay eval harness in eval/ that scores Deepgram and Soniox against a domain-specific golden set of event audio — batch WER/CER, domain-term recall (Entity F1), keyterm-boosting impact, and real-time streaming realism (flicker + finalization latency). On this material Soniox clearly outperforms Deepgram (0.242 vs 0.350 normalized WER; 0.77 vs 0.49 Entity F1), which is why the app ships Soniox as its preferred provider. See the generated scorecard and eval/README.md.

Architecture

:composeApp        — app UI (feed, login, player), navigation, DI wiring
:mediakit          — KMP playback SDK
  commonMain       — PlaylistInspector (pure-Kotlin HLS parser), LadderSynthesizer
                     (multivariant playlist builder), LadderResolver (JIT rendition
                     probing), EventCatalog (parallel event probing), MediaKitConfig,
                     EventInfo/ProbedRendition/RenditionTier models
  androidMain      — ExoPlayer (in PlaybackService), PreviewFrameEngine,
                     DownloadCenter, MemoryGovernor, LadderMediaSourceBuilder
  iosMain          — AVPlayer playback (UIKitView + AVPlayerLayer cinterop),
                     PiP, background audio, native offline downloads
  wasmJsMain       — thin web target (poster tiles + in-app player screen)

The playback engine lives in :mediakit, owned by a MediaSessionService, not by a composable — it survives config changes and enables background audio/PiP.

• Shared UI and app state live in composeApp/src/commonMain
• Shared image resources live in composeApp/src/commonMain/composeResources
• Android entry points and playback integration live in composeApp/src/androidMain
• iOS entry points and AVPlayer integration live in composeApp/src/iosMain
• Web entry points live in composeApp/src/wasmJsMain

Engineering decisions

The load-bearing design calls and the reasoning behind them live in plan.md.

TODO: deferred from the AGP 9 migration

The supported Gradle 9/AGP 9 migration is planned in plan.md under “Standalone PR plan — Gradle 9 and AGP 9 migration.” The migration PR intentionally leaves these follow-ups out:

• Rename :composeApp to :shared and extract a separate :webApp (JetBrains' recommended full platform restructure). The minimum supported split keeps iOS and Wasm in :composeApp.
• Revisit the ordinary dependency upgrades grouped into Dependabot PR #33. AndroidX, Media3, Ktor, coroutines, Metro, and other runtime updates should be reviewed in smaller dependency-only PRs.
• Extract repeated build configuration into convention plugins after the migrated module boundaries have stabilized.
• Refresh checked-in or developer-local Android Studio run configurations to target :androidApp; this is IDE metadata, not part of the Gradle migration itself.
• Consider a later full AGP-defaults/R8 audit with release smoke testing. The migration adopts defaults needed for AGP 9, but does not redesign keep rules, shrinking policy, packaging, or release delivery.

Building

Android

./gradlew assembleDebug            # debug APK → composeApp/build/outputs/apk/debug/
./gradlew installDebug             # install on a connected device

iOS

./gradlew :composeApp:linkDebugFrameworkIosSimulatorArm64   # Compose framework incl. the AVPlayer cinterop

A runnable SwiftUI host app lives in iosApp/ (generated from project.yml via xcodegen):

cd iosApp && xcodegen generate
xcodebuild -project ICSApp.xcodeproj -scheme ICSApp \
    -sdk iphonesimulator -destination 'platform=iOS Simulator,name=iPhone 17' build

cinterop note. Kotlin 2.3.x's cinterop parses the def's sources = AVPlayerBridge.m but does not compile the Obj-C implementation into the framework, so _OBJC_CLASS_$_AVPlayerBridge stays undefined at link time. The Gradle build works around this by compiling AVPlayerBridge.m to LLVM bitcode (.bc) and injecting it into the cinterop klib's natives/ directory, where the Kotlin/Native linker picks it up. No manual steps needed — linkDebugFrameworkIosSimulatorArm64 handles it end-to-end.

Web (Wasm)

./gradlew wasmJsBrowserDevelopmentRun

To serve the production build from a static server:

./gradlew wasmJsBrowserDistribution
# serve composeApp/build/dist/wasmJs/productionExecutable/

Tests & SDK discipline

./gradlew :mediakit:test                # the pure-Kotlin SDK core (JVM)
./gradlew :composeApp:testDebugUnitTest # Robolectric unit tests (resize matrix,
                                        # MemoryGovernor tiers, MainViewModel)
./gradlew :mediakit:apiCheck            # binary-compatibility validation
./gradlew :mediakit:dokkaHtml           # API docs → mediakit/build/dokka/html
./gradlew koverXmlReport                # coverage report

The :mediakit public API surface is tracked under mediakit/api/ — accidental binary-breaking changes fail CI. Dokka API docs are published to GitHub Pages on main.

Configuration

Transcription API keys (for Deepgram and Soniox) are read from secrets.properties in the project root. See secrets.properties.example for details.

Manual Test Scenarios

If you are verifying feature parity or evaluating a PR, run through these core scenarios:

1. Live Captions: Enable CC from the player overlay and verify real-time text flows in seamlessly across platforms.
2. ABR Ladder & Engine Playback:
   • ABR Synthesis: The AI synthesized a ladder client-side from 4 sibling renditions. Use a network throttler to ensure the player gracefully auto-switches between 720p, 360p, 160p, and audio-only tiers. Check the debug stats overlay to confirm.
   • Viewport-Aware ABR (FU-5): Verify that playing on a small device doesn't over-download the 720p stream if 360p fits the physical display perfectly.
   • Quality Menu: Test manual override controls (Auto, 720p, 360p, 160p, Audio) to make sure they stick.
   • Orientation & Layout: Test a 9:16 vertical video. Ensure it pillarboxes correctly in portrait mode rather than cropping awkwardly. Verify that sensor-based rotate-to-fullscreen works for landscape videos.
3. Scrubbing & Frame Previews:
   • Scrub Previews: Drag the seekbar and confirm a floating thumbnail bubble appears. The PreviewFrameEngine relies on a secondary muted player hitting the _160p stream. Ensure ~2s granularity.
   • Disk Caching (FU-2): Scrub a video, force close the app completely, reopen, and scrub again. It should be instant because of the disk cache.
   • Graceful Fallback: Scrub rapidly. If frames aren't decoded fast enough, verify it falls back to a time-only bubble.
4. Background, PiP, & Memory Governor (Mobile Only):
   • Background Audio: Start playback, and put the app in the background (no PiP). Ensure playback shifts down to the audio-only tier (~51kbps) to save data, and continues with a MediaSession notification.
   • Picture-in-Picture (PiP): Enable PiP, minimize the app. Verify the PiP window shows with the correct aspect ratio (especially crucial for vertical videos) and player controls collapse properly.
   • Memory Governor: Simulate Android memory pressure (adb shell am send-trim-memory <pid> TRIM_MEMORY_CRITICAL). Ensure background resources like the PreviewFrameEngine and thumbnail LRU are immediately released and the player doesn't OOM.
5. Live Event Verification: Verify the jump-to-live button works, seeking is restricted to the DVR window, and that the UI seek slider doesn't infinitely "drift" forward.
6. Offline Downloads: Turn off all networking. Start the app and play a downloaded event to verify it properly resolves via the local cache.