Skip to content

Conversation

@AhmedNMahran
Copy link
Collaborator

renovate bot and others added 30 commits May 18, 2025 19:22
Update dependency gradle to v8.14.2
Update dependency androidx.compose:compose-bom to v2025.06.01
Update dependency com.vanniktech.maven.publish to v0.33.0
* Define the public Bookmark model

* Update gradle to handle tests

* Create BookmarksRepositoryImpl and BookmarkRepositoryTest

* Add to local mutations and read

* Merge the streams of persisted and mutated bookmarks

* Guard against duplicating bookmarks in mutated tables

* Check duplications in persisted bookmarks as well

* Delete bookmarks from mutated

* Delete persisted bookmarks

* Test adding a bookmark after it was deleted

* Implement public functions related to mutations in the sync-process

* Handle persistRemoteUpdates

* Handle migrateBookmarks

* Some cleanup

* Add a test that the steam captures the mutations as they happen

* Make sure that Bookmarks are initialized with valid data

* Add check constraints to the SQL tables

* Add some documentation

* Rename BookmarkRepository to BookmarksRepository

* Fix some failing tests regarding persist remote updates

* Ensure DB actions are handled on an IO context

* Split Bookmark into PageBookmark and ayahBookmark

* Add ayah and page bookmarks stream functions

* Extract synchronization functions to a new interface and a new model BookmarkMutation

* Prepare for refactoring remote ID

* Make remoteID internal in Bookmark

* A minor fix

* Remove the confusion on generated tables' names

* Add logging

* Add Xcode and macos's related ignorables to .gitignore

* Move some hardcoded settings from build.gradle.kts to libs.versions.toml

* Refactor the tests to delegate the DB conn to a platform-specific class

* Switch to a single-table structure

* Fix fetch mutations not retuning deleted bookmarks

* Add setToSyncedState instead of both clear mutations and persist remote updates

* Update the interface to remove ayah bookmarks from the interface

* Remove ayah and sura from the DB

* Remove ayah and sura from BookmarkMutation

* Rename types after page bookmarks

* Fix: adding a bookmark after deleting a remote one should reset delete flag

* Refactor deletions queries

* Refactor queries used for syncrhonization

* Enforce naming consistency for types

* Condense adding logic into a single SQL transaction

* Rename bookmarks table to page_bookmarks

* Remove existence checking done before deletion

* Add localID propert to PageBookmarkMutation

* Introduce the new sync function: accepts a list of processed local mutations

* Add documentation
* Add a github action

* Update yml

* Switch workflows to macos and run all tests
* Configure the gradle for the syncengine

* Define a first draft of the SynchronizationClient

* Create mutations-definitions module

* Use Local and Remote mutation types

* Add needed details to SynchronizationClient's interface

* Crreate SynchronizationClientImpl

* Create GetMutationsRequest and PostMutationsRequest

* Add ktor

* Fill in the main details of SynchronizationClientImpl

* Extract MutationsResponse to a separate file

* Fill in GetMutationsRequest's details

* Fill in the details of PostMutationsRequest

* Cteate HttpClientFactory

* Call API requests in SynchronizationClientImpl

* Provide a temporary SynchronizationClientIntegrationTest

* Add ketmit to syncengine

* Add exhastive loggings

* Log errors in mutations APIs requests

* Fix an issue in preparing local mutations to push

* Add a test for posting mutations to the integration test as well

* Ignore all SynchronizationClientIntegrationTest tests

* Refactor some stuff in the requests

* Deliver explicit failure responses

* Add HttpClientFactory.ios.kt
* Create PageBookmarksRepositoryFactory
* Refactor PageBookmarksRepository to use the mutations package instead of PageBookmarkMutation

* Create sync-pipelines package

* Create SyncEnginePipeline

* Provide main glue to SyncEnginePipeline

* Refactor persistence's PageBookmark to get a localID

* Deal with local ID conversion in SyncEnginePipeline

* Remove remoteID of persistence's PageBookmark (main model)

* Add syncPipelines to umbrella's gradle

* Read models timestamps in seconds not millisecond

* Delete records by remote IDs in synchronizing remote changes in PageBookmarksRepositoryImpl

* Handle DELETE events in PostMutationsRequest not having certain fields

* Conver MODIFY mutations into CREATE in syncengine

* Nullify model data in PostMutationsRequest for delete mutations

* Update sync-pipelines/src/commonMain/kotlin/com/quran/shared/pipeline/SyncEnginePipeline.kt

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update poms with proper names and descriptions

* Handle UPDATE mutations in PostMutationsRequest instead of SynchronizationClientImpl

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
* Create ConflictDetector and ModelConflict

* Detect one-to-one conflicts

* Detect lists of conflicting mutations

* Handle conflicts for delete events with zeroed-attrs model

* Create ConflictResolver

* Introduce a little DSL for scenarios checking in ConflictResolver

* Filter out remote delete events of resources that are not persisted locally

* Move UPDATE mutations conversions to SynchronizationClientImp

* Extract the main logic into PageBookmarksSynchronizationExecutor where SynchronizationClientImpl remains an integration point

* Provide a local mutations filtering pre-processing step

* Handle illogical scenarios in ConflictResolver

* Replace syncengine's PageBookmark to use Instant instead of a Long for dates

* Change persistence's PageBookmark to be Instant instead of Long for dates

* Rename ConflictGroup to ResourceConflict

* Remove room for unexpected behavior: inputting modified mutations

* Fix a bug in SynchronizationClientImpl: first auth headers in a session were cached

* Add a document SYNCENGINE_ARCHITECTURE

* Maintain relative order in preprocessors' output
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…6.x-compat (quran#27)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
* Create Scheduler

* Execute simple app_start trigger

* Reschedule regular tasks in Scheduler

* Add a local_data_modified trigger

* Cancel any ongoing job when triggers change

* Add a test for invoking local_data_modified after creation

* Track triggers internally

* Rejecct jobs if times exceed that scheduler currently.

* Add IMMEDIATE trigger

* Define failure-retries-related types

* Handle failure retries

* Convert times to be in milliseconds

* Some cleanup

* Reduce timings used in the test

* Cleanup tests

* Update DefaultTimings in Scheduler

* Refactor Scheduler to use exception-based error handling

- Change taskFunction from returning Result<Unit> to throwing exceptions
- Update reachedMaximumFailureRetries callback to receive Exception instead of Error
- Add comprehensive KDoc explaining the new exception-based error handling approach
- Wrap taskFunction execution in try-catch for proper exception handling
- Update all tests to use exception throwing instead of Result.failure()
- Preserve actual exception information through retry chain for better debugging
- Maintain cross-platform compatibility (verified on Android and iOS)

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* Implement comprehensive logging for Scheduler with appropriate levels

- Replace all println statements with Kermit logging framework
- Add DEBUG level logging for internal scheduler operations and state transitions
- Add INFO level logging for key operational events visible to SynchronizationAgentImpl:
  * Trigger applications (apply/stop scheduler)
  * Task completion success/failure outcomes
  * Default scheduling operations
  * Maximum retry exhaustion (critical failure state)
- Add ERROR level logging for task execution failures with exception details
- Use logger.withTag("Scheduler") for consistent log identification
- Maintain detailed context in all log messages for troubleshooting
- Enable proper production monitoring while preserving debug detail

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* Increase timeout in SchedulerTest to 30s

* Show more test output in workflows

* Increase times and latency allowance.

* Improve CI test reliability and output clarity

## GitHub Actions Enhancements:
- Add --stacktrace and --continue flags for better failure diagnostics
- Remove excessive --info logging to keep successful runs clean

## Gradle Test Configuration:
- Configure selective test logging: show details only for failed tests
- Enable verbose output when --info flag is explicitly used
- Preserve full exception details with stack traces for debugging

## Test Timing Improvements:
- Increase timing tolerance from 200ms to 250ms for GitHub Actions
- Reduce default timeout from 30s to 6s for faster feedback
- Keep 3x scaled intervals (600ms/900ms/300ms) for reliable CI execution

## Benefits:
- Cleaner CI logs for successful runs (easier to navigate)
- Detailed failure information when tests actually fail
- More reliable timing-sensitive tests in virtualized CI environment
- Full diagnostic capability available on-demand with --info flag

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* Implement sophisticated trigger buffering system for Scheduler

## Core Implementation:
- Add trigger buffering mechanism with `bufferedTrigger` state variable
- Implement state-based trigger routing in `apply()` method
- Add `buffer()` function for selective trigger handling during job execution
- Add `processSuccess()` to execute buffered triggers after successful completion

## Exceptional Scenario Handling:
- **APP_START during job execution**: Completely ignored, next job uses standard timing
- **LOCAL_DATA_MODIFIED during successful job**: Buffered and executed with LOCAL_DATA_MODIFIED timing
- **LOCAL_DATA_MODIFIED during failed job**: Ignored during retries, buffer cleared on failure

## State Machine Enhancements:
- `WaitingForReply`/`Replied` states: Route to buffering logic
- `Retrying` state: Complete trigger ignore to prevent interference
- `Initialized`/`RegularWait`/`Triggered` states: Direct trigger execution

## Buffer Management:
- Single buffer slot for efficiency (LOCAL_DATA_MODIFIED triggers only)
- Automatic buffer clearing in `processSuccess()` via `let` scope
- Buffer reset in `reportFailureAndSeize()` to ensure clean state

## Test Coverage:
- Add comprehensive tests for all three exceptional scenarios
- Add retry-specific variant: "LOCAL_DATA_MODIFIED during retry should be ignored"
- Enhanced timing validation and post-retry verification delays
- Updated timing constants for improved CI reliability

## Benefits:
- Intelligent trigger prioritization during active jobs
- Clean separation of success vs failure trigger handling
- Minimal memory overhead with sophisticated behavior
- Comprehensive debug logging for operational visibility

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* Fix bufferedTrigger not being cleared in some points

* Synchronize access to Scheduler's internal state

The comment about direct reads and indirect writes was removed
as it was no longer relevant after mutex synchronization implementation.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* Add some clarifying comments for critical sections

* Provide clear groupings for critical-section bound functions

* Clarify some identifiers

* Integrate the scheduler in SynchronizationClientImpl

* Preventing future changes if scheduler was stopped.

* Adding CLAUDE.md

* Tests cleanup

* Update timings in SchedulerTest

* Use Duration instead of Long for timings

* Cleanup

---------

Co-authored-by: Claude <[email protected]>
ahmedre and others added 28 commits December 30, 2025 02:17
Add methods for ayah bookmarks in persistence
The synchronization to the backend should be 2 api calls - one to get
the mutations, and one to persist changes. This should be global and not
per resource. This pr updates it to make it as such.
Organize and update sync call
The backend expects timestamps in milliseconds, so use millis
everywhere.
Use milliseconds instead of seconds
The backend returns a list of results the same size as the modifications
that are POST-ed and should be used in order to figure out the resource
ids correctly. This is mostly important for things like notes where the
text of two notes could be identical but the remote resource ids could
be different. While notes are not added yet, they will be added soon
in sha' Allah.
Respect the ordering of results from the backend
Implement collections persistence and sync
Today, when migrating bookmarks (or when the backend gives us a new
bookmark to persist), there is no local id, but in every other case,
there is always a local id. This patch splits out those models to make
things easier.
Separate input and general usage models
Implement sync of bookmarks within collections
Minor renaming for consistency
Fix deletion of notes, add indices to tables, and save a query before
adding a bookmark.
Minor performance improvements and fixes
Run fewer queries for collection bookmarks
…form-agnostic functions without expect/actual need
…UI .borderedProminent style in AuthView.swift
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants