[Bug] Janitor drops active staging partition causing location ingestion failure

[mdaugustine1]

Smartphone:

• Pixel 9 (Graphene OS 2026070500)
• Owntracks 2.5.9 (installed via F-Droid)

Environment:

• TrueNAS Scale 25.10
• Deployed using Portainer
• Reitti: dedicatedcode/reitti:5
• Tile Cache: dedicatedcode/reitti-tile-cache:5
• PostGIS: postgis/postgis:17-3.5-alpine
• Redis: redis:7-alpine
• Fresh installation on Sunday July 5.
• No upgrades performed after deployment

Problem:

The application worked normally for approximately two days after deployment.

On July 7, location ingestion stopped and the logs began filling with:

ERROR: no partition of relation "staging_location_points" found for row
Detail: Partition key of the failing row contains (partition_key) = (stream_1_1_20260707)

The application repeatedly logged:

Failed to flush batch for partition stream_1_1_20260707

Database investigation:

The only partition present was:

staged_stream_1_1_20260706

No partition existed for:

staged_stream_1_1_20260707

The application was attempting to insert rows with:

partition_key = stream_1_1_20260707

which caused PostgreSQL to reject inserts.

Significant log entry:

While reviewing logs, I found:

2026-07-07T04:00:00.099Z INFO
Janitor: Dropping fully promoted and inactive partition
[staged_stream_1_1_20260707]

Later, the application continued attempting to write to:

stream_1_1_20260707

and failed because the partition no longer existed.

Recovery

I manually recreated the partition:

CREATE TABLE staged_stream_1_1_20260707
PARTITION OF staging_location_points
FOR VALUES IN ('stream_1_1_20260707');

Immediately after creating the partition:

• Location ingestion resumed
• Errors stopped
• Live processing resumed
• New rows were successfully inserted

Verification:

SELECT COUNT(*)
FROM staged_stream_1_1_20260707;

returned rows successfully.

SELECT MAX(timestamp)
FROM staging_location_points;

showed current timestamps again.

Question

Is it expected for the Janitor job to drop:

staged_stream_1_1_20260707

at 04:00 UTC while the system is still actively ingesting data using:

stream_1_1_20260707

This appears to have caused the ingestion pipeline to fail until the partition was recreated manually.

[dgraf-gh]

Hi @mdaugustine1,

Thank you for the detailed report.

There are two points:

• it should not drop the current partition
• even if it is dropped, ingestion should happily recreate that. Could be some cache in validation missing.

I will fix it and create a new release

[dgraf-gh]

What app are you using to get the data? Maybe one endpoint is missing something. I have this running for one week and never encountered that error

[mdaugustine1]

I recently edited my post to include phone and apl information.

It is owntracks 2.5.9 (installed via F-Droid) and it is connecting via HTTP to this endpoint:

"https://location.mydomain.com/api/v1/ingest/owntracks?token=MY_API_KEY"

[dgraf-gh]

Thanks. Thank you for the additional information. I also have owntracks setup and it is chucking along besides my main gpslogger setup.

I will find out first why it has decided to drop that partition and then tackle the rest.

[dgraf-gh]

I think I found the issue why the table got dropped. The code tries to determine if a partition is active by checking when the last analyze or vacuum was run on that, falls back to a year when that never run and drops all which are older than 12h. Since you are on a fresh install, this propably runs into the fallback and it drops it :(

That is also the reason I did not experienced this since I am running on an established database.

I will rewrite this whole check, it does not really make sense to do it that way.

[dgraf-gh]

And as expected, the second problem is that in LocationBatchingService we check:

            if (!initializedPartitions.contains(pKey)) {
                locationPointStagingService.ensurePartitionExists(pKey);
                initializedPartitions.add(pKey);
            }

And the initializedPartitions list never gets cleared when the partition is actually dropped.

[mdaugustine1]

I had the same issue again today. I wanted to give it a couple more days.

It's easy for me to see when ingestion stops because when I open Reitti, it shows me in the ocean off the coast of Africa.

Again, i manually created the partition with:

CREATE TABLE public.staged_stream_1_1_20260711
PARTITION OF public.staging_location_points
FOR VALUES IN ('stream_1_1_20260711');

Ingestion resumed with no problems.

For now I made a default partition just to get data into the database with no errors, obviously it is just there, not sorted:
CREATE TABLE public.staged_stream_default PARTITION OF public.staging_location_points DEFAULT;

[dgraf-gh]

Did you pull the latest 5.0.2? You can verify this by either going to "Settings > About" or check if the table partition_registry exists.

The code works now like this:

• create the partition
• store it with the creation date in the partition_registry
• nighly check if something there was created 7 days ago and drop it.

At the moment I have no idea why this should be dropped then. Or it was never created, but then it would have been broken before.

Anything of maybe interest in the log?

[mdaugustine1]

Application Version: v5.0.2

Commit Details: 98e9557 (2026-07-08T07:17:47Z)

Build Time: 2026-07-08T07:21:14Z

Unfortunately I cannot pull anything of interest from any logs, I did a force recreate on the container. I ended up adding an environment variable to solve another issue. I did not even consider that this would wipe the logs.

I would like to see what happens in two days from now, as that seems to be the timing window when this is occurring. I can get you something concrete at that time.

[dgraf-gh]

I totally missed your comment.

Thank you for helping me here. Lets see what happens :)

[mdaugustine1]

Follow-up on partition issue (v5.0.2)

I waited approximately two days, which was previously the window where ingestion failures would occur.

Current status:

• Reitti version: v5.0.2
• Ingestion is still functioning normally.
• Daily staging partitions continue to be created.
• partition_registry appears healthy.

Current partition_registry contents:

stream_1_1_20260706
stream_1_1_20260707
stream_1_1_20260708
stream_1_1_20260709
stream_1_1_20260710
stream_1_1_20260711
stream_1_1_20260712
stream_1_1_20260713

As a safety net after the second failure, I created a default partition:

CREATE TABLE public.staged_stream_default
PARTITION OF public.staging_location_points
DEFAULT;

To verify whether the previous partition-drop problem was still occurring silently, I checked:

SELECT COUNT(*) FROM staged_stream_default;

Result:

0

This means no rows have been routed to the default partition and every location point has been written into a matching daily partition.

Based on that, I cannot currently reproduce the original ingestion failure. The partition_registry is populated, daily partitions exist, ingestion is working, and the default partition has not been used.

While investigating, I did discover what may be a separate issue related to visit processing:

The logs changed from:

1 visits → 1 processed visits → 0 trips

to:

2 visits → 1 processed visits → 0 trips

after new location data arrived at an existing significant place ("Thermal Piping").

The detector appears to recognize two visits, but only one processed visit survives, resulting in no trip generation.

The relevant observation is:

2 visits → 1 processed visits → 0 trips

This may indicate a problem somewhere between visit detection and processed visit generation rather than an ingestion or partition issue.