Skip to content

Preserve metadata for multi timestamp records#219

Open
Jacob-gr wants to merge 1 commit intofox-it:mainfrom
Jacob-gr:fix-multi-timestamp-metadata
Open

Preserve metadata for multi timestamp records#219
Jacob-gr wants to merge 1 commit intofox-it:mainfrom
Jacob-gr:fix-multi-timestamp-metadata

Conversation

@Jacob-gr
Copy link
Copy Markdown

Preserve metadata for multi timestamp records

Fixes metadata fields (_source, _classification, _generated) being set to null
when iter_timestamped_records() expands a record into multi-timestamped records
by explicitly copying metadata fields to new TimestampRecord object.

Also fixes an issue with shadowed record variable in loop: use a separate result variable
instead of reassigning record in the loop, which previously caused subsequent iterations
to extend an already-extended record.

Includes a test to validate metadata values are retained. Due to using a new variable in the loop,
a change to test_multi_timestamp_descriptor_cache() is required to adjust the expected cache misses
and hits. Because we are not mutating record each loop, we should only expect 1 cache miss.

fixes #218

@Jacob-gr
Copy link
Copy Markdown
Author

I'm new to contributing, but wanted to give it a shot.

I tested this proposed code by creating a venv for both the main dissect and this change using. From withing the flow.record local repo:

  1. Created a dissect venv and installed the primary bundle

    uv venv .dissect
    source .dissect/bin/activate
    uv pip install dissect
    deactivate
  2. Created a rdump vevn and installed the modified version

    uv venv .rdump
    source .rdump/bin/activate
    uv pip install -e .
    deactivate
  3. Call vanilla target-query and use modified rdump with:

    .dissect/bin/target-query ~/test-images/owl/evidence/HD1.E01 -f amcache.applications | .rdump/bin/rdump --multi-timestamp --record-classification TEST -w jsonfile://test-ammcache-applications.ndjson

Viewing the output, I see the metadata fields populated. Running tox it appears linting and testes passed.

I'm not sure if there is a better way to test, so please let me know. I'm happy to learn.

In addition, I'm not sure if there are other metadata fields that should be included. It appeard _type, _version, and _recorddescriptor were all merged over correctly into the created TimestampRecord

@Jacob-gr Jacob-gr marked this pull request as ready for review March 28, 2026 20:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Using multi-timestamp causes loss of _source and other metadata fields

1 participant