Conversation
I don't know if 2 Timelines will ever exist at once, but if so then this might be a problem. For now it seems to be an okay solution
This reverts commit d7ede60.
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
hayekr
requested changes
Jul 1, 2026
| Attributes: | ||
| owner_name: Node name for counter and fidelity metrics. | ||
| storage: In-memory store of recorded events for the trial. | ||
| delivery_owner: Node name for delivery-time metrics; defaults to ``owner_name``. |
Collaborator
There was a problem hiding this comment.
It currently defaults to None not owner_name
Collaborator
There was a problem hiding this comment.
also inline code entries are `` not ````
| delivery_owner: str | None = None | ||
| target_pairs: int = 500 | ||
| reservation_start_time: int | None = None | ||
| throughput: float | None = None |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR continues the work from #368/#374 by introducing a centralized metrics tracking system under sequence.utils.metrics, with a registry of metric types/event types, an in-memory storage backend, and protocol instrumentation to record EG/EP/ES/delivery events during simulations.
Changes:
- Adds a generalized metrics module (
sequence/utils/metrics/*) with event type registration, metric type abstractions, and a metric registry. - Instruments entanglement generation, purification, and swapping protocols to call
metrics.record(...)on success/failure. - Adds a comprehensive test suite covering recording, querying, collection, aggregation, and custom metric/event registration.
Reviewed changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/utils/test_metrics.py | Adds unit tests validating metrics recording, collection, aggregation, and extensibility. |
| sequence/utils/metrics/init.py | Implements public metrics API (enable/configure/record/collect/aggregate) and registers built-ins. |
| sequence/utils/metrics/event_types.py | Introduces EventType value object and event type registry + built-in EventTypes. |
| sequence/utils/metrics/metric_types.py | Adds metric type hierarchy (counters, rates, fidelity lists, delivery time) and CollectContext. |
| sequence/utils/metrics/registry.py | Provides metric registration/lookup/reset helpers and counter lookup table. |
| sequence/utils/metrics/storage.py | Adds in-memory record storage with basic query helpers. |
| sequence/utils/metrics/builtins.py | Defines and registers built-in EG/EP/ES counters and fidelity/delivery/throughput metrics. |
| sequence/utils/init.py | Exposes metrics from sequence.utils. |
| sequence/kernel/timeline.py | Registers the Timeline instance as the metrics time provider. |
| sequence/entanglement_management/generation/generation_base.py | Records EG failure events centrally. |
| sequence/entanglement_management/generation/single_heralded.py | Records EG success events. |
| sequence/entanglement_management/generation/barret_kok.py | Records EG success events. |
| sequence/entanglement_management/purification/bbpssw_circuit.py | Records EP success/failure events. |
| sequence/entanglement_management/purification/bbpssw_bds.py | Records EP success/failure events for BDS formalism. |
| sequence/entanglement_management/swapping/swapping_circuit.py | Records ES success/failure events for circuit-based swapping. |
| sequence/entanglement_management/swapping/swapping_bds.py | Records ES success/failure events for BDS swapping. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+101
to
+108
| storage.append( | ||
| { | ||
| "event_type": event_type, | ||
| "owner_name": owner_name, | ||
| "sim_time": time_provider.now(), | ||
| **record_kwargs, | ||
| } | ||
| ) |
| trial_values = trial[metric] | ||
| if list_metric_cap is not None: | ||
| trial_values = trial_values[:list_metric_cap] | ||
| all_values.extend(trial_values) |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Continuation of implementation of #368 and direct continuation of development after #374.
This PR implements a more generalized central metrics tracking system for SeQUeNCe. The module has been split up until multiple modules:
metrics/__init__.py: The front-facing API methods that are used to interact with the metrics module. Includesenable,record,configure,collect_trial_metrics, andaggregate_trial_metrics.event_types.py: Defines methods for creating and handling event types. Event types are what is used by therecordmethod to record a metric.builtins.py: Built-in event types and metrics. Used for abstraction and categorizing the different types of metrics used throughout SeQUeNCe. Also gives an example of how a user could create their own metrics for their own experiments!metrics.py: Mostly defines a Metric abstract class and the types of Metrics that are available. Each metric includescollectmethod that collects the information from those metrics, and this is typically used with thecollect_trial_metricsmethod.registry.py: Metric registry that provides convenient methods for registering and creating new metrics. All Metric types are stored as a list_metricsin this module.storage.py: Mostly unchanged from MVP implementation of centralized metrics tracking system #374, just all storage methods have been moved to their own module, along with the the time provider classes.Metric Superclass
Metrics have been generalized into certain metric types, that change the behavior of certain methods.
Metricis now an abstract superclass of all the other metric types, which are implemented as data classes.Abstract Properties
Metrics requires the abstract properties
event_typesandoutput_keysto be set:event_typesis an immutable set of the EventTypes that the metric reacts to duringrecord.output_keysis an immutable set of the keys that thecollect()method produces in it's output dictionary. Useful for aggregation of data.collect()is discussed in the next section.Collect Method
Metric also requires an abstract method in the form of
collect(). Thecollect()method is to generalize the globalcollect_trial_metrics()method, which now just calls thecollect()method of each metric and puts them all into one list.collect()is implemented differently and outputs different sets of information based on the metric.CounterPairMetriccollects the failures, successes, and success rate of the passed owner. This helps generalize the successes and failures that all the entanglement protocols produce.LastValueMetriccollects the last scalar field value from matching events. This helps generalize values like the throughput of an app.EventFieldListMetriccollects a list of field values matching a certain event type. This helps generalize giant lists of fidelities, like purified fidelities over time.DeliveryTimeMetriccollects the time to deliver N purified pairs relative to reservation start. This helps generalized values like purified delivery of the entanglement purification protocol (RequestApp, etc.).On Record Method
on_record()is an optional method that is called when the metric is recorded. It is only used byCounterPairMetric, where it updates the internal success and failure counters and updates the success rate.Collect Context
CollectContextis a simple data class that lays out a template context to pass to thecollect()method of each metric. It also tells thecollect()method what it can expect when it gets called. It is only used by thecollect_trial_metrics()method. This need some more work to be properly generalized or at least easily extended by the user.Collect Trial Metrics Method
The
collect_trial_metricsmethod is now more generalized for all types of metrics. All it does now is iterate through the list of metrics and calls thecollect()method of each metric. It uses the createdCollectContextobject with the passed arguments, to pass those arguments through to the metricscollect()methods. However, these arguments are currently specific to certain protocols and are not currently generalized.