Skip to content

Conversation

mlincett
Copy link
Collaborator

@mlincett mlincett commented Jun 17, 2025

In order to be able to load the background from a different analysis, the pickle directory structure can no longer being semi-statically defined at the ResultsHandler level, but is defined, starting from a base path, by means of a utility class PickleCache. Likewise, the entire managing logic (read/merge/write/clean) of the pickle cache is now encoded there.

A background_source parameter can be passed to the ResultsHandler constructor, so background data can be additionally loaded (and merged).

The solution is still not the cleanest, but works. Still at RFC stage.

@mlincett mlincett marked this pull request as ready for review July 2, 2025 08:35
@mlincett mlincett requested a review from Copilot July 2, 2025 08:35
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors the results‐handling logic by introducing a dedicated PickleCache class to manage pickle I/O and adds an optional background_source parameter to allow separate loading and merging of background trials.

  • Introduced PickleCache to encapsulate pickle directory (read/merge/clean) logic.
  • Added background_source to ResultsHandler to load and merge background data separately.
  • Refactored ResultsHandler to use PickleCache and removed the legacy merge_pickle_data method.

class PickleCache:
def __init__(self, pickle_path: Path, background_only: bool = False):
self.path = pickle_path
# self.pickle_path.mkdir(parents=True, exist_ok=True)
Copy link
Preview

Copilot AI Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Remove or implement the commented-out directory creation logic in PickleCache.__init__ to avoid dead code.

Suggested change
# self.pickle_path.mkdir(parents=True, exist_ok=True)
self.path.mkdir(parents=True, exist_ok=True)

Copilot uses AI. Check for mistakes.


if scale_label == background_label and background_label in output_dict:
logger.info(
f"Appending f{n_pending} background data to {len(output_dict[background_label]['TS'])} existing trials ({scale_label=})"
Copy link
Preview

Copilot AI Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's an extra 'f' before {n_pending} causing the log to print 'f'; remove the 'f' so it reads {n_pending}.

Suggested change
f"Appending f{n_pending} background data to {len(output_dict[background_label]['TS'])} existing trials ({scale_label=})"
f"Appending {n_pending} background data to {len(output_dict[background_label]['TS'])} existing trials ({scale_label=})"

Copilot uses AI. Check for mistakes.

self.pickle_cache.merge_and_load(output_dict=self.results)
# Load the background trials. Will override the existing one.
if self.pickle_cache_bg is not None:
print("NOTE!!!! Loading BG")
Copy link
Preview

Copilot AI Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace this print statement with a logger call (e.g., logger.info) to maintain consistent logging practices.

Suggested change
print("NOTE!!!! Loading BG")
logger.info("NOTE!!!! Loading BG")

Copilot uses AI. Check for mistakes.

print("NOTE!!!! Loading BG")
self.pickle_cache_bg.merge_and_load(output_dict=self.results)
else:
print("NOTE!!!! No BG pickle cache")
Copy link
Preview

Copilot AI Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace this print statement with a logger call (e.g., logger.warning) for consistency and better control over output.

Suggested change
print("NOTE!!!! No BG pickle cache")
logger.warning("NOTE!!!! No BG pickle cache")

Copilot uses AI. Check for mistakes.

Comment on lines +262 to +268
sys.exit()
if not scale_shortener(0.0) in self.results:
logger.error(
f"No key equal to '0' in results! Keys are {self.results.keys()}"
)

sys.exit()
Copy link
Preview

Copilot AI Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid calling sys.exit during initialization; consider raising an exception so callers can handle errors more gracefully.

Suggested change
sys.exit()
if not scale_shortener(0.0) in self.results:
logger.error(
f"No key equal to '0' in results! Keys are {self.results.keys()}"
)
sys.exit()
raise RuntimeError("No data was found by ResultsHandler object!")
if not scale_shortener(0.0) in self.results:
logger.error(
f"No key equal to '0' in results! Keys are {self.results.keys()}"
)
raise KeyError(f"No key equal to '0' in results! Keys are {self.results.keys()}")

Copilot uses AI. Check for mistakes.

Comment on lines +262 to +268
sys.exit()
if not scale_shortener(0.0) in self.results:
logger.error(
f"No key equal to '0' in results! Keys are {self.results.keys()}"
)

sys.exit()
Copy link
Preview

Copilot AI Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid calling sys.exit during initialization; consider raising an exception so callers can handle errors more gracefully.

Suggested change
sys.exit()
if not scale_shortener(0.0) in self.results:
logger.error(
f"No key equal to '0' in results! Keys are {self.results.keys()}"
)
sys.exit()
raise RuntimeError("No data was found by ResultsHandler object!")
if not scale_shortener(0.0) in self.results:
logger.error(
f"No key equal to '0' in results! Keys are {self.results.keys()}"
)
raise RuntimeError("No key equal to '0' in results!")

Copilot uses AI. Check for mistakes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant