Skip to content

Memory optimizations #160

@psathyrella

Description

@psathyrella
  • Throw out cached values once you've merged them into irrelevance
    • somewhat nontrivial, since you have to also remove it from all the other processes' caches
  • don't give the whole cache to every process
    • probably not worthwhile, it'd introduce all sorts of shenanigans to work out who needs to go where, and then the final process still has to use just as much memory.
  • reduce the number of things the master python process keeps track of, or caching more of them on disc (although the latter isn't necessarily a great option, because we already spend a significant fraction of our time reading output files).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions