Skip to content

Notes & Quick Ideas #12

@djdarcy

Description

@djdarcy

Notes & Quick Ideas

Quick notes, small bugs, and ideas that don't warrant their own issue yet.

Ideas

  • dazzlesum profile or dazzlesum preset command -- save named configs (exclusions, shadow dir, algorithm) for reuse across projects
  • library-sha-snapshot.py (currently in D:\M) could become a contrib example or built-in preset
  • dazzlesum diff command -- compare two .shasum files and show added/removed/changed files
  • dazzlesum watch -- filesystem watcher mode using inotify/ReadDirectoryChangesW for real-time change detection
  • Consider msgpack or cbor as alternative to SQLite for tree cache (smaller, faster serialize)

Quick bugs / observations

  • --force-python is ~500x slower than native tools (certutil/sha256sum) on large runs. Should come with a warning.
  • update_mode parameter is wired through CLI but completely unimplemented in process_directory_tree()
  • STATE_FILENAME (.dazzle-state.json) defined at line 82 but never read or written
  • .shasum files collect file mtime in memory (stat_info.st_mtime) but never persist it -- low-hanging fruit for change detection
  • Network share (UNC path) performance: iterdir() is ~10ms per SMB roundtrip. 180K dirs = 30+ minutes just for listing. DazzleTreeLib async could help.
  • Path.resolve() on Windows converts drive letters to UNC paths (e.g., D:\M -> \\aktuldjr\d\M). This doesn't affect I/O on subst drives but confuses users in output.

Notes from D:\M library pilot (2026-04-07)

  • D:\M: 180K dirs, 3.3M files, library distributed via Resilio Sync
  • Software section pilot (615 files, 18.1 GB) took ~4 minutes with --force-python
  • Full library counting phase took 8+ hours over network share
  • Running dazzlesum directly on host machine (local NTFS) is significantly faster
  • STRL exclusion during counting was the single biggest performance fix in this session
  • Everything (Voidtools) can provide instant change detection on local NTFS but not on network shares

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestpinnedPermanently open issue

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions