Skip to content

L1 importance metadata never written during ingestion — wake-up ranking is effectively flat #2409

Description

@Swigler

Summary

Layer 1 (Essential Story) sorts drawers by importance metadata to build the wake-up context, but no ingestion path ever writes an importance value. Every drawer defaults to 3.0, collapsing the sort to filed_at order. This means frequently filed trivia can appear in wake-up context while rare-but-critical facts (e.g. health info, allergies, credentials) are pushed out.

Where the gap is

Reading side (works): layers.py L1 already reads importance, emotional_weight, or weight from metadata and sorts descending — this code is correct and ready.

Writing side (missing): None of these ingestion paths set importance:

  • miner.py — project file ingestion
  • convo_miner.py — conversation ingestion
  • mcp_server.py tool_add_drawer — MCP drawer creation

All build metadata dicts with wing, room, source_file, filed_at, etc. but never importance.

Impact

  • Wake-up context (L1) cannot prioritize critical memories over trivial ones
  • The AAAK dialect's ★ scale and semantic flags (ORIGIN, CORE, PIVOT, SENSITIVE) are stored but never influence retrieval ranking
  • Users have no way to mark a memory as critical via MCP tools

Proposed fix

1. Add a keyword-based importance scorer (zero dependencies, ~15 lines):

import re

CRITICAL_PATTERNS = re.compile(
    r"allerg|medical|medication|disease|emergency|password|"
    r"credential|api.key|ssn|birth.?date|blood.?type|"
    r"never.?forget|always.?remember|critical|life.?threatening",
    re.IGNORECASE
)
IDENTITY_PATTERNS = re.compile(
    r"my name is|i am a|my job|my role|i work at|"
    r"i live in|born in|native language|nationality",
    re.IGNORECASE
)

def score_importance(text: str) -> float:
    if CRITICAL_PATTERNS.search(text):
        return 5.0
    if IDENTITY_PATTERNS.search(text):
        return 4.0
    return 3.0

2. Write importance in all ingestion paths:

metadata["importance"] = score_importance(text)

3. Expose optional importance param in tool_add_drawer so MCP callers can override:

"importance": {"type": "number", "minimum": 1, "maximum": 5}
# In handler:
metadata["importance"] = params.get("importance") or score_importance(content)

Why this is safe

  • Old drawers without importance keep defaulting to 3.0 — no migration needed
  • No new dependencies
  • No schema changes
  • L1 retrieval code needs zero modifications
  • The scorer is conservative — only elevates obvious critical content, never demotes

Versions

Found on develop branch, v3.9.0.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2mediumarea/searchSearch and retrievalbugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions