Skip to content

refactor: replace WeakMap with Grafast meta system + adopt Benjie's plugin patterns#776

Merged
pyramation merged 5 commits intomainfrom
devin/1772607889-meta-system-rewrite
Mar 4, 2026
Merged

refactor: replace WeakMap with Grafast meta system + adopt Benjie's plugin patterns#776
pyramation merged 5 commits intomainfrom
devin/1772607889-meta-system-rewrite

Conversation

@pyramation
Copy link
Contributor

@pyramation pyramation commented Mar 4, 2026

refactor: replace WeakMap with Grafast meta system + adopt Benjie's plugin patterns

Summary

Replaces the module-level WeakMap pattern with PostGraphile's built-in setMeta/getMeta system for passing computed data (rank/distance/score indices) between the condition apply phase and the output field plan. This follows the pattern recommended by Benjie (Graphile maintainer) in his postgraphile-plugin-fulltext-filter reference implementation.

All three search plugins receive identical changes:

  • graphile-search-plugin (tsvector full-text search)
  • graphile-pgvector-plugin (pgvector similarity search)
  • graphile-pg-textsearch-plugin (BM25 ranked search)

What changed internally:

  • Removed module-level WeakMap + slot interfaces
  • Removed getPgSelectStep() duck-typing helper
  • Added getQueryBuilder(build, $condition) which walks PgCondition.parent chain (via build.dataplanPg.PgCondition instanceof)
  • Condition apply now calls qb.setMeta(key, { selectIndex }) instead of storing in WeakMap
  • Output field plan now uses $select.getMeta(key) (returns a Grafast Step) with lambda([$details, $row], ...) instead of closure-captured WeakMap lookup
  • Replaced $condition.dangerouslyGetParent() with getQueryBuilder() for a more stable traversal pattern

No API changes intended — GraphQL schema (field names, input types, orderBy enums) should remain identical, but see review checklist item about inflection methods.

Benjie's Plugin Pattern Improvements

Six improvements from Benjie's reference implementation, applied across all three plugins:

  1. TypeScript namespace augmentationsdeclare global blocks for GraphileBuild.Inflection, BehaviorStrings, ScopeObjectFieldsField, and GraphileConfig.Plugins interfaces
  2. Custom inflection methods — Named, overridable methods for field/enum naming (e.g. pgTsvRank, pgVectorDistance, pgBm25Score, pgTsvOrderByColumnRankEnum, etc.)
  3. Behavior registry + entity behavior — Declarative opt-in/opt-out system via smart tags (e.g. @behavior -attributeFtsRank:select to hide rank field for a specific column). By default everything is auto-exposed (no behavior change for existing schemas).
  4. qb.mode === 'normal' check — Prevents injecting SELECT expressions into aggregate queries. Applied only to the selectAndReturnIndex / setMeta block (pgvector + BM25 plugins); intentionally not applied to the ORDER BY meta-read block.
  5. TYPES.float.fromPg() instead of parseFloat() — Proper codec conversion instead of JS-level parsing
  6. Computed column (proc) support (tsvector only) — Auto-discovers PostgreSQL functions returning tsvector whose first parameter matches the table codec, adds rank fields for them

Updates since last revision

Fix: orderBy enum timing mismatch (graphile-search-plugin)

Root cause identified and fixed for the "sort by full text rank orderBy enums works" test failure.

The problem: The orderBy enum apply runs at planning time (receives PgSelectStep), while the condition apply runs at execution time (receives a runtime proxy). These two objects have separate meta stores, so they cannot share data via setMeta/getMetaRaw directly between each other.

The solution: Leverage the fact that PgSelectStep._meta gets copied to the proxy's meta closure during execution. Flow:

  1. Enum apply (planning time, PgSelectStep): stores direction flag in PgSelectStep._meta via setMeta
  2. PgSelectStep._meta gets copied to proxy.meta via getStaticInfobuildTheQueryCore
  3. Condition apply (execution time, proxy): reads direction flag from proxy.getMetaRaw, adds ORDER BY with scoreFragment

This fixes the test — ASC and DESC now produce different result orderings. Also removes the stale _ftsRankBridge WeakMap that was attempted as an intermediate fix but didn't work.

Note: pgvector and BM25 plugins already had the correct pattern and did not need this fix.

Entity behavior phase change

Changed entityBehavior callbacks from inferred phase to override phase with after: ['inferred'], before: ['override'] ordering. This ensures the behavior additions (e.g. 'attributeFtsRank:select') are prepended to the behavior array before any user-specified overrides, allowing users to negate them with smart tags.

Codec identity check improvement

Replaced direct codec identity comparisons (e.g. attr.codec === build.dataplanPg.TYPES.tsvector) with isTsvectorCodec() helper that checks codec.extensions.pg.name === 'tsvector'. More robust across PostGraphile RC versions.

Review & Testing Checklist for Human

This PR cannot be fully tested locally (requires PostgreSQL + pgvector/pg_textsearch extensions in CI). Priority items for review:

  • CRITICAL: Verify inflection methods don't change GraphQL schema — The new custom inflection methods (e.g. pgTsvOrderByColumnRankEnum) call this._attributeName({codec, attributeName, skipRowId: true}) instead of using raw attributeName. If _attributeName produces different output than the raw attribute name, orderBy enum names will change, breaking clients. Compare a GraphQL introspection query before/after this PR to confirm field/enum names are identical.
  • CRITICAL: Verify CI passes for all 3 plugins — Integration tests are the only verification that the meta system + behavior registry + orderBy timing work correctly end-to-end. The filter.test.ts failure was caught by CI and fixed; ensure no other edge cases exist.
  • CRITICAL: Verify orderBy enum timing assumption — The fix relies on PgSelectStep._meta being copied to the execution-time proxy's meta closure. This is confirmed by reading PostGraphile source (lines 1322 + 1785 of pgSelect.js) but could break if PostGraphile internals change. If orderBy enums stop working in future PostGraphile versions, this is the first place to check.
  • Check entity behavior phase ordering — The behavior callbacks now use override phase with after: ['inferred']. If other plugins also use override phase, the ordering might cause behaviors to be evaluated in unexpected order. Verify rank/distance/score fields still appear on expected columns.
  • Verify computed column discovery (tsvector only) — If your schema has PostgreSQL functions returning tsvector (e.g. CREATE FUNCTION article_combined_search(article) RETURNS tsvector ...), verify that rank fields appear for them and work correctly.

Notes

  • Session: https://app.devin.ai/sessions/7f9b6b69895d49b7b3f7fddf52f5d4f6
  • Requested by: @pyramation
  • The @pgsql/quotes TypeScript error in BM25 plugin is pre-existing on main (not caused by this PR) — BM25 plugin cannot be built locally but CI has the necessary dependencies
  • getQueryBuilder() is duplicated across all 3 files (not extracted to shared utility) — could be refactored in a follow-up
  • The behavior registry is strictly additive — users can opt out per-column, but by default everything is auto-exposed (no breaking changes)
  • Computed column support only applies to tsvector plugin (not pgvector/BM25) since vector/score-returning functions are less common

…in all search plugins

Replaces the module-level WeakMap pattern with PostGraphile's built-in
meta system for passing data between the condition apply phase and
output field plan, following the pattern from Benjie's
postgraphile-plugin-fulltext-filter reference implementation.

Changes across all three plugins (graphile-search-plugin,
graphile-pgvector-plugin, graphile-pg-textsearch-plugin):

- Remove module-level WeakMap and slot interfaces
- Replace dangerouslyGetParent() with getQueryBuilder() traversal
  (walks PgCondition .parent chain to find PgSelectQueryBuilder)
- Condition apply now calls qb.setMeta(key, { selectIndex }) instead
  of storing in WeakMap
- Output field plan now calls $select.getMeta(key) which returns a
  proper Grafast Step, used with lambda([$details, $row], ...)
- Remove getPgSelectStep helper, use $row.getClassStep() directly

GraphQL interfaces remain identical - no API changes for consumers.
@devin-ai-integration
Copy link
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

- TypeScript namespace augmentations (Inflection, BehaviorStrings, ScopeObjectFieldsField, Plugins)
- Custom inflection methods (pgTsvRank, pgVectorDistance, pgBm25Score, etc.)
- Behavior registry + entity behavior for per-column opt-out via smart tags
- qb.mode === 'normal' check to prevent injection into aggregate queries
- TYPES.float.fromPg() instead of parseFloat() for proper codec conversion
- Computed column (proc) support for tsvector plugin
- Duck-typing enhancement for query builder detection
@devin-ai-integration devin-ai-integration bot changed the title refactor: replace WeakMap with Grafast meta system (setMeta/getMeta) in all search plugins refactor: replace WeakMap with Grafast meta system + adopt Benjie's plugin patterns Mar 4, 2026
- TypeScript namespace augmentations (declare global)
- Custom inflection methods (overridable naming)
- Behavior registry with entityBehavior callbacks
- TYPES.float.fromPg() instead of parseFloat()
- Computed column (proc) support for tsvector
- Remove extra where check from getQueryBuilder duck-typing
- Keep simple constantCase enum naming for direct columns to prevent mismatch
… condition reads it at execution time

The orderBy enum apply runs at PLANNING time (receives PgSelectStep) while
condition apply runs at EXECUTION time (receives runtime proxy). The proxy's
meta is initialized from PgSelectStep._meta, so data stored at planning time
IS available at execution time.

Flow:
1. Enum apply (planning): stores direction flag in PgSelectStep._meta
2. PgSelectStep._meta gets copied to proxy's meta closure
3. Condition apply (execution): reads direction, adds ORDER BY with scoreFragment

This fixes the 'sort by full text rank orderBy enums works' test - ASC and
DESC now produce different result orderings. Also removes the stale WeakMap
bridge that was no longer needed.
@pyramation pyramation merged commit 0e7cec1 into main Mar 4, 2026
42 checks passed
@pyramation pyramation deleted the devin/1772607889-meta-system-rewrite branch March 4, 2026 11:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant