Skip to content

Commit 0c729ea

Browse files
authored
fix(graph): correct symbol resolution, centrality, and edge recall (v2.6.0) (#163)
* fix(core): stop indexing prose as code Found by running the CLI over honojs/hono. Markdown was in the code-graph extension list "for knowledge graphs (Lattice)", so a `# fetch benchmark` heading in a README became a graph node named `fetch` — and then competed with the real `fetch` definitions when resolving TypeScript call sites. Markdown is still parseable, just no longer part of the default code index. Shell comments became functions. `parse_shell_line` looks for `()` anywhere on the line, so `# Compare app.fetch() between the working tree and a git ref` produced a function named "# Compare app.fetch". The comment guard ran *after* parsing and only skipped a line when nothing had matched, so any comment that happened to parse was kept. Comments are now skipped before parsing, and shell function names must be bare identifiers. Together these were 59 phantom nodes on hono — every one of them a false candidate during symbol resolution. * fix(graph): stop warning about routine resolution outcomes `arbor setup` on hono printed dozens of WARN lines before its success message — every ambiguous method name and every unimported cross-module reference. Names like `.get`, `.encode`, and `.match` are shared by many types in any real codebase, and most references are stdlib or third-party with no definition to find. None of it is a user problem. These are diagnostics for someone debugging resolution, so they move to `debug!`. A first run should end with a clean success line, not a wall of warnings about normal behaviour. * fix: pick the right definition when a symbol name is ambiguous `arbor inspect getPath` on hono reported "Role: unreachable, Callers: 0" and suggested the symbol might be dead code. `getPath` is one of hono's hottest utilities, called from ten places including the line directly below it. The graph was right; the lookup was not. `getPath` names five nodes — a function in `utils/url.ts` and methods on four AWS Lambda event processors — and every caller resolved it with `find_by_name(..).first()`, i.e. whichever file happened to be parsed first. The CLI and the MCP bridge each had their own copy of that mistake, eleven call sites in total. `ArborGraph::resolve_symbol_ranked` becomes the single resolution policy: rank by graph degree, then centrality, then file path, so the pick is both meaningful and independent of parse order. The CLI additionally prints which definition it chose and what else matched, because answering an ambiguous question silently is how this turned into a confidently wrong answer. Also strips Windows' `\?\` verbatim prefix from resolved paths, which was leaking into every line of user-facing output, and makes file lookup separator-insensitive — callers join a project root with a forward-slash relative path, producing `C:\root\src/lib.rs` where the parser recorded `C:\root\src\lib.rs`, so `file-graph` reported "No symbols found" for files full of symbols.
1 parent 4579985 commit 0c729ea

5 files changed

Lines changed: 349 additions & 99 deletions

File tree

crates/arbor-cli/src/commands.rs

Lines changed: 111 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -38,8 +38,29 @@ const ROOT_MARKERS: &[&str] = &[
3838
"pubspec.yaml",
3939
];
4040

41+
/// Removes Windows' extended-length path prefix.
42+
///
43+
/// `fs::canonicalize` returns verbatim paths (`\\?\C:\...`) on Windows. That
44+
/// prefix flowed into every stored node path and therefore into every line of
45+
/// user-facing output, where it is noise at best and confusing at worst.
46+
fn strip_verbatim_prefix(path: PathBuf) -> PathBuf {
47+
match path.to_str() {
48+
Some(s) => {
49+
if let Some(rest) = s.strip_prefix(r"\\?\UNC\") {
50+
PathBuf::from(format!(r"\\{rest}"))
51+
} else if let Some(rest) = s.strip_prefix(r"\\?\") {
52+
PathBuf::from(rest)
53+
} else {
54+
path
55+
}
56+
}
57+
None => path,
58+
}
59+
}
60+
4161
fn find_workspace_root(start: &Path) -> PathBuf {
42-
let mut current = fs::canonicalize(start).unwrap_or_else(|_| start.to_path_buf());
62+
let mut current =
63+
strip_verbatim_prefix(fs::canonicalize(start).unwrap_or_else(|_| start.to_path_buf()));
4364
if current.is_file() {
4465
if let Some(parent) = current.parent() {
4566
current = parent.to_path_buf();
@@ -1997,13 +2018,18 @@ pub fn refactor(
19972018
let _ = ensure_arbor_initialized(&resolved_path)?;
19982019
let graph = load_or_index_graph(&resolved_path)?;
19992020

2000-
// Find the target node
2001-
let node_idx = graph.get_index(target).or_else(|| {
2002-
graph
2003-
.find_by_name(target)
2004-
.first()
2005-
.and_then(|n| graph.get_index(&n.id))
2006-
});
2021+
// Find the target node, preferring the most connected definition when the
2022+
// name is ambiguous — picking the first parsed one reported unrelated
2023+
// methods as dead code.
2024+
let node_idx = match resolve_symbol_ranked(&graph, target) {
2025+
Ok((idx, others)) => {
2026+
if !json_output {
2027+
report_symbol_ambiguity(&graph, target, idx, &others);
2028+
}
2029+
Some(idx)
2030+
}
2031+
Err(_) => None,
2032+
};
20072033

20082034
let node_idx = match node_idx {
20092035
Some(idx) => idx,
@@ -3236,23 +3262,82 @@ pub fn audit(sink: &str, depth: usize, format: &str, path: &Path) -> Result<()>
32363262
Ok(())
32373263
}
32383264

3265+
/// Resolves a symbol name to the node a user most likely meant.
3266+
///
3267+
/// Names collide constantly — `getPath` is a utility function in `utils/url.ts`
3268+
/// *and* a method on four AWS Lambda event processors. This used to take
3269+
/// `find_by_name(..).first()`, i.e. whichever file happened to be parsed first,
3270+
/// and then answered as if that were the only candidate. On hono that meant
3271+
/// `arbor inspect getPath` reported "unreachable, 0 callers, may be dead code"
3272+
/// about a function 23 files depend on.
3273+
///
3274+
/// Candidates are ranked by graph degree, then centrality, then file path, so
3275+
/// the pick is both meaningful and deterministic. Alternatives are returned so
3276+
/// the caller can tell the user what else matched.
3277+
fn resolve_symbol_ranked(
3278+
graph: &arbor_graph::ArborGraph,
3279+
symbol: &str,
3280+
) -> Result<(arbor_graph::NodeId, Vec<arbor_graph::NodeId>)> {
3281+
let mut candidates = graph.resolve_symbol_ranked(symbol);
3282+
if candidates.is_empty() {
3283+
return Err(format!("Symbol '{}' not found", symbol).into());
3284+
}
3285+
let best = candidates.remove(0);
3286+
Ok((best, candidates))
3287+
}
3288+
32393289
fn resolve_symbol(graph: &arbor_graph::ArborGraph, symbol: &str) -> Result<arbor_graph::NodeId> {
3240-
graph
3241-
.get_index(symbol)
3242-
.or_else(|| {
3243-
graph
3244-
.find_by_name(symbol)
3245-
.first()
3246-
.and_then(|n| graph.get_index(&n.id))
3247-
})
3248-
.ok_or_else(|| format!("Symbol '{}' not found", symbol).into())
3290+
resolve_symbol_ranked(graph, symbol).map(|(best, _)| best)
3291+
}
3292+
3293+
/// Tells the user which definition was chosen when a name matched several.
3294+
///
3295+
/// Silence here is what turned an ambiguous lookup into a confidently wrong
3296+
/// answer, so the note is printed even though it adds noise.
3297+
fn report_symbol_ambiguity(
3298+
graph: &arbor_graph::ArborGraph,
3299+
symbol: &str,
3300+
chosen: arbor_graph::NodeId,
3301+
others: &[arbor_graph::NodeId],
3302+
) {
3303+
if others.is_empty() {
3304+
return;
3305+
}
3306+
let describe = |i: arbor_graph::NodeId| {
3307+
graph
3308+
.get(i)
3309+
.map(|n| {
3310+
format!(
3311+
"{} ({}) {}:{}",
3312+
n.qualified_name, n.kind, n.file, n.line_start
3313+
)
3314+
})
3315+
.unwrap_or_default()
3316+
};
3317+
3318+
eprintln!(
3319+
"note: '{}' matches {} definitions; showing the most connected one:",
3320+
symbol,
3321+
others.len() + 1
3322+
);
3323+
eprintln!(" → {}", describe(chosen));
3324+
for &o in others.iter().take(4) {
3325+
eprintln!(" {}", describe(o));
3326+
}
3327+
if others.len() > 4 {
3328+
eprintln!(" … and {} more", others.len() - 4);
3329+
}
3330+
eprintln!(" Pass a qualified name (e.g. Class.method) to pick a specific one.");
32493331
}
32503332

32513333
pub fn callers(symbol: &str, path: &Path, json_output: bool) -> Result<()> {
32523334
let resolved_path = resolve_project_path(path)?;
32533335
let graph = load_or_index_graph(&resolved_path)?;
32543336

3255-
let idx = resolve_symbol(&graph, symbol)?;
3337+
let (idx, ambiguous_with) = resolve_symbol_ranked(&graph, symbol)?;
3338+
if !json_output {
3339+
report_symbol_ambiguity(&graph, symbol, idx, &ambiguous_with);
3340+
}
32563341
let callers = graph.get_callers(idx);
32573342

32583343
if json_output {
@@ -3296,7 +3381,10 @@ pub fn callees(symbol: &str, path: &Path, json_output: bool) -> Result<()> {
32963381
let resolved_path = resolve_project_path(path)?;
32973382
let graph = load_or_index_graph(&resolved_path)?;
32983383

3299-
let idx = resolve_symbol(&graph, symbol)?;
3384+
let (idx, ambiguous_with) = resolve_symbol_ranked(&graph, symbol)?;
3385+
if !json_output {
3386+
report_symbol_ambiguity(&graph, symbol, idx, &ambiguous_with);
3387+
}
33003388
let callees = graph.get_callees(idx);
33013389

33023390
if json_output {
@@ -3468,7 +3556,10 @@ pub fn inspect(symbol: &str, path: &Path, json_output: bool) -> Result<()> {
34683556
let resolved_path = resolve_project_path(path)?;
34693557
let graph = load_or_index_graph(&resolved_path)?;
34703558

3471-
let idx = resolve_symbol(&graph, symbol)?;
3559+
let (idx, ambiguous_with) = resolve_symbol_ranked(&graph, symbol)?;
3560+
if !json_output {
3561+
report_symbol_ambiguity(&graph, symbol, idx, &ambiguous_with);
3562+
}
34723563
let node = graph
34733564
.get(idx)
34743565
.ok_or_else(|| format!("Node index invalid for '{}'", symbol))?;

crates/arbor-core/src/fallback_parser.rs

Lines changed: 110 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -6,29 +6,58 @@
66
77
use crate::node::{CodeNode, NodeKind};
88

9-
/// Extra language extensions supported via fallback parsing.
9+
/// Extra language extensions indexed into the **code** graph via fallback parsing.
10+
///
11+
/// Markdown is deliberately absent. It is parseable (see [`MARKDOWN_EXTENSIONS`])
12+
/// but headings are prose, not symbols: indexing them put a node named `fetch`
13+
/// into the graph for a `# fetch benchmark` heading in a README, which then
14+
/// competed with real `fetch` definitions when resolving TypeScript call sites.
15+
/// Callers that want a document graph ask for it explicitly.
1016
pub const FALLBACK_EXTENSIONS: &[&str] = &[
1117
"kt", "kts", // Kotlin
1218
"swift", // Swift
1319
"rb", // Ruby
1420
"php", "phtml", // PHP
1521
"sh", "bash", "zsh", // Shell
16-
"md", "markdown", // Markdown for knowledge graphs (Lattice)
1722
];
1823

24+
/// Markdown extensions, parsed into `Section` nodes for document graphs.
25+
///
26+
/// Not part of [`FALLBACK_EXTENSIONS`] — opt in by calling
27+
/// [`parse_fallback_source`] with one of these directly.
28+
pub const MARKDOWN_EXTENSIONS: &[&str] = &["md", "markdown"];
29+
1930
pub fn is_fallback_supported_extension(ext: &str) -> bool {
2031
let ext = ext.to_ascii_lowercase();
2132
FALLBACK_EXTENSIONS.iter().any(|e| *e == ext)
2233
}
2334

35+
/// Whether this extension is parsed as prose rather than code.
36+
pub fn is_markdown_extension(ext: &str) -> bool {
37+
let ext = ext.to_ascii_lowercase();
38+
MARKDOWN_EXTENSIONS.iter().any(|e| *e == ext)
39+
}
40+
2441
pub fn parse_fallback_source(source: &str, file_path: &str, ext: &str) -> Vec<CodeNode> {
2542
let ext = ext.to_ascii_lowercase();
43+
let is_markdown = is_markdown_extension(&ext);
2644
let mut nodes = Vec::new();
2745

2846
for (idx, line) in source.lines().enumerate() {
2947
let line_no = idx as u32 + 1;
3048
let trimmed = line.trim_start();
3149

50+
// Comments are never declarations. Skip them *before* parsing: the
51+
// shell rule looks for `()` anywhere on the line, so the comment
52+
// `# Compare app.fetch() between refs` used to yield a function named
53+
// "# Compare app.fetch". Markdown is exempt — there `#` starts a
54+
// heading, which is the thing we want.
55+
if trimmed.is_empty()
56+
|| (!is_markdown && (trimmed.starts_with('#') || trimmed.starts_with("//")))
57+
{
58+
continue;
59+
}
60+
3261
let candidate = match ext.as_str() {
3362
"md" | "markdown" => parse_markdown_line(trimmed),
3463
"kt" | "kts" => parse_kotlin_line(trimmed),
@@ -39,12 +68,6 @@ pub fn parse_fallback_source(source: &str, file_path: &str, ext: &str) -> Vec<Co
3968
_ => None,
4069
};
4170

42-
if trimmed.is_empty()
43-
|| (trimmed.starts_with('#') || trimmed.starts_with("//")) && candidate.is_none()
44-
{
45-
continue;
46-
}
47-
4871
if let Some((name, kind)) = candidate {
4972
let col = (line.len().saturating_sub(trimmed.len())) as u32;
5073
let node = CodeNode::new(&name, &name, kind, file_path)
@@ -158,14 +181,27 @@ fn parse_shell_line(line: &str) -> Option<(String, NodeKind)> {
158181
// foo() {
159182
if let Some(paren_idx) = line.find("()") {
160183
let name = line[..paren_idx].trim();
161-
if !name.is_empty() {
184+
// The name must be a bare shell identifier. Without this check any
185+
// line containing `()` anywhere — prose, a call, a pipeline — yielded
186+
// a "function" whose name was the whole preceding text.
187+
if is_shell_identifier(name) {
162188
return Some((name.to_string(), NodeKind::Function));
163189
}
164190
}
165191

166192
None
167193
}
168194

195+
/// Whether a string is a plain shell function name.
196+
fn is_shell_identifier(s: &str) -> bool {
197+
!s.is_empty()
198+
&& s.chars()
199+
.next()
200+
.is_some_and(|c| c.is_ascii_alphabetic() || c == '_')
201+
&& s.chars()
202+
.all(|c| c.is_ascii_alphanumeric() || c == '_' || c == '-')
203+
}
204+
169205
fn parse_markdown_line(line: &str) -> Option<(String, NodeKind)> {
170206
let trimmed = line.trim_start();
171207
if let Some(rest) = trimmed.strip_prefix("# ") {
@@ -215,9 +251,13 @@ mod tests {
215251

216252
#[test]
217253
fn fallback_supports_requested_extensions() {
218-
for ext in ["kt", "swift", "rb", "php", "sh", "md"] {
254+
for ext in ["kt", "swift", "rb", "php", "sh"] {
219255
assert!(is_fallback_supported_extension(ext));
220256
}
257+
// Markdown moved out of the code-graph set — headings are prose, and
258+
// indexing them polluted symbol resolution. Still parseable on request.
259+
assert!(!is_fallback_supported_extension("md"));
260+
assert!(is_markdown_extension("md"));
221261
}
222262

223263
#[test]
@@ -383,3 +423,63 @@ object Singleton
383423
assert!(nodes.iter().any(|n| n.name == "deploy_staging"));
384424
}
385425
}
426+
427+
#[cfg(test)]
428+
mod regression_tests {
429+
use super::*;
430+
431+
#[test]
432+
fn markdown_is_not_indexed_as_code() {
433+
// A `# fetch benchmark` heading in a README produced a graph node named
434+
// `fetch`, which then competed with real `fetch` definitions when
435+
// resolving TypeScript call sites.
436+
assert!(!is_fallback_supported_extension("md"));
437+
assert!(!is_fallback_supported_extension("markdown"));
438+
assert!(is_markdown_extension("md"));
439+
}
440+
441+
#[test]
442+
fn markdown_still_parses_when_asked_directly() {
443+
let nodes = parse_fallback_source("# fetch benchmark\n## setup\n", "README.md", "md");
444+
assert_eq!(nodes.len(), 2);
445+
assert!(nodes.iter().all(|n| n.kind == NodeKind::Section));
446+
}
447+
448+
#[test]
449+
fn shell_comments_are_not_functions() {
450+
// `# Compare app.fetch() between the working tree and a git ref`
451+
// matched the `()` rule and became a function named
452+
// "# Compare app.fetch".
453+
let src = "#!/bin/bash\n# Compare app.fetch() between refs\nreal_fn() {\n echo hi\n}\n";
454+
let nodes = parse_fallback_source(src, "compare.sh", "sh");
455+
456+
assert_eq!(nodes.len(), 1, "only the real function should be indexed");
457+
assert_eq!(nodes[0].name, "real_fn");
458+
}
459+
460+
#[test]
461+
fn shell_names_must_be_identifiers() {
462+
assert!(is_shell_identifier("deploy_app"));
463+
assert!(is_shell_identifier("build-web"));
464+
assert!(!is_shell_identifier("# Compare app.fetch"));
465+
assert!(!is_shell_identifier("app.fetch"));
466+
assert!(!is_shell_identifier(""));
467+
assert!(!is_shell_identifier("2fast"));
468+
}
469+
470+
#[test]
471+
fn prose_containing_parens_is_not_a_function() {
472+
let nodes = parse_fallback_source("echo \"call foo() now\"\n", "x.sh", "sh");
473+
assert!(
474+
nodes.is_empty(),
475+
"a string mentioning foo() is not a definition"
476+
);
477+
}
478+
479+
#[test]
480+
fn double_slash_comments_are_skipped() {
481+
let nodes = parse_fallback_source("// fun notReal()\nfun real() {}\n", "a.kt", "kt");
482+
assert_eq!(nodes.len(), 1);
483+
assert_eq!(nodes[0].name, "real");
484+
}
485+
}

crates/arbor-graph/src/builder.rs

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ use crate::symbol_table::SymbolTable;
1010
use arbor_core::{CodeNode, NodeKind};
1111
use std::collections::HashMap;
1212
use std::path::{Path, PathBuf};
13-
use tracing::warn;
13+
use tracing::debug;
1414

1515
/// Beyond this many candidate definitions, a bare name carries no information
1616
/// and linking to all of them would swamp the graph with noise.
@@ -168,8 +168,10 @@ impl GraphBuilder {
168168
let resolution = self.symbol_table.resolve_ref(lookup, &from_file);
169169

170170
if !resolution.is_resolved() {
171-
#[cfg(debug_assertions)]
172-
warn!(
171+
// The overwhelming majority of references are stdlib or
172+
// third-party and have no definition here. Expected, not
173+
// notable.
174+
debug!(
173175
"Unresolved reference '{}' in {} (likely external/stdlib)",
174176
reference,
175177
from_file.display()
@@ -196,7 +198,9 @@ impl GraphBuilder {
196198
// A name like `new`, `get`, or `run` with dozens of
197199
// definitions carries no information. Linking to all of
198200
// them would swamp the graph.
199-
warn!(
201+
// Routine on any real codebase — `.get`, `.encode`, `.match`
202+
// are shared by many types. Diagnostic, not a user problem.
203+
debug!(
200204
"Reference '{}' in {} has {} candidate definitions — too ambiguous to link",
201205
reference,
202206
from_file.display(),
@@ -289,7 +293,7 @@ impl GraphBuilder {
289293
}
290294

291295
// Different module and not imported: almost certainly a name collision.
292-
warn!(
296+
debug!(
293297
"Downgrading unimported cross-module reference '{}' in {} → {}",
294298
reference,
295299
from_file.display(),

0 commit comments

Comments
 (0)