46リポジトリに跨るコンテキストを、AIがセマンティックに検索できるようにした話

comments (27)

1. Eryc Tri Juni Svia dev.to2026-07-04

   The refusal architecture part of this thread really got me — coming at it from a different angle (public content/GEO, not internal code graphs), but I think I'm seeing the same thing. In my own RAG testing, when the model doesn't have enough retrieved context it almost never says so, it just... fills the gap and sounds confident about it anyway. Same root problem, just showing up in content retrieval instead of code search. What stood out is you pushing for server-side gating instead of trusting the agent to flag its own uncertainty. Makes sense — if you leave it up to the model, it'll turn "not sure" into a fluent sentence every time, because that's what it's optimized to do. Now I'm curious if something like that could work outside a controlled internal setup — like, could a public site expose some kind of confidence signal to crawlers, or does this only work because you control both sides of the pipeline?

   1. Ryosuke Tsujivia dev.to2026-07-04

      Glad that part landed. Your RAG observation matches exactly what pushed us to server-side gating. The model turning "not enough context" into a fluent answer isn't a bug in any one model, it's what generation is. So the "I don't know" has to be produced by something that isn't a generator. On your question, I think the honest answer is that the enforcement half only works because we control both sides. What the server-side gate actually does is refuse to hand the model ambiguous material in the first place. Below a similarity threshold it returns a typed "no confident match" instead of the top-N-whatever, so the model never gets the raw ingredients to hallucinate from. That refusal happens before generation, and it requires owning the retrieval server. A public site can't enforce that, because the gate has to sit on the consumer's side of the pipe and you don't own the crawler. What a public site can do is make the honest path cheaper than the guessing path. Structured data, explicit scope statements ("this page covers X as of date Y, it does not cover Z"), stable anchors, machine-readable freshness. None of that forces a model to admit uncertainty, but it changes what retrieval hands the model, which is the same lever we pull internally, just without the guarantee. For crawlers specifically, my guess is confidence signals end up like robots.txt. A convention that well-behaved consumers respect and nothing enforces. Publisher-supplied relevance signals also have a trust problem baked in, since they're exactly the kind of thing that gets gamed (meta keywords walked this road already). There is one path where a publisher genuinely gets the gate back, though. Expose your content as an MCP server and let agents call it directly instead of scraping. Then you own the retrieval endpoint again, and everything we do internally becomes available to you. Typed refusals below a threshold, explicit scope answers, even query logs showing what AIs are actually asking about your content. [Stripe](https://docs.stripe.com/mcp) and [Cloudflare](https://developers.cloudflare.com/agents/model-context-protocol/cloudflare/servers-for-cloudflare/) already do this for their docs, and coding agents really do call those endpoints. To be clear, this doesn't move the needle on how search engines rank or summarize you. It's a separate serving channel for the agentic side of consumption, not an SEO lever. But for the slice of traffic that comes from agents rather than crawlers, it's the one place the publisher controls both sides of the pipe.

   2. Eryc Tri Juni Svia dev.to2026-07-05

      Really appreciate you taking the time to lay that out — "refusal has to happen before generation, and that requires owning the retrieval server" clarified something I'd been fuzzy on. To answer my own question with what I think the honest scope is: MCP is for agents that act — bidirectional, you own both ends, you can gate before generation. What I built (AGP) is aimed upstream of that, at crawlers that only ever read, once, passively, with no request/response loop to gate at all. Mechanically: I picked Google Sites deliberately as a worst-case CMS — it can't produce clean canonical tags or structured data, and it's on a zero-authority .my.id domain, so if this works at all, it isn't explainable by domain trust or CMS quality. A Cloudflare Worker sits in front of it and uses HTMLRewriter to edit the HTML mid-flight, as it streams past — not replacing the origin, just correcting what it hands off. Crawlers get a separately-stored clean semantic payload injected in; humans get the untouched page. Same content, different container, no attempt at persuading anyone of anything. So probably not competing solutions — MCP fixes the action side, AGP (at best) fixes whether the read side is even structurally intact in the first place. Still an open question whether "structurally clean" is enough on its own, or whether a crawler needs a way to signal back "was this actually sufficient" — which sounds like exactly the gap your server-side gating closes on your end.

   3. Ryosuke Tsujivia dev.to2026-07-05

      The worst-case-CMS setup is good experimental design. Zero-authority domain plus a CMS that can't emit structured data means if retrieval quality improves, you've actually isolated the variable. Most GEO claims never control for domain trust, so this alone puts your test ahead of the field. The part I'd watch is the crawler/human split itself. In classic SEO that shape has a name, cloaking, and engines penalize it regardless of intent because they can't verify intent, only divergence. Your defense is semantic equivalence, same content in a cleaner container, and I think that's the right line to hold. But it's worth designing as if someone will eventually check. If AGP-like layers spread, the obvious countermeasure for engines is to render both variants and diff the semantics, and the setups that survive that check are the ones where the equivalence is real. Yours sounds like it is. The gray zone arrives when people copy the architecture and start "improving" the crawler payload beyond what the human page says. On the feedback question, you have more signal than you might think. You own the Worker, which means you own the access logs, and crawl patterns are a crude but real read on what retrieval is doing. Which paths get fetched, how often, whether a crawler comes back after the clean payload shipped. It's nowhere near a "was this sufficient" signal from the model side, but it's the same move we made internally, when you can't gate the consumer, instrument the part of the pipe you do own.

   4. Eryc Tri Juni Svia dev.to2026-07-05

      Went back and actually audited the code against your point rather than just agreeing with it. Two findings worth sharing. First, the element-stripping for AI bots (script/style/iframe/header/footer) turned out to be justified, but I wanted to verify rather than assume: Google Sites ships large non-semantic JS blobs inside those wrapper tags, and the bot-facing payload carries its own replacement footer with the actual contact/nav info. So the stripping is noise reduction, not content removal — checked it, the semantic substance survives in the KV payload, the JS bloat doesn't. Good distinction to have confirmed rather than assumed. Second, honestly, the audit surfaced that not every page in the architecture is at the same maturity level. The case study page holds up well on content parity. Older parts of the routing logic still have some rough edges I'm working through — exactly the kind of thing your diffing prediction would catch, which is the point. Better to find it now going page by page than have it found later. Appreciate the push to actually check instead of just asserting it.

   5. Ryosuke Tsujivia dev.to2026-07-05

      Glad the comment turned into an actual audit, that's the best outcome I could have hoped for from this thread. And finding the uneven pages yourself, before any crawler-side diff exists, is the whole game. Good luck with AGP, I'll be curious how the experiment reads in a few months.

   6. Eryc Tri Juni Svia dev.to2026-07-05

      Appreciate that, genuinely. This whole thread ended up being worth more than four months of trying to get anyone to look at it — the diffing point specifically is going to stick with me long after this conversation ends. Going to keep working through the rest of the pages the same way, on my own time now rather than in public. Good talking through this with someone who actually knows the terrain. Take care.

2. Mudassir Khanvia dev.to2026-07-04

   the annotation maintenance loop has one structural risk: the AI that writes and the AI that reviews likely share model weights. a systematic generation error might look plausible to a reviewer with the same priors. the SLO catches missing coverage but not plausibly wrong coverage. the owner sampling Mike proposed closes that gap. what I am curious about is the annotation generation trigger: when a new boundary appears in a diff, the webhook fires and the AI annotates what is in the diff. but if a new route is registered in a config file and the actual handler lives in a file not touched by that diff, does the pipeline catch the handler as needing annotation, or does it slip through and stay unannotated until the SLO eventually fires?

   1. Ryosuke Tsujivia dev.to2026-07-04

      Great decomposition on both sides. On shared model weights, you're right, and Mike's owner sampling is exactly what closes that gap. AI review reduces the review load; it doesn't eliminate it. A small sample of ground-truth eyes stays in the loop specifically for the "plausibly wrong but covered" failure mode the SLO can't see. On the config-in-diff-but-handler-elsewhere case, in principle it should catch it, and this is where the code-graph approach diverges most clearly from diff-only SaaS AI Reviewer bots. Annotation-author and annotation-reviewer both traverse via the code graph, not the diff alone, so they can reason about code that never appears in the PR diff at all. When a new route registration lands in a config file, the graph resolves the handler that route points to (the boundary → handler edge that static analysis extracts up front). The pipeline sees "this route is new, and its handler in some-other-file.ts has no @graph-* annotation" as one connected piece of work, even when the handler file isn't in the touched set. That gets queued for annotation on the same PR. Where this would actually slip is if the config → handler resolution itself is dynamic in a way static analysis can't follow. Handler paths constructed from runtime variables, dynamic imports from computed strings, and similar patterns fall back to the SLO picking them up later. That's honestly the weakest link in that chain.

3. Wren Callowayvia dev.to2026-07-01

   [@ryantsuji](https://dev.to/ryantsuji) The exact-equality-after-transformation point is the correction that matters, and I want to make sure I absorb it rather than nod past it. You're right that "the transformation pointed at a different real entity than the author intended" is a categorically different animal than "the scorer picked a similar-looking node." My original framing smuggled in the fuzzy-match failure mode without earning it. Granularity loss is the real shape here, and it's actually the more insidious one to alert on precisely because every individual bridge is defensible in isolation — the boundary does contain the component. There's no single wrong edge to point at, just a distribution that's quietly coarser than the annotation author's mental model. Which makes me think the drift alert wants a second signal beyond "% resolving via strategies 3–6." That catches the cascade sliding down the priority order, but the parent_dir absorption you're describing might not move that number much if strategy 5 was always doing the work — it'd show up instead as fan-in: multiple SPG nodes converging on one boundary. So maybe the thing to threshold is the ratio of collapsed nodes per boundary per repo, not just which strategy won. A boundary that suddenly absorbs six Pages where it used to hold two is the granularity-loss tell even when match_strategy never changed. The daily cron diffing stored bridges against a from-scratch re-resolve is the piece I'd underweighted, and it's the closest thing you have to a correctness guard that isn't a human noticing. The gap is that it catches strategy flips, not stable-but-coarse joins that never flip. Pairing that with the quarterly owner-sampling is right — owners are the only ones who actually hold the intended granularity, and no cron recovers that from the derived identifiers alone. Good thread. I learned the shape of my own critique better by being wrong about half of it.

   1. Ryosuke Tsujivia dev.to2026-07-01

      The fan-in signal is the piece I'd been missing. You're right that strategy-drift alone misses stable-but-coarse joins, because "strategy 5 has always won" won't move the distribution while the number of SPG nodes pointing at a single boundary quietly climbs. Ratio of collapsed nodes per boundary per repo, tracked over time with a delta threshold, is a sharper monitor. It catches the failure mode where every individual bridge is defensible but the aggregate is coarser than the author intended. One layer I'd add on top: the same signal has a runtime use, not just a monitoring use. When an agent queries a boundary that aggregates N SPG Pages, that N itself is a confidence hint the agent should surface ("this boundary corresponds to N distinct annotated pages, treat the granularity accordingly"). Same information, shifted from "we notice granularity drift once a week" to "the consumer sees the granularity every query." Neither is implemented, but the second one might actually change AI behavior in practice, since the LLM in the loop can be told to treat high-fan-in boundaries as ambiguous rather than definitive. The gap in the cron you identified is exactly right. It's a strategy- flip detector, not a granularity-drift detector. That's a good reminder that "we already have a monitor for that" and "we have a monitor for the specific failure mode you're pointing at" aren't the same thing, and I probably lean on the cron too hard as a catch-all mental model. Owner-sampling as the correctness backbone is where this lands for me too. Deterministic identifiers can encode "which entity does this transformation point at" but they can't encode "what granularity did the author actually want." Only the owner holds that. The sampling is expensive compared to a cron, but it's the only piece that closes the loop on intent-vs-derivation. Combined with fan-in ratio alerts, that's a stack that starts to look like it's guarding correctness rather than existence. "I learned the shape of my own critique better by being wrong about half of it." That's the read I aspire to when I comment on other people's technical work. Really appreciate it.

4. Wren Callowayvia dev.to2026-07-01

   The SAME_ENTITY bridge coverage hitting 100% is the number I'd worry about most, precisely because you got there by adding strategies. Your issue #2 admits the Page bridge maps multiple annotation Pages to one boundary — which means "100% connected" and "100% correctly connected" have quietly diverged, and once they diverge the SLO stops measuring what it looks like it measures. A HANDLES_API handler can be 95%-connected downstream and still be bridged to the wrong annotation, and the vector search will happily return that wrong node as a "verified fact." That's a nastier failure than grep-inference, because it wears the costume of ground truth. The thing your SLOs guard is edge existence; what you can't cheaply guard is edge correctness, and the 6-strategy priority order is exactly where a silent mis-join hides — strategy 5 (parent-directory linking) firing when strategy 1 should have. If you're not already, I'd track how often the winning bridge is a low-priority fallback per repo and alert on drift there, not just on connection rate. A repo where suddenly 30% of Page joins resolve via "strip dynamic segments and match parent" is telling you the confident-but-wrong rate is climbing even while your dashboard stays green. One genuine question, not a gotcha: when a mis-join surfaces, does an agent or a human catch it, and how? That correction loop is the actual trust boundary of the whole system, and it's the one piece the post doesn't show.

   1. Ryosuke Tsujivia dev.to2026-07-01

      The 6-strategy priority order framing is accurate for the Page bridge.　That's literally what the code does: candidates are UNION'd across six　matching strategies (url_pattern / component_path / item_id / url_pascal　/ parent_dir / boundary_dynamic_strip), ranked by strategy priority,　top-1 wins per SPG node. The Api bridge has the same shape with four　stages (exact normalized match, trailing ? variance, trailing　/:dynamic strip, boundary :dynamic regex against literal segments for　dynamic dispatch). So the cascade isn't unique to Page. One nuance that changes the failure-mode profile though: each strategy　still resolves to exact equality on a deterministically-derived　identifier. No fuzzy string matching, no edit distance, no substring　scoring. The "heuristic" part is the choice of transformation rule　(strip this prefix, PascalCase these segments, walk up to parent　directory). Once the transformation runs, the actual join is =. That　constrains how mis-joins can happen: not "the algorithm picked a　similar-looking wrong entity" but "this transformation rule pointed at a　different real entity than the annotation author intended." The mis-join mode we actually see (issue #2) is the second kind, and　it's granularity loss rather than confidently-wrong: multiple SPG Pages　collapse onto one boundary when a shared parent directory catches them　(strategy 5, parent_dir). Each individual bridge is "correct" in the　sense that the boundary does contain that component, but the annotation　author often wanted more granular boundaries than exist. You're right　that this still isn't what a "% connected" SLO catches, so your critique　lands, just aimed at collapsed granularity rather than plausible-but-　wrong. Your alerting suggestion is the right shape for detecting that.　Per-strategy tagging is present on each edge (the match_strategy field　is stored on the SAME_ENTITY row), so per-repo distribution is queryable　retroactively. What isn't wired up is the drift alert. Threshold on　"% of joins resolving via strategies 3 through 6 per repo per week" and　pager on delta would surface the parent_dir absorption pattern you're　pointing at. Not implemented today. To your genuine question on correction loops. Mis-joins currently　surface through three paths: An engineer notices during use (agent returns a plausibly wrong　neighbor, they dig in) The daily boundary-analysis cron re-runs the strategy resolver from　scratch and diffs against stored bridges; a bridge that flipped　strategies shows up in the change log A downstream test fails (rare, because most consumers are LLMs that　don't verify) Human is the primary catcher today. AI in the loop is good at using　bridges but bad at questioning them, precisely because they look like　ground truth. Making the correction loop first-class rather than an　artifact of use is the actual gap you're identifying. The owner-sampling　proposal from an earlier thread in these comments (a quarterly sample of　~50 bridges per relationship type, marked mismatch by entity owners) is　aimed at exactly this. Combined with your strategy-drift alerting, that　gets a lot closer to guarding correctness rather than just connectivity. "Confident-but-wrong wearing the costume of ground truth" is a phrase　I'll be borrowing. Thanks for the read.

5. Nazar Boykovia dev.to2026-06-30

   On the no dynamic analysis gap at the end, one thing that might fold in cleanly is treating production call counts as weights on the edges you already have. You're not adding a new graph, just annotating the existing SAME_ENTITY and call edges with how often each one actually fired last week, and the dead edges fall out as the ones that never lit up. It also gives your blast radius queries a way to sort by real traffic instead of treating every reachable path as equally likely. The choice to annotate only the boundaries is the part I found most convincing, since it's the same instinct as documenting a system for a new teammate. You explain the seams, not every private helper.

   1. Ryosuke Tsujivia dev.to2026-06-30

      Yes, that's exactly the direction I want to go. Annotating existing static edges with runtime weights (call counts, last-fired timestamps) rather than building a parallel graph fits naturally on top of the boundary-level structure we already have. At the boundary granularity (API / Event / Page entrypoints), this works cleanly: how often a boundary fires last week is a reasonable importance and freshness signal. The complication shows up when you push the weighting inside the boundary and start annotating individual call edges / nodes along the internal path. There, what the count means depends on whether the path is happy or error. On a happy path, the usual logic holds (frequent = important, rare = candidate for cleanup). For error-path code (catch blocks, retry handlers, fallback branches), frequent execution still carries value because it surfaces upstream problems worth investigating. But infrequent execution doesn't imply the handler is unimportant. A rarely-fired catch block might be the safety net that's quietly kept the system intact all year. So for internal-path weighting, you'd want to classify the execution path itself first (happy / error / recovery / one-off init) before you can interpret what the count actually means. I don't have a clean answer for how to do that classification reliably yet. Some combination of structural cues (try/catch position, branch reachability) plus runtime correlation with error logs / 5xx rates seems plausible, but it's the main open problem for me on the dynamic side.

6. Kartik N V J Kvia dev.to2026-06-30

   Embedding raw function bodies tends to bury the signal under boilerplate, so fixing the entry-point problem first makes sense. Did you embed a generated summary of each node instead of the source span, and if so how do you keep those summaries from drifting as the code changes under them? Static-analysis-first seems like the only way the graph stays trustworthy across 46 repos.

   1. Ryosuke Tsujivia dev.to2026-06-30

      Yeah, that's basically the split. One AI writes annotations on each PR diff, another reviews that they still match the code. A few extra details worth flagging: We don't embed a runtime-generated summary, we embed the @graph-business JSDoc field that the annotation AI commits directly above each declaration. It's 1-2 sentences of business semantics co-located with the code, so the thing being indexed is small, human-readable, and survives in git history. Drift detection is mostly free: since the annotation lives in JSDoc on top of the declaration, any code change in that file pulls the annotation into the same PR diff. The reviewer AI sees both sides side-by-side. Both AIs have a hard rule that they can ONLY modify JSDoc, never application logic. The reviewer enforces it (any non-JSDoc diff line is treated as REQUEST_CHANGES). Without this constraint the two roles could cooperatively rewrite code "to keep things consistent", which is exactly the drift you'd want to prevent. Static analysis (tree-sitter) still owns the structural part of the graph (files, functions, imports, calls). The AI layer only fills business-level metadata (@graph-business, @graph-connects), so the drift surface is the annotation layer only, never the structural backbone. And yes, static-analysis-first is exactly why this holds across 46 repos. The annotation layer just adds the part static analysis can't see (e.g. "this endpoint is the Slack OAuth callback"), and the two AI roles keep that layer in sync with the code.

7. Sloan the DEV Moderatorvia dev.to2026-06-30

   Hey, this article appears to have been generated with the assistance of ChatGPT or possibly some other AI tool. We allow our community members to use AI assistance when writing articles as long as they abide by our [guidelines](https://dev.to/guidelines-for-ai-assisted-articles-on-dev). Please review the guidelines and edit your post to add a disclaimer. Failure to follow these guidelines could result in DEV admin lowering the score of your post, making it less visible to the rest of the community. Or, if upon review we find this post to be particularly harmful, we may decide to unpublish it completely. We hope you understand and take care to follow our guidelines going forward!

   1. Ryosuke Tsujivia dev.to2026-06-30

      Thanks for flagging this — I've added an AI assistance disclosure to the top of the article. (And going forward, all my dev.to articles will carry the same disclosure automatically.)

   2. FrancisTRᴅᴇᴠ (っ◔◡◔)っvia dev.to2026-06-30

      Thanks for Correcting the issue. This issue is now resolved. Great post as well :D

   3. Ryosuke Tsujivia dev.to2026-06-30

      Thanks Francis, appreciate the quick follow-up. Glad the post landed well!

8. Siyuvia dev.to2026-06-30

   The "entry-point problem" you articulate, where the knowledge graph exists but has no semantic surface for natural-language queries, maps directly onto a parallel problem in professional discovery. LinkedIn has the data, CVs have the data, but there is no way for an AI agent to ask "who in my network has led a database migration, communicates asynchronously, and prefers written briefs over meetings" and get verified results. I work on Opportunity Skill, which builds structured semantic impressions from daily work patterns that other agents can query, solving the same category of problem for professional knowledge instead of code knowledge. In both cases the cost is the same: the data already exists, but no one can reach it. What interests me most about your annotation philosophy is the deliberate minimalism: only boundary nodes get annotated, and AI handles it entirely in a parallel branch with no human review. This is not a compromise, it is a design insight. Boundary nodes carry disproportionate meaning relative to their count. A single @graph-business tag on a page explains more about what the system does than annotating every internal function ever could. The parallel-branch decision goes further: it acknowledges that maintenance cost, not annotation quality, is the bottleneck at scale. Humans cannot be relied upon to keep annotations current across 46 repos because annotation drift is invisible to them. AI can, because annotation generation is just another pipeline step. The same principle applies to maintaining professional semantic profiles: humans forget to update their bios, but an agent observing daily work patterns can refresh impressions continuously without the user lifting a finger. The non-engineer adoption data, 2,800 calls from stylists and customer support, reveals something beyond "semantic search works." A graph queried by meaning reallocates who gets to ask questions. When the access interface shifts from grep to natural language, knowledge stops belonging exclusively to the people who know where it lives and starts belonging to anyone who knows what they need. The same threshold exists in professional discovery: the barrier between needing a connection and finding one is not missing information, it is missing semantic entry points. The moment profiles become agent-queryable rather than human-readable, the entire cost structure of discovery changes. The graph was always there. The entry point was not.

   1. Ryosuke Tsujivia dev.to2026-06-30

      This is the parallel I was hoping someone would see — and your cross-domain framing is sharper than what I'd been holding internally. On boundary-node minimalism: yes, exactly. Once I accepted that internal helpers don't carry semantic weight (only the entry/exit surfaces do), the cost equation flipped. I'd been thinking about "how to annotate everything" when the answer was "annotate almost nothing, but pick the right almost-nothings." Boundary nodes are where intent is expressed because they're the only places where the system communicates with anything outside itself. I'd extend your maintenance-cost framing by one layer: humans don't just forget — they actively don't notice drift, because the drift is between code and meaning, not between code and code. Compilers catch code-vs-code drift. Nothing catches code-vs-meaning drift except a verifier that also reads meaning, which, pre-LLM, was a human reviewer context-switching every PR. AI verifying code-vs-meaning at PR time, every PR, asymmetric cost-wise — that's the actual unlock. The parallel-branch implementation is adaptation to the constraint we have today: 46 long-running repos where retrofitting annotations into main isn't realistic. For new environments we're building AI-native from day one, annotations sit in main alongside code, generated and reviewed in the same PR — convergent steady state, different starting point. The parallel-branch is the legacy path; main-branch annotations are the greenfield path. The "who gets to ask questions" point is where my own framing was still incomplete. I'd been describing it as "non-engineers can search now," but yours is more accurate: the entry point is where access is gatekept. grep was a competency moat masquerading as a tool. The moment the interface becomes natural language, knowledge becomes available to whoever knows what they need. The graph was always there — the entry point wasn't. Curious about Opportunity Skill. The piece I'd most want to understand: how do you handle freshness/staleness on observed-work impressions? Code annotations regenerate on PR diff, which is a discrete signal. Continuous work observation has no such discrete trigger, so I'd imagine you need a different mechanism for "this impression is current" vs "this is stale."

9. Mike Czerwinskivia dev.to2026-06-30

   The recall math from Part 1 frames the accuracy ceiling at >99% per hop. Part 2's SLOs sit one layer under: they measure intra-graph connectivity (API chain >=95%, DB completeness >=80%, event fields >=70%, ambiguous = 0). Completeness, not correctness of the bridge itself. Worth a second pass for one shape the SAME_ENTITY normalizations could be hiding. Recall counted at the edge measures edges that exist. It does not catch edges that connect what they should not. A PascalCase normalization with 99% recall can also produce a 5% wrong-pair join rate if the normalization over-fits on local naming. The recall number is intact. The graph's downstream truth is not. The way to surface that error mode is an exogenous check the bridge author does not control: pull a sampled 50 SAME_ENTITY edges per quarter, hand to whoever actually owns the entities on each side (the API team, the page team, the task team), and have them mark mismatch. The cost is finite, the sample is small, and the SLO threshold becomes "% of sampled bridges confirmed correct by entity-owner >= 95%", distinct from "% of handlers reachable >= 95%". Same target, different layer of trust. The 30% non-engineer usage from this part is the more telling receipt for the system than any of the connectivity numbers. Stylists, ops, executives querying natural language through MCP and trusting the answer means refusal architecture matters as much as connectivity. The question I keep landing on for systems at this scale: what does the MCP return when the graph genuinely does not know? A confident wrong answer at 95% connectivity is more dangerous than a low-confidence answer at 70%. Curious whether SPG has explicit "I don't know" semantics, or whether it always returns the closest annotation it can find.

   1. Ryosuke Tsujivia dev.to2026-06-30

      Thanks for the careful read. The framing I'd push back on first: those SLOs aren't measuring bridge correctness — they're connectivity floors, set deliberately conservative. We'd rather under-recall than produce false-positive joins, so the 95/80/70 isn't a ceiling we're chasing; it's a "don't let this regress" floor we won't drop below. Each SAME_ENTITY normalization is tiered exact-match-first with eager gating on the looser fallbacks. Chasing higher recall would mean loosening those gates, and the regression alarm would trip. A confidently wrong neighbor is worse than a missing one — especially with non-engineers in the consumer loop, where the cost of a wrong-but-plausible answer changes character entirely. That said, the entity-owner sampling check you described is a layer we don't currently have, and the framing is clean: "% of sampled bridges confirmed by owner ≥ 95%" as a trust signal distinct from connectivity. Worth adding. On "what does MCP return when the graph genuinely doesn't know" — semantic search returns ranked similarity scores, so the agent sees signal it could reason about, but explicit refusal isn't first-class yet. A real gap. Coverage SLOs guarantee that high-confidence neighbors exist when they should, but they aren't refusal signals on their own.

   2. Mike Czerwinskivia dev.to2026-06-30

      Recasting the SLO as a floor rather than a ceiling clears most of the confusion I was carrying into the read. A floor on connectivity says "do not regress below this, because below this the consumer reasoning collapses." A ceiling on correctness says "this is the truth we will defend." Those are completely different commitments, and conflating them is what makes recall-at-95 sound like a quality claim when it is actually a non-regression contract. The owner-sampling check then sits cleanly orthogonal to the SLOs. SLOs guarantee neighbors exist; sampling tells you whether the ones that exist are the right ones. Reporting them separately keeps each one honest, because either can move while the other holds. Refusal as first-class output is the part I would push hardest on next. Ranked similarity scores are useful debugging signal, but until the consumer agent can receive an explicit "the graph does not know this" the silent failure mode is plausible-but-wrong, which is exactly the failure consumers cannot detect. The SLOs you have today can promise that high-confidence neighbors exist when they should; they cannot promise that absence is communicated. Those are different guarantees and a non-engineering consumer needs both.

   3. Ryosuke Tsujivia dev.to2026-06-30

      Recast nails it. Floor for non-regression, ceiling for truth, completely different commitments, and yes, "recall at 95" reads as a quality claim when the actual commitment is non-regression. The article didn't separate those carefully enough; that's on me. Owner-sampling as a separate report line is going on the roadmap. Treating SLOs (neighbors exist) and owner-confirmed correctness (they're the right ones) as independent gauges that can drift independently is the right framing. The cost of running ~50 sampled edges per relationship type quarterly is bounded, and it gives entity owners a recurring stake in the bridges that point at their domain. Refusal as first-class output is the part I have the least good answer for. Today the MCP server returns ranked similarity, which means the trust boundary sits on the consumer agent: it has to look at the scores and decide whether to back off. In practice, an LLM in the agent loop will happily turn a 0.62 cosine into a confident sentence. The shape of the fix is roughly: Server-side threshold gating. Below a per-relationship similarity floor, return an explicit no_match / empty result rather than a top-K with low scores. The trust boundary moves from agent to server. Tiered confidence in the response itself (high / low / none) rather than a single score the agent re-interprets. Removes the "is 0.7 good?" judgment from the consumer. Pair with the owner-sampling check so the thresholds stay calibrated. Otherwise the threshold becomes another opaque number we can't defend. The shape I keep landing on is that absence and uncertainty are different failure modes and want different signals: absence is just honest, uncertainty at least lets the consumer ask a follow-up, but plausible-but-wrong is invisible to a non-engineering consumer. That last one is where the server has to prevent the bad answer at source rather than rely on the agent to decline. Appreciate the read; the threads have sharpened the framing more than the original article did.