Tags: PlateerLab/synaptic-memory
Tags
chore: bump 0.26.0 → 0.27.0 for PyPI release
Single coherent release bundling:
- v0.26 editable graph — knowledge_update / knowledge_unlink /
knowledge_update_edge / knowledge_merge_nodes MCP tools + facade
methods + auto re-embed on update.
- v0.26 graph.calibrate() opt-in diagnostic (FTS-only MRR sampling +
per-corpus rerank_blend) — never auto-applied (paraphrase-blind
signal would regress 4/5 quick benches; see CLAUDE.md "Known
limitation: AutoRAG cross-encoder regression").
- v0.27 phrase embedding infrastructure — EntityLinker.link(embedder=...),
inline PhraseExtractor en/ko phrase embedding, EvidenceSearch
_seed_via_phrase_bridges + numpy cache. Bridge default-OFF
(query_phrase_seed_k=0; SYNAPTIC_PHRASE_SEED_K env override) per
MuSiQue ablation measurement — kept as opt-in foundation for
paraphrase-aware / triple-level work.
- examples/ablation/run_tier1_benchmarks.py: --embedder-backend
{ollama|openai} flag; --phrase-seed-k flag.
- eval/run_all.py: --only NAME[,...] dataset filter; auto-passes
embedder to EntityLinker so phrase nodes get embedded by default
on full-pipeline runs.
Test suite: 1088 passed, 2 skipped, lint clean. Build verified
(dist/synaptic_memory-0.27.0-{whl,tar.gz}), smoke install in fresh
venv imports cleanly and exposes the full edit API surface.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
feat(graph): graph.search(engine='evidence') opt-in modern path (v0.1… …5.0) Phase C of the v0.14.x cleanup, RESCOPED from the original "force-migrate graph.search() to EvidenceSearch" plan. PyPI: https://pypi.org/project/synaptic-memory/0.15.0/ ## Why rescoped Tracing every `graph.search()` caller turned up 67 sites across tests/ and eval/, plus features (synonym expansion, query rewriter fallback, resonance-ordering contracts) that the legacy HybridSearch carries and EvidenceSearch does not. A forced migration would silently break every benchmark and every UI that branches on `stages_used == "synonym"`. The user pain that motivated Phase C — magic `cos >= 0.45` cutoff in the legacy path — was already removed in v0.14.1 (relative threshold) and v0.14.2 (MCP route to EvidenceSearch). What was actually still missing: SDK users had no clean way to reach EvidenceSearch without instantiating it themselves. This release adds that path additively, with zero behaviour change for the 67 existing callers. ## What changed - `SynapticGraph.search(query, *, limit=10, embedding=None, engine="legacy")` — new keyword-only `engine` parameter. - `"legacy"` (default) → HybridSearch. Identical to v0.14.x. - `"evidence"` → EvidenceSearch via a new `_search_via_evidence()` adapter that returns a SearchResult (not EvidenceSearchResult) so legacy iteration + sorting + `result.nodes[i].resonance` contracts keep working. - Anything else raises ValueError. - The adapter populates `stages_used = ["evidence", "fts"]` (plus `"vector"` when an embedder is wired). Legacy stages ("synonym", "rewriter") are intentionally absent on the modern path because those steps don't exist in EvidenceSearch. - Adapter forwards `self._embedder`, `self._phrase_extractor`, `self._reranker` into the EvidenceSearch instance — graphs built with `from_data()` get the full modern pipeline with no extra setup. - `Evidence.score` maps to BOTH `ActivatedNode.activation` and `ActivatedNode.resonance` so legacy code that sorts by resonance keeps producing the right order. ## Deprecation timeline - 0.15.0 (this) — `engine="legacy"` default, `engine="evidence"` opt-in - 0.16.0 — default flips to `engine="evidence"`, legacy still available - 0.17.0 — legacy engine removed New code should pass `engine="evidence"` explicitly today. ## Tests `tests/test_search_engine_param.py` (6 new): - Default engine is "legacy". - `engine="legacy"` matches default in node IDs and stages_used. - `engine="evidence"` returns SearchResult with "evidence" in stages_used. - `engine="evidence"` finds the doc with the shared phrase and excludes the unrelated doc from top-2. - Unknown engine name raises ValueError. - `engine="evidence"` preserves descending-resonance ordering. The existing 54 test_search.py + test_graph.py tests pass unchanged because the default path is the same HybridSearch they always exercised. Full suite: 809 passing (up from 803 in v0.14.4). ## Bumps - pyproject.toml + synaptic/__init__.py + synaptic/mcp/__init__.py: 0.14.4 → 0.15.0 (minor — new feature) - CHANGELOG.md: new [0.15.0] section with deprecation timeline - CLAUDE.md: PyPI badge Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
feat(graph): graph.backfill() + knowledge_backfill MCP tool (v0.14.4) Recovery path for the v0.14.x silent-failure modes that landed in v0.14.1 and v0.14.3 fixes. Two distinct gaps used to require a full re-ingest from source to repair: PyPI: https://pypi.org/project/synaptic-memory/0.14.4/ ## Why 1. Empty embeddings — graphs ingested without an embedder stored `Node.embedding=[]`. Wiring an embedder afterwards did NOT retroactively embed those nodes; the HNSW index stayed empty and vector search degraded to "FTS only" on the affected slice. 2. Missing phrase hubs — graphs ingested without a `phrase_extractor` (the default for the MCP server before v0.14.3) had no cross-document bridges. PPR / GraphExpander could not walk across files. Both were "feature is in the code but wiring is missing" silent failures of the same family the v0.14.x series has been chasing. Until now the only fix was re-ingesting from source. ## What `SynapticGraph.backfill()` walks the existing graph in place and repairs each node where the relevant signal is missing, without touching nodes that are already healthy. Idempotent — running twice produces zero work on the second pass. ```python graph = await SynapticGraph.from_data("./old_corpus/", embed_url="...") result = await graph.backfill() print(result.embeddings_filled, result.phrases_linked) ``` MCP tool: `knowledge_backfill(scope="all" | "embeddings" | "phrases", batch_size=64, max_nodes=None)`. Tool count 35 → 36. New `BackfillResult` dataclass exported from `synaptic.models`: - scanned, embeddings_filled, phrases_linked, skipped_no_text, elapsed_ms, errors ## Implementation notes - Embedding pass batches via `embed_batch` (configurable `batch_size`). Phrase pass is per-node since the extractor is already per-passage. - Both passes are best-effort — single-row failures append to `errors` but never abort the rest of the run. - `max_nodes` lets large graphs be processed incrementally. - Skip conditions: - embedding pass: `if node.embedding: continue` - phrase pass: `if any(e.kind == CONTAINS for e in outgoing): continue` - Phrase hubs (tagged `_phrase`) are never re-extracted — would create infinite hubs of hubs. ## Tests `tests/test_backfill.py` (10 new): - `TestEmbeddingBackfill` (4): no-op without embedder, fills missing, idempotent on healthy, skips text-less without crash. - `TestPhraseBackfill` (4): no-op without extractor, creates bridge after wiring, idempotent on healthy, skips phrase-hub nodes (no infinite recursion). - `TestCombinedBackfill` (2): default repairs both, max_nodes limit respected. Includes a `FakeEmbedder` test fixture that returns deterministic 4-dim vectors so the embedding pass can be tested without any network calls. Full suite: 803 passing. ## Bumps - pyproject.toml + synaptic/__init__.py + synaptic/mcp/__init__.py: 0.14.3 → 0.14.4 - CHANGELOG.md: new [0.14.4] section - CLAUDE.md: PyPI badge + tool count 35 → 36 + Knowledge CRUD row Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
fix(mcp): wire PhraseExtractor for cross-document phrase-hub bridges … …(v0.14.3) Real bug observed in the wild: ingesting N files through MCP created N disconnected clusters of nodes that shared no edges. PyPI: https://pypi.org/project/synaptic-memory/0.14.3/ ## Root cause Synaptic implements a HippoRAG2-style dual-node KG: each chunk has its salient phrases extracted and lifted into ENTITY "phrase-hub" nodes. Multiple chunks sharing the same phrase all CONTAINS-edge into the same hub, which makes the hub a bridge between documents. The mechanism is in `PhraseExtractor.extract_and_link()` and fires from `graph.add()` only when a `phrase_extractor` is wired into `SynapticGraph`. `SynapticGraph.from_data()` and `SynapticGraph.full()` always wire one. The MCP server's `_ensure_graph()` factory wired `ChunkEntityIndex` (the read-side index PPR consumes) but **forgot the extractor that populates it**. Result: an empty phrase-hub set, no CONTAINS edges, no bridges. Symptom: ingesting "Project README.md", "Architecture.md", and "Setup.md" — three files that obviously talk about the same project — produced three islands. PPR could not surface cross-document evidence; search degraded to "FTS over disjoint files". The bug had been silently degrading every MCP-driven graph since v0.14.0 added the ingest tools. ## Fix `mcp/server.py:_ensure_graph()` now passes `phrase_extractor=PhraseExtractor()` alongside the existing `chunk_entity_index=ChunkEntityIndex()`. The boot log line gained a `phrase_extractor=on` field so misconfigurations are visible immediately. ## Tests `tests/test_mcp_ingest_tools.py::TestCrossDocumentBridges` (2 new): - `test_shared_phrase_creates_bridge_node`: two documents that both mention "Synaptic Memory" must share at least one phrase-hub ENTITY node reached via CONTAINS from both. - `test_disjoint_documents_have_no_bridge`: a pizza recipe and a quantum tunneling note must NOT spuriously bridge — the phrase hub mechanism is precision-aware, not a "connect everything" hack. Full suite: 793 passing. ## Migration note Existing graphs created with v0.14.0~v0.14.2 through MCP do not have phrase hubs and need to be re-ingested to gain cross-document bridges. There is no in-place backfill yet (related: the embedding-backfill gap from the v0.14.x follow-up plan). Re-ingest from source if you want the bridges. ## Bumps - pyproject.toml + synaptic/__init__.py + synaptic/mcp/__init__.py: 0.14.2 → 0.14.3 - CHANGELOG.md: new [0.14.3] section - CLAUDE.md: PyPI badge Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
feat(mcp): route knowledge_search through EvidenceSearch (v0.14.2) Phase 2 of the magic-number cleanup started in v0.14.1. The MCP `knowledge_search` tool now calls EvidenceSearch directly instead of the legacy `graph.search()` / `HybridSearch` pipeline. PyPI: https://pypi.org/project/synaptic-memory/0.14.2/ ## Why `knowledge_search` was the last MCP tool still pointing at the legacy HybridSearch path. Even with v0.14.1's relative-threshold fix, that path treats vector hits as a *supplement* to FTS via a hardcoded cascade — so the deep tail of the positive distribution on low-cosine embedders (OpenAI v3 small/large, MiniLM) was still partially lost. EvidenceSearch (the engine already backing `agent_search`, `agent_deep_search`, `compare_search`, and the eval bench) has no threshold cutoff at all — it uses min-max normalised cosine in its hybrid reranker, so absolute cosine values disappear from the decision entirely. Routing knowledge_search through it makes the "semantic search returns 0 hits" failure mode impossible by construction. ## What changed - `mcp/server.py:knowledge_search` now constructs `EvidenceSearch` per-call and surfaces: - `reason` ("top_score" / "category_coverage" / "document_quota") - `category` from node properties - `anchors` (categories + entities the extractor pulled from query) - `total_candidates` from EvidenceSearch reranker pool size - `search_time_ms` from EvidenceSearch end-to-end timing - `agent_search` / `agent_deep_search` were already on EvidenceSearch — knowledge_search now matches their behaviour exactly. ## Tests `tests/test_mcp_ingest_tools.py::TestKnowledgeSearch` (4 new tests): - Lexical query still finds the right doc. - Response carries the EvidenceSearch-specific fields (`reason`, `category`) — regression guard against accidental revert. - Empty corpus returns `{success: True, results: []}` with message. - Unrelated query does not put an irrelevant doc at the top. Full suite: 791 passing. ## Stale baseline finding While validating this release with `eval/run_all.py --quick` I discovered that the committed `eval/baselines/qa_latest.json` (last updated under v0.13.0) is stale. The underlying KRRA chunks file was re-parsed on 2026-04-09 and the corpus shape changed. Running the bench at the baseline commit (`d1f229e`) reproduces the *current* numbers (KRRA Easy 0.450, X2BEE Hard 0.263), confirming zero code regression between v0.13.0 and v0.14.2. CHANGELOG documents this so future readers don't chase a phantom regression. The baseline JSON should be regenerated against the current corpus snapshot — separate task. ## Migration scope The legacy HybridSearch path is preserved for backward compatibility. `graph.search()` and the AgentSearch wrapper still use it; only `MCP knowledge_search` moved. No plan to delete the legacy path before v0.16. ## Bumps - pyproject.toml + synaptic/__init__.py + synaptic/mcp/__init__.py: 0.14.1 → 0.14.2 - CHANGELOG.md: new [0.14.2] section - CLAUDE.md: PyPI badge Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
fix(search): embedder-agnostic vector cascade threshold (v0.14.1) Replace the legacy hardcoded `cos >= 0.45` cutoff in HybridSearch with a relative threshold whose floor scales with the embedder's natural cosine distribution. PyPI: https://pypi.org/project/synaptic-memory/0.14.1/ ## The bug `search.py:201` had `if nid not in fts_ids and cos >= 0.45`. That number was tuned in 2026-03-26 against bge-m3-style models where true positives sit at cosine 0.55+. It was never benchmarked afterward — `eval/run_all.py` always routes through EvidenceSearch when an embedder is wired, so the legacy path's tuning rotted in silence. When users ran the same code with OpenAI text-embedding-3-small / 3-large the cosine distribution was much lower (p50 ≈ 0.40, p75 ≈ 0.48), and the absolute 0.45 cutoff silently rejected 50–75% of true positives. Symptom: pure-semantic queries returned zero results unless they happened to share at least one keyword with the document. Field measurement that triggered the fix: | query (text-embedding-3-small) | cos | 0.45 cutoff | |---|---|---| | `purchase return policy` | 0.515 | ✅ | | `money back guarantee` | 0.393 | ❌ | | `shipping time` | 0.447 | ❌ (0.003 below) | | `delivery` | 0.296 | ❌ | ## The fix ```python floor = max(vector_min_cosine, top_cos * (1 - vector_relative_drop)) ``` Defaults: `vector_min_cosine=0.10`, `vector_relative_drop=0.30`. Effective floor scales automatically per embedder family: | Embedder | top hit | floor | |---|---|---| | bge-m3 / qwen3-embedding-4b | ~0.80 | ~0.56 | | multilingual-e5 | ~0.85 | ~0.595 | | **text-embedding-3-small** | **~0.55** | **~0.385** | | text-embedding-3-large | ~0.62 | ~0.434 | The same fixture returns the same number of vector candidates on every embedder family. Magic number is gone — replaced by a ratio that is portable across cosine distributions. ## Override hierarchy 1. `HybridSearch(vector_min_cosine=, vector_relative_drop=)` 2. `SynapticGraph(vector_min_cosine=, vector_relative_drop=)` 3. `synaptic-mcp --vector-min-cosine 0.10 --vector-relative-drop 0.30` 4. `SYNAPTIC_VECTOR_MIN_COSINE` / `SYNAPTIC_VECTOR_RELATIVE_DROP` environment variables 5. The defaults above `_resolve_float()` (module-level helper) handles the layered fallback and is robust to garbage env values. ## Tests `tests/test_hybrid_search_threshold.py` (13 new tests): - `TestOverrideHierarchy`: default → constructor → env var → garbage env handling. - `TestVectorCascadeAgnostic`: - The same fixture under "bge-shape" (cosines 0.30–0.85) and "openai-shape" (0.20–0.55) returns the **same** count of vector candidates. This is the embedder-agnostic property. - `test_top_hit_always_passes` — recall regression guard. - `test_legacy_threshold_would_have_failed_openai` — explicit regression test using a 0.44 cosine that the old hardcoded cutoff would have rejected. Documents the bug being fixed. - `TestAbsoluteFloor`: noise floor kicks in when even the top hit is weak. Floor can be disabled by setting `min_cosine=0`. - `TestRelativeDropTuning`: drop=0 (only the top hit) and drop=1 (everything above abs floor) corner cases. The test fixtures use 2-D unit vectors `(cos θ, sin θ)` with query `(1, 0)` so the cosine of every fixture node is authored directly — no embedder needed, deterministic on every machine. Full suite: 787 passing. ## Scope note This only fixes the legacy `HybridSearch` / `graph.search()` / `MCP knowledge_search` path. `agent_search`, `agent_deep_search`, `compare_search`, and the eval bench all use `EvidenceSearch` which never had this issue (it uses min-max normalised cosine). Phase 2 (next PR) will migrate `knowledge_search` to use `EvidenceSearch` so the magic number disappears entirely. ## Bumps - pyproject.toml + synaptic/__init__.py + synaptic/mcp/__init__.py: 0.14.0 → 0.14.1 - CHANGELOG.md: new [0.14.1] section - CLAUDE.md: PyPI badge Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
release: v0.14.0 — Live database CDC + MCP ingest tools PyPI: https://pypi.org/project/synaptic-memory/0.14.0/ ## Headline features - **Live database CDC** — `from_database(mode="cdc")` + `sync_from_database()` for incremental sync. X2BEE production PostgreSQL validation: 35s full reload → 6s incremental (~6× faster), zero search-quality regression. - **MCP ingest + CDC tools** — 6 new tools (29 → 35) so Claude can ingest documents / tables / chunks / files and run CDC syncs without dropping to a CLI. ## Bumps - pyproject.toml: 0.13.0 → 0.14.0 - src/synaptic/__init__.py: 0.13.0 → 0.14.0 - src/synaptic/mcp/__init__.py: 0.12.0 → 0.14.0 (was stale) - CHANGELOG.md: Unreleased → [0.14.0] - 2026-04-14 - CLAUDE.md: PyPI badge + "Ingest / CDC 도구 (v0.14.0+)" subsection Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
PreviousNext