aaron
9e86297e2a
Removes classify_retrieval_intent and the type/folder filter parameters on retrieve_context. The keyword classifier was the same anti-pattern as the formatting-driven docx chunker: a heuristic that locks the user into specific phrasings and fails silently on anything novel. A scope enum (personal / library / conversations / memory) would have been the same heuristic in a fancier wrapper — the categories themselves are mine, not Aaron's. New shape: a retrieve_documents tool exposed to Claude. Tool takes a single query argument; the model decides when to call it, what to search for, and how many times per turn (multi-query falls out naturally for compound asks). Pre-LLM retrieval is gone — memory still rides as ground truth in the prompt, but corpus content is fetched on demand by the model with concrete queries it crafts itself, not the user's raw phrasing. retrieve_context is now pure: hybrid retrieval + cross-encoder rerank + dedup, no filters. The reranker ranks, the model judges relevance. When ranking fails (e.g. abstract instructional queries pulling philosophy books), the right fix is a better reranker, not another query-time taxonomy. That work is acknowledged but deferred. System prompt updated to teach the model about the tool and to prefer concrete tokens (named entities, project names, course codes) over abstract phrasing when constructing search queries.
54 lines
1.7 KiB
Python
54 lines
1.7 KiB
Python
"""End-to-end test of retrieve_context with intent routing + reranking.
|
|
|
|
Avoids loading the full FastAPI app; replicates the chat-handler retrieval
|
|
call shape and prints classifier output + final ranked sources for each query.
|
|
"""
|
|
|
|
import os
|
|
import sys
|
|
from pathlib import Path
|
|
|
|
from dotenv import load_dotenv
|
|
load_dotenv(Path.home() / "aaronai" / ".env", override=True)
|
|
|
|
sys.path.insert(0, str(Path(__file__).parent))
|
|
|
|
# Stub anthropic so api.py import doesn't fail without the SDK loaded.
|
|
# We only need retrieve_context.
|
|
import types
|
|
sys.modules.setdefault("anthropic", types.ModuleType("anthropic"))
|
|
sys.modules["anthropic"].Anthropic = lambda **kw: None
|
|
|
|
# Same for whisper if present
|
|
if "faster_whisper" not in sys.modules:
|
|
sys.modules["faster_whisper"] = types.ModuleType("faster_whisper")
|
|
|
|
import importlib.util
|
|
spec = importlib.util.spec_from_file_location("api", Path(__file__).parent / "api.py")
|
|
api = importlib.util.module_from_spec(spec)
|
|
# Don't execute the whole module (it starts FastAPI). Instead, exec only definitions.
|
|
# Easier: just import the functions we need by exec'ing the file but catching errors.
|
|
try:
|
|
spec.loader.exec_module(api)
|
|
except Exception as e:
|
|
print(f"(continuing despite api.py side-effect error: {e})")
|
|
|
|
retrieve_context = api.retrieve_context
|
|
|
|
QUERIES = [
|
|
"write me a bio",
|
|
"my professional bio",
|
|
"Aaron Nelson CV consulting and design work",
|
|
"FWN3D consulting",
|
|
"syllabi I have taught",
|
|
"philosophy of teaching",
|
|
"Hudson Valley Additive Manufacturing Center",
|
|
"Aaron Nelson is an artist and educator working in additive manufacturing",
|
|
]
|
|
|
|
for q in QUERIES:
|
|
pieces, sources = retrieve_context(q)
|
|
print(f"\n=== {q!r} ===")
|
|
for i, src in enumerate(sources, 1):
|
|
print(f" {i}. {src}")
|