aaron
e96bf40b2f
After establishing that single-episode Graphiti writes take ~20 min against
the existing graph (the dedup loop is structurally slow regardless of the
patches, the bridge, or the LLM model), the salvage plan is to stop trying
to write to Graphiti and instead:
1. Use the existing 4,300-entity graph as a read-only fact layer at chat
time via a new search_facts tool. Graphiti's /search endpoint is fast
(~15ms direct, ~400ms over HTTP); the graph is stale-as-of-early-May
but covers most biographical / relational content that "write me a bio"
and similar queries care about.
2. Pipe Stage 2's document-level orientations into pgvector via a new
orientation_indexer worker. Stage 2 already runs and writes orientation
text to stage_3_queue for every Mistral-processed document; the worker
reads those, embeds them, and writes one row per source to embeddings
with metadata->>'kind'='orientation'. retrieve_documents now ranks
against both chunk text and document-level concept summaries.
Idempotent: the indexer's "is this already indexed" check is an EXISTS
subquery against embeddings, so restarts and partial runs are safe.
Out of scope (deliberately): no Graphiti writes from chat, no Stage 2 ->
Graphiti bridge, no draining the 711-item stage_3_queue backlog into
Graphiti. Rich-extraction posture stays a BirdAI concern.