Implements the rest of dreamer-design-spec.md's Stage 3 alongside the
prescriptions from the external literature review:
- Hardcoded seed query strings are gone. _llm_generate_queries() produces
4 mode-appropriate retrieval queries per call from the observation signal
(Park et al. 2023 reflection pattern). NREM queries probe RECENT additions;
Early REM bridges associative/emotional threads; Late REM forces cross-
domain pairs; Lucid decomposes the task. Empirical first-run output:
queries like "SUNY New Paltz Fall 2026 registration moratorium" instead of
the fixed "research fabrication teaching practice recent work" — vector
neighborhood now drifts with what the user has been actually doing.
- TIME_WINDOWS_HOURS makes per-mode retrieval windows mutable
(dreamer-multimodal-design.md §2's tech-debt item): NREM 72hr / Early REM
30d / Late REM 90d / Lucid no-window. NULL created_at rows are excluded
from windowed modes — correct since they predate the cursor by definition.
- NREM bias toward under-processed chunks via "ORDER BY consolidation_count
ASC" before vector distance. Biologically motivated: sharp-wave-ripple
replay is tagged/biased, not uniform. Chunks that haven't been replayed
recently win the tiebreak.
- MMR merge (Carbonell & Goldstein 1998) over the union of all queries'
candidates. λ=0.5. Directly attacks the cluster-dominance failure mode
where 8 dossier-narrative variants filled all 8 slots in 5 consecutive
nights.
- _bump_consolidation_cursor() called after NREM completes. Each source
used gets consolidation_count += 1 and last_consolidated_at = NOW().
Tomorrow's signal sees these as more-processed, less under-processed.
- dream_pipeline now runs observe_corpus + select_mode at the top per spec
lines 27-34. If select_mode returns None — corpus unchanged + no new
journal entry — pipeline exits with no dream rather than manufacturing
novelty (spec line 67's "dreamer goes quiet").
Back-compat preserved:
- retrieve()'s signature gains `signal` as optional kwarg; default behavior
calls observe_corpus() inline so dream_single / dream_lucid keep working
unchanged.
- Graphiti substrate (E3 experiment) path untouched.
- Manifest schema keeps the "query" field; value is now
"[llm-generated from observation signal]" so historical manifest
consumers don't break.
aaron
f185ed60cb