conversations.db, sessions.db: enable WAL, add message index; update backup.sh

Both databases ran with journal_mode=delete — every write rewrote the
rollback journal per transaction. WAL eliminates the journal-rewrite and
lets readers run without blocking writers.

Index on messages(conversation_id, timestamp DESC) is preventive — only
280 rows today, but the access pattern (load conversation history in
order) is exactly what a composite index serves, and we don't want to
re-revisit this when the table grows.

backup.sh updated in the same commit because WAL changes the on-disk
layout: a bare `cp` of just the .db file can miss recently-committed
transactions that still live in the -wal sidecar, and can race with
concurrent writes to produce a torn file. Switched to the SQLite Online
Backup API via python3 -c "...src.backup(dst)..." — same mechanism as
the sqlite3 CLI's `.backup` (which isn't installed on this host),
handles WAL correctly without forcing a checkpoint, and is non-locking
from the writer's perspective. Verified backup integrity_check returns
ok and row counts match.

Note: synchronous=NORMAL was considered but deferred — it's a
per-connection PRAGMA, and applying it correctly requires a connect
helper that wraps every sqlite3.connect() call site in api.py (~14
sites). Out of scope for this commit; tracked as a follow-up. WAL alone
delivers the journal-rewrite elimination and reader/writer concurrency
improvements; the additional fsync reduction from synchronous=NORMAL is
a smaller marginal win on top.

Confirmed via concurrency audit that api.py is the sole writer to both
databases. ingest_conversations.py and dream.py are read-only consumers
of conversations.db; nothing else touches sessions.db.

This commit is contained in:

Aaron Nelson

2026-05-04 03:24:51 +00:00

parent c5fc517fef

commit 4204806c80

2 changed files with 4 additions and 1 deletions

									
										scripts/backup.sh
									
		+1
		-1
	
												View File
												
				@@ -6,7 +6,7 @@ mkdir -p "$BACKUP_DIR"

				# Copy critical files

				cp ~/aaronai/memory.md "$BACKUP_DIR/memory-$DATE.md"

				cp ~/aaronai/settings.json "$BACKUP_DIR/settings-$DATE.json"

				cp ~/aaronai/conversations.db "$BACKUP_DIR/conversations-$DATE.db"

				python3 -c "import sqlite3, sys; src = sqlite3.connect('$HOME/aaronai/conversations.db'); dst = sqlite3.connect('$BACKUP_DIR/conversations-$DATE.db'); src.backup(dst); dst.close(); src.close()"

				# Keep only last 7 days

				find "$BACKUP_DIR" -name "*.md" -mtime +7 -delete