aaron
4204806c80
Both databases ran with journal_mode=delete — every write rewrote the rollback journal per transaction. WAL eliminates the journal-rewrite and lets readers run without blocking writers. Index on messages(conversation_id, timestamp DESC) is preventive — only 280 rows today, but the access pattern (load conversation history in order) is exactly what a composite index serves, and we don't want to re-revisit this when the table grows. backup.sh updated in the same commit because WAL changes the on-disk layout: a bare `cp` of just the .db file can miss recently-committed transactions that still live in the -wal sidecar, and can race with concurrent writes to produce a torn file. Switched to the SQLite Online Backup API via python3 -c "...src.backup(dst)..." — same mechanism as the sqlite3 CLI's `.backup` (which isn't installed on this host), handles WAL correctly without forcing a checkpoint, and is non-locking from the writer's perspective. Verified backup integrity_check returns ok and row counts match. Note: synchronous=NORMAL was considered but deferred — it's a per-connection PRAGMA, and applying it correctly requires a connect helper that wraps every sqlite3.connect() call site in api.py (~14 sites). Out of scope for this commit; tracked as a follow-up. WAL alone delivers the journal-rewrite elimination and reader/writer concurrency improvements; the additional fsync reduction from synchronous=NORMAL is a smaller marginal win on top. Confirmed via concurrency audit that api.py is the sole writer to both databases. ingest_conversations.py and dream.py are read-only consumers of conversations.db; nothing else touches sessions.db.
17 lines
668 B
Bash
Executable File
17 lines
668 B
Bash
Executable File
#!/bin/bash
|
|
BACKUP_DIR="/home/aaron/nextcloud/data/data/aaron/files/Admin/Backups"
|
|
DATE=$(date +%Y-%m-%d)
|
|
mkdir -p "$BACKUP_DIR"
|
|
|
|
# Copy critical files
|
|
cp ~/aaronai/memory.md "$BACKUP_DIR/memory-$DATE.md"
|
|
cp ~/aaronai/settings.json "$BACKUP_DIR/settings-$DATE.json"
|
|
python3 -c "import sqlite3, sys; src = sqlite3.connect('$HOME/aaronai/conversations.db'); dst = sqlite3.connect('$BACKUP_DIR/conversations-$DATE.db'); src.backup(dst); dst.close(); src.close()"
|
|
|
|
# Keep only last 7 days
|
|
find "$BACKUP_DIR" -name "*.md" -mtime +7 -delete
|
|
find "$BACKUP_DIR" -name "*.json" -mtime +7 -delete
|
|
find "$BACKUP_DIR" -name "*.db" -mtime +7 -delete
|
|
|
|
echo "Backup complete: $DATE"
|