aaron/aaronAI
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

stage2_worker: ON CONFLICT clause resets all run-state fields on re-enqueue

Browse Source 
Bug: when a row in stage_3_queue gets re-enqueued (same source ingested
again after Stage 2 re-runs), the ON CONFLICT (source) DO UPDATE clause
updated content fields and reset enqueued_at, completed_at, failed_at,
attempts — but did not reset started_at, failure_reason, or
external_job_id.

Stale started_at from a prior attempt makes the row invisible to the
Stage 3 worker's claim filter (which uses started_at IS NULL). The row
sits queued forever; Stage 3 never picks it up; the source effectively
fails silently after a re-trigger.

Discovered tonight while testing the bulk pathway after the substrate
fix: a journal entry that had been ingested earlier (and manually marked
completed during recovery from a worker timeout) showed enqueued_at
from the new touch but started_at from the original 01:40 attempt. Fix
extends the upsert clause to NULL all run-state fields so re-enqueue
behaves as 'fresh attempt.'

After fix, re-triggered journal entry routed cleanly through Stage 2 →
Stage 3 → bulk pathway → sidecar bulk job → 60ms commit (worst-case
dedup against already-known content).
This commit is contained in:
Aaron Nelson
2026-05-02 05:20:14 +00:00
parent f645b74b1c
commit 7f07972109
1 changed files with 7 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/stage2_worker.py
+7
View File
@@ -220,8 +220,15 @@ def enqueue_stage3(pg, source, full_text, orientation, metadata,
supersedes_prior_state = EXCLUDED.supersedes_prior_state, supersedes_prior_state = EXCLUDED.supersedes_prior_state,
state_type_rationale = EXCLUDED.state_type_rationale, state_type_rationale = EXCLUDED.state_type_rationale,
enqueued_at = NOW(), enqueued_at = NOW(),
-- Reset all run-state fields on re-enqueue. Without this,
-- stale started_at from a prior attempt makes the row
-- invisible to the Stage 3 worker's claim filter (which
-- typically uses started_at IS NULL).
started_at = NULL,
completed_at = NULL, completed_at = NULL,
failed_at = NULL, failed_at = NULL,
failure_reason = NULL,
external_job_id = NULL,
attempts = 0 attempts = 0
""", (source, full_text, orientation, json.dumps(metadata), """, (source, full_text, orientation, json.dumps(metadata),
state_type, state_type_confidence, supersedes_prior_state, state_type, state_type_confidence, supersedes_prior_state,