Changing a stream's fetch interval skips its immediate fetch run #111

New issue

Open

opened 2026-09-08 07:28:57 +00:00 by agent-pi · 1 comment

agent-pi commented

2026-09-08 07:28:57 +00:00

Collaborator

Symptom

Every time a stream's fetch interval is edited in the UI, the scheduler logs Execution of job "StreamScheduler._fetch_stream_job" skipped: maximum number of running instances reached (1) and the stream does not poll again until a full new interval has elapsed.

Observed 8 times across two testing sessions (2026-09-08, fenliu-all.log), each skip second-aligned with an interval edit — e.g. six skips while streams 3–8 were all set to 48 minutes in the 17:00 session.

Impact

Not a crash and no data loss, but every interval edit wastes the immediate fetch cycle: a user shortening an interval expecting a prompt next poll waits a full interval instead. With long intervals (hours) that delay is very noticeable.

## Symptom Every time a stream's fetch interval is edited in the UI, the scheduler logs `Execution of job "StreamScheduler._fetch_stream_job" skipped: maximum number of running instances reached (1)` and the stream does not poll again until a full new interval has elapsed. Observed 8 times across two testing sessions (2026-09-08, `fenliu-all.log`), each skip second-aligned with an interval edit — e.g. six skips while streams 3–8 were all set to 48 minutes in the 17:00 session. ## Impact Not a crash and no data loss, but every interval edit wastes the immediate fetch cycle: a user shortening an interval expecting a prompt next poll waits a full interval instead. With long intervals (hours) that delay is very noticeable.

agent-pi commented

2026-09-08 07:29:13 +00:00

Author

Collaborator

Findings from the log analysis (both 2026-09-08 sessions):

The skip is deterministic, 1:1 with interval edits: each occurrence lines up second-for-second with remove_job stream_N + _real_add_job (the StreamScheduler._adapt_schedule path) plus the UPDATE hashtag_streams SET fetch_interval_minutes=... for that stream.
Mechanism: when the job is re-added with the new interval, the new trigger's computed next run lands immediately ("now"), while the old instance for the same stream is still being torn down — APScheduler's max_instances=1 rejects it. The success log of the old instance and the skip appear within the same event batch.
Cost: exactly one missed immediate run per edit; the next poll happens a full new interval later. Repeated edits (as during testing) repeat the skip.

Plan of attack (sketch)

Decide the desired semantics: on interval change, either (a) don't schedule an immediate run — next poll at now + new interval, or (b) run immediately once the previous instance has drained. Either is defensible; (a) is simpler and matches APScheduler's model.
Adjust _adapt_schedule accordingly (e.g. pass an explicit next_run_time/start_date when re-adding, or set the trigger start_date forward).
Cover with a test that changes a stream's interval while a fetch is marked running and asserts no skipped: maximum instances warning and a next run consistent with the chosen semantics.

Findings from the log analysis (both 2026-09-08 sessions): - The skip is deterministic, 1:1 with interval edits: each occurrence lines up second-for-second with `remove_job stream_N` + `_real_add_job` (the `StreamScheduler._adapt_schedule` path) plus the `UPDATE hashtag_streams SET fetch_interval_minutes=...` for that stream. - Mechanism: when the job is re-added with the new interval, the new trigger's computed next run lands immediately ("now"), while the old instance for the same stream is still being torn down — APScheduler's `max_instances=1` rejects it. The success log of the old instance and the skip appear within the same event batch. - Cost: exactly one missed immediate run per edit; the next poll happens a full new interval later. Repeated edits (as during testing) repeat the skip. ## Plan of attack (sketch) 1. Decide the desired semantics: on interval change, either (a) don't schedule an immediate run — next poll at now + new interval, or (b) run immediately once the previous instance has drained. Either is defensible; (a) is simpler and matches APScheduler's model. 2. Adjust `_adapt_schedule` accordingly (e.g. pass an explicit `next_run_time`/start_date when re-adding, or set the trigger start_date forward). 3. Cover with a test that changes a stream's interval while a fetch is marked running and asserts no `skipped: maximum instances` warning and a next run consistent with the chosen semantics.

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

marvin8/dujiangyan#111

No description provided.

Rows
Columns