Replace Batch API vision classification with inline classification in the fetch job #84

New issue

Closed

opened 2026-08-13 07:53:07 +00:00 by agent-pi · 1 comment

agent-pi commented

2026-08-13 07:53:07 +00:00

Collaborator

PR #81 shipped vision classification as a nightly Batch API job (ai_batch.py + an ai_vision_batch cron). That should change: the Batch API gives no completion-time guarantee, so "classified by 5am" can't be met reliably.

Why. Moonshot's Batch API only guarantees a job expires if it doesn't finish within completion_window (12h/1d/3d); it makes no promise about when within that window it runs (docs describe it as for "low real-time requirements"). Empirically, a 2-image test batch sat in_progress for 5+ hours with 0/2 done. The 40% batch discount is ~$0.24/week at our volume — not worth losing deterministic completion.

Decision. Classify images inline in the stream update job, immediately after the text check:

save → auto-reject → text check → vision check (first image) → commit

Work items.

Remove the Batch API machinery (ai_batch.py JSONL/sharding/upload/poll) and the ai_vision_batch cron job.
Add an inline vision step to the fetch job (after _classify_fetched_text), classifying the first image of each newly-fetched, text-passing post via VisionClassifier.classify() (real-time single request).
Remove the now-unneeded AI_BATCH_HOUR setting.
Fix "couldn't check" semantics: a failed image download / API call must leave ai_would_reject = NULL ("pending"), not False ("pass") — the current ai_batch.py sets False on download failure.
Update docs and the .superpowers spec/plan to drop Batch API references.

Agreed consequences.

The ~28k existing un-reviewed backlog stays unclassified (AI is forward-looking); a one-off backfill script can cover it later if wanted.
Transient failures leave a post "pending" with no automatic retry; it's reviewed manually.
Fetch job gets a few minutes slower per cycle (async, background, ≥15min intervals) — acceptable.

PR #81 shipped vision classification as a nightly **Batch API** job (`ai_batch.py` + an `ai_vision_batch` cron). That should change: the Batch API gives no completion-time guarantee, so "classified by 5am" can't be met reliably. **Why.** Moonshot's Batch API only guarantees a job *expires* if it doesn't finish within `completion_window` (12h/1d/3d); it makes no promise about *when* within that window it runs (docs describe it as for "low real-time requirements"). Empirically, a 2-image test batch sat `in_progress` for 5+ hours with 0/2 done. The 40% batch discount is ~$0.24/week at our volume — not worth losing deterministic completion. **Decision.** Classify images inline in the stream update job, immediately after the text check: save → auto-reject → text check → vision check (first image) → commit **Work items.** 1. Remove the Batch API machinery (`ai_batch.py` JSONL/sharding/upload/poll) and the `ai_vision_batch` cron job. 2. Add an inline vision step to the fetch job (after `_classify_fetched_text`), classifying the first image of each newly-fetched, text-passing post via `VisionClassifier.classify()` (real-time single request). 3. Remove the now-unneeded `AI_BATCH_HOUR` setting. 4. Fix "couldn't check" semantics: a failed image download / API call must leave `ai_would_reject = NULL` ("pending"), not `False` ("pass") — the current `ai_batch.py` sets `False` on download failure. 5. Update docs and the `.superpowers` spec/plan to drop Batch API references. **Agreed consequences.** - The ~28k existing un-reviewed backlog stays unclassified (AI is forward-looking); a one-off backfill script can cover it later if wanted. - Transient failures leave a post "pending" with no automatic retry; it's reviewed manually. - Fetch job gets a few minutes slower per cycle (async, background, ≥15min intervals) — acceptable.

agent-pi commented

2026-08-13 07:53:21 +00:00

Author

Collaborator

Files that reference the batch approach (for when this is picked up):

.superpowers/specs/2026-08-13-fenliu-vision-ai-design.md — sections 5 (architecture diagram) and 8 (ai_batch.py), plus the "vision-in-batch unconfirmed" risk note.
.superpowers/plans/2026-08-13-fenliu-vision-ai.md — Task 5 (ai_batch.py) and Task 6 (scheduler wiring).
packages/fenliu/src/fenliu/services/ai_batch.py — the batch processor (JSONL build/shard/upload/poll) to be replaced by a simple inline classify loop.
packages/fenliu/src/fenliu/services/scheduler.py — _schedule_ai_batch_job / _ai_batch_job cron to be removed; the vision step moves into _fetch_stream_job after _classify_fetched_text.
packages/fenliu/src/fenliu/config.py — ai_batch_hour to be removed.
packages/fenliu/docs/user-guide/ai-classification.md — batch references to update.

Files that reference the batch approach (for when this is picked up): - `.superpowers/specs/2026-08-13-fenliu-vision-ai-design.md` — sections 5 (architecture diagram) and 8 (ai_batch.py), plus the "vision-in-batch unconfirmed" risk note. - `.superpowers/plans/2026-08-13-fenliu-vision-ai.md` — Task 5 (`ai_batch.py`) and Task 6 (scheduler wiring). - `packages/fenliu/src/fenliu/services/ai_batch.py` — the batch processor (JSONL build/shard/upload/poll) to be replaced by a simple inline classify loop. - `packages/fenliu/src/fenliu/services/scheduler.py` — `_schedule_ai_batch_job` / `_ai_batch_job` cron to be removed; the vision step moves into `_fetch_stream_job` after `_classify_fetched_text`. - `packages/fenliu/src/fenliu/config.py` — `ai_batch_hour` to be removed. - `packages/fenliu/docs/user-guide/ai-classification.md` — batch references to update.

agent-pi referenced this issue from a pull request that will close it,

2026-08-13 09:26:38 +00:00

Classify images inline instead of via batch API #86