Remove XGBoost ML and spam scorer; add AI image/text classification #81

Manually merged

marvin8 merged 10 commits from feat/issue-80-remove-ml-add-vision-ai into feat/issue-80-collect 2026-08-13 08:54:52 +00:00

Conversation 5 Commits 10 Files changed 95 +1352 -7760

agent-pi commented 2026-08-13 05:34:31 +00:00

Collaborator

Copy link

Removes Fenliu's XGBoost ML integration and rule-based spam scorer, and replaces them with a two-stage AI classifier:

• Text stage (inline, at fetch): flags posts whose status/alt text uses a cat as an excuse to promote something.
• Vision stage (overnight batch via the Moonshot Batch API, 60% price): classifies the first image as a real-cat photograph vs comic/art/mural/meme/etc.

The result is an ai_would_reject recommendation surfaced as a badge + filters in the review UI. No auto-reject at launch (behind the ai_auto_reject setting, default off). Model is configurable via env vars (AI_API_BASE_URL, AI_API_KEY, AI_TEXT_MODEL, AI_VISION_MODEL, AI_BATCH_HOUR).

Also:

• Removes the dead score field from Zhongli's QueuedPost (it was fed by Fenliu's removed spam_score).
• Fixes leftover migration source references (zaojun pre-commit hook + wuming CI action → forge.marvin8.zone).

Closes #80

Removes Fenliu's XGBoost ML integration and rule-based spam scorer, and replaces them with a two-stage AI classifier: - **Text stage** (inline, at fetch): flags posts whose status/alt text uses a cat as an excuse to promote something. - **Vision stage** (overnight batch via the Moonshot Batch API, 60% price): classifies the first image as a real-cat photograph vs comic/art/mural/meme/etc. The result is an `ai_would_reject` recommendation surfaced as a badge + filters in the review UI. No auto-reject at launch (behind the `ai_auto_reject` setting, default off). Model is configurable via env vars (`AI_API_BASE_URL`, `AI_API_KEY`, `AI_TEXT_MODEL`, `AI_VISION_MODEL`, `AI_BATCH_HOUR`). Also: - Removes the dead `score` field from Zhongli's `QueuedPost` (it was fed by Fenliu's removed `spam_score`). - Fixes leftover migration source references (zaojun pre-commit hook + wuming CI action → forge.marvin8.zone). Closes #80

agent-pi added 9 commits 2026-08-13 05:34:31 +00:00

🔧 point source references at forge.marvin8.zone 2d481d3343

🔥 remove XGBoost ML and spam scorer 31c716b3ea

✨ add AI classification settings fdcec0f173

✨ add AI text and vision classifiers c65e1a63be

🗃️ add AI classification columns 02a2897cc0

✨ add overnight batch vision processor 2b3b50473b

✨ wire AI classification into scheduler 80d22ca33c

✨ add AI badge and filters to review UI 5246c44317

📝 document AI classification and update release notes c7cb138ef4

forgejo-actions commented 2026-08-13 05:34:39 +00:00

Copy link

This PR is too large to review automatically (11947 lines; limit is 3000). Please consider splitting it into smaller PRs.

This PR is too large to review automatically (11947 lines; limit is 3000). Please consider splitting it into smaller PRs.

forgejo-actions commented 2026-08-13 05:34:39 +00:00

Copy link

This PR is too large to review automatically (11947 lines; limit is 3000). Please consider splitting it into smaller PRs.

This PR is too large to review automatically (11947 lines; limit is 3000). Please consider splitting it into smaller PRs.

agent-pi referenced this pull request 2026-08-13 07:39:46 +00:00

Alembic autogenerate emits spurious schema drift; reconcile model vs database #83

agent-pi added 1 commit 2026-08-13 07:40:08 +00:00

📝 track stale review.html cleanup in TODO fd7669a0eb

forgejo-actions commented 2026-08-13 07:40:16 +00:00

Copy link

This PR is too large to review automatically (11948 lines; limit is 3000). Please consider splitting it into smaller PRs.

This PR is too large to review automatically (11948 lines; limit is 3000). Please consider splitting it into smaller PRs.

forgejo-actions commented 2026-08-13 07:40:17 +00:00

Copy link

This PR is too large to review automatically (11948 lines; limit is 3000). Please consider splitting it into smaller PRs.

This PR is too large to review automatically (11948 lines; limit is 3000). Please consider splitting it into smaller PRs.

agent-pi referenced this pull request 2026-08-13 07:53:07 +00:00

Replace Batch API vision classification with inline classification in the fetch job #84

marvin8 changed target branch from main to feat/issue-80-collect 2026-08-13 08:24:54 +00:00

agent-pi referenced this pull request 2026-08-13 08:36:08 +00:00

Vestigial reviewer_notes field: rename, deduplicate, or add real UI #85

marvin8 approved these changes 2026-08-13 08:52:07 +00:00

marvin8 manually merged commit 17bc44c5a7 into feat/issue-80-collect 2026-08-13 08:54:52 +00:00

marvin8 deleted branch feat/issue-80-remove-ml-add-vision-ai 2026-08-13 20:30:54 +00:00

agent-pi referenced this pull request 2026-08-14 03:15:17 +00:00

Remove XGBoost ML + spam scorer; add image-aware AI curation #80

agent-pi referenced this pull request 2026-08-14 04:35:14 +00:00

Remove XGBoost ML + spam scorer; add image-aware AI curation #93

Sign in to join this conversation.

Reviewers

No reviewers

marvin8

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

3 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

marvin8/dujiangyan!81

No description provided.