Generalize AI classification from cat-specific to domain-agnostic #100

Closed
opened 2026-08-14 22:28:53 +00:00 by agent-pi · 6 comments
Collaborator

Fenliu's AI classification pipeline is hard-coded to a single use case: curating real-cat photographs and rejecting posts that use a cat as an excuse to promote something. The vision stage asks "is this a real-cat photograph", the text stage asks "is this cat-as-excuse promotion", and those rules are baked into the hard-coded prompts (TEXT_PROMPT / VISION_PROMPT in services/ai_classification.py) and the field names (vision_is_real_cat, text_is_promotional).

As a result Fenliu can no longer be pointed at any other curation topic without code changes — it went from a generic curation tool to a cat-specific one.

Fenliu's AI classification pipeline is hard-coded to a single use case: curating real-cat photographs and rejecting posts that use a cat as an excuse to promote something. The vision stage asks "is this a real-cat photograph", the text stage asks "is this cat-as-excuse promotion", and those rules are baked into the hard-coded prompts (`TEXT_PROMPT` / `VISION_PROMPT` in `services/ai_classification.py`) and the field names (`vision_is_real_cat`, `text_is_promotional`). As a result Fenliu can no longer be pointed at any other curation topic without code changes — it went from a generic curation tool to a cat-specific one.
Author
Collaborator

Findings from exploring the AI classification code:

Cat-specificity lives in a few places, not just the prompts:

  • PromptsTEXT_PROMPT / VISION_PROMPT are module-level constants in services/ai_classification.py; the classifiers reference them directly.
  • Field namesvision_is_real_cat is the clearly cat-shaped column (a generic name would be vision_accepted). text_is_promotional is borderline ("promotion" is one specific rule; full neutrality would be text_flagged).
  • Derive stepderive_ai_would_reject(text_is_promotional, vision_is_real_cat) is already structurally generic (reject = text-flag OR not-vision-accept; "no image → accept"; "unclassified → None"). The only real policy wrinkle is the "no image → accept" default, which a generic tool might want configurable.
  • UI — mostly neutral already: the badge renders "AI: reject — {category}" / "AI: OK" / "AI: pending", and vision_category is a free string. Only the "promotional" fallback literal is cat-adjacent.

Proposed decomposition (natural sub-PR sequence on the collecting branch):

  1. Extract prompts into settings — configurable text/vision prompts on the settings page (string-setting helper + two textareas); keep current constants as defaults. No schema change. Highest value-per-effort.
  2. Rename verdict fields to domain-neutral names (vision_is_real_catvision_accepted, optionally text_is_promotionaltext_flagged) + one migration + schema-reconciliation test.
  3. Optional polish — configurable "no image" accept/reject policy, reword the "promotional" badge fallback, docs.

Estimate: ~15–20 files + 1 migration total; the effort concentrates in the settings-page UI (multi-line string inputs on a bool-only form) and pinning down the generic semantics, not the renames themselves.

Findings from exploring the AI classification code: Cat-specificity lives in a few places, not just the prompts: - **Prompts** — `TEXT_PROMPT` / `VISION_PROMPT` are module-level constants in `services/ai_classification.py`; the classifiers reference them directly. - **Field names** — `vision_is_real_cat` is the clearly cat-shaped column (a generic name would be `vision_accepted`). `text_is_promotional` is borderline ("promotion" is one specific rule; full neutrality would be `text_flagged`). - **Derive step** — `derive_ai_would_reject(text_is_promotional, vision_is_real_cat)` is already structurally generic (reject = text-flag OR not-vision-accept; "no image → accept"; "unclassified → None"). The only real policy wrinkle is the "no image → accept" default, which a generic tool might want configurable. - **UI** — mostly neutral already: the badge renders "AI: reject — {category}" / "AI: OK" / "AI: pending", and `vision_category` is a free string. Only the `"promotional"` fallback literal is cat-adjacent. Proposed decomposition (natural sub-PR sequence on the collecting branch): 1. **Extract prompts into settings** — configurable text/vision prompts on the settings page (string-setting helper + two textareas); keep current constants as defaults. No schema change. Highest value-per-effort. 2. **Rename verdict fields** to domain-neutral names (`vision_is_real_cat` → `vision_accepted`, optionally `text_is_promotional` → `text_flagged`) + one migration + schema-reconciliation test. 3. **Optional polish** — configurable "no image" accept/reject policy, reword the `"promotional"` badge fallback, docs. Estimate: ~15–20 files + 1 migration total; the effort concentrates in the settings-page UI (multi-line string inputs on a bool-only form) and pinning down the generic semantics, not the renames themselves.
Author
Collaborator

Plan of attack (approved design in .superpowers/specs/2026-08-21-generic-ai-classification-design.md):

  1. Sub-PR 1 feat/issue-100-ai-prompts — prompts become editable app_settings rows (ai_text_prompt / ai_vision_prompt) with settings-page textareas; code appends a pinned JSON output contract (keys is_flagged / is_accepted). Classifier classes/verdicts renamed to neutral semantics. No schema change.
  2. Sub-PR 2 feat/issue-100-rename-ai-verdict-columns — one migration renaming posts.text_is_promotionaltext_flagged and posts.vision_is_real_catvision_accepted; models + REST schema follow.
  3. Sub-PR 3 feat/issue-100-ai-badge-and-docs — neutral badge fallback + docs update.

Refs #100

Plan of attack (approved design in `.superpowers/specs/2026-08-21-generic-ai-classification-design.md`): 1. Sub-PR 1 `feat/issue-100-ai-prompts` — prompts become editable `app_settings` rows (`ai_text_prompt` / `ai_vision_prompt`) with settings-page textareas; code appends a pinned JSON output contract (keys `is_flagged` / `is_accepted`). Classifier classes/verdicts renamed to neutral semantics. No schema change. 2. Sub-PR 2 `feat/issue-100-rename-ai-verdict-columns` — one migration renaming `posts.text_is_promotional` → `text_flagged` and `posts.vision_is_real_cat` → `vision_accepted`; models + REST schema follow. 3. Sub-PR 3 `feat/issue-100-ai-badge-and-docs` — neutral badge fallback + docs update. Refs #100
Author
Collaborator

Sub-PR 1 #101 (feat/issue-100-ai-prompts, Tasks 0–5) merged into feat/issue-100-generic-ai-classification (71d3bd6). Issues: #100 (this one, Refs only).

Includes the ci-watch follow-ups: prompt-injection hardening (nonce-tagged delimiting + fail-closed output validation on both stages), call-site policy-loading tests, and release-notes/CHANGELOG wording fixes.

Next: sub-PR 2 feat/issue-100-rename-ai-verdict-columns.

Refs #100

Sub-PR 1 **#101** (`feat/issue-100-ai-prompts`, Tasks 0–5) merged into `feat/issue-100-generic-ai-classification` (`71d3bd6`). Issues: #100 (this one, Refs only). Includes the ci-watch follow-ups: prompt-injection hardening (nonce-tagged delimiting + fail-closed output validation on both stages), call-site policy-loading tests, and release-notes/CHANGELOG wording fixes. Next: sub-PR 2 `feat/issue-100-rename-ai-verdict-columns`. Refs #100
Author
Collaborator

Sub-PR 2 #102 (feat/issue-100-rename-ai-verdict-columns) merged into feat/issue-100-generic-ai-classification (7ea648e). Issues: #100 (Refs only).

Migration c4e8a1f3b7d2 renames posts.text_is_promotionaltext_flagged and posts.vision_is_real_catvision_accepted across models/schemas/pipeline, data-preserving with verified downgrade.

Next: sub-PR 3 feat/issue-100-badge-and-docs — neutral badge fallback + docs sweep.

Refs #100

Sub-PR 2 **#102** (`feat/issue-100-rename-ai-verdict-columns`) merged into `feat/issue-100-generic-ai-classification` (`7ea648e`). Issues: #100 (Refs only). Migration `c4e8a1f3b7d2` renames `posts.text_is_promotional` → `text_flagged` and `posts.vision_is_real_cat` → `vision_accepted` across models/schemas/pipeline, data-preserving with verified downgrade. Next: sub-PR 3 `feat/issue-100-badge-and-docs` — neutral badge fallback + docs sweep. Refs #100
Author
Collaborator

Sub-PR 3 #103 (feat/issue-100-badge-and-docs) merged into feat/issue-100-generic-ai-classification (513be5c). Issues: #100 (Refs only).

Neutral badge fallback (AI: reject — text), docs sweep to neutral field names, configurable-prompts documentation, stale nightly-batch docs removed.

All three sub-PRs merged. Running full DoD against the collecting branch, then the final PR into main (carrying Closes #100).

Refs #100

Sub-PR 3 **#103** (`feat/issue-100-badge-and-docs`) merged into `feat/issue-100-generic-ai-classification` (`513be5c`). Issues: #100 (Refs only). Neutral badge fallback (`AI: reject — text`), docs sweep to neutral field names, configurable-prompts documentation, stale nightly-batch docs removed. All three sub-PRs merged. Running full DoD against the collecting branch, then the final PR into main (carrying `Closes #100`). Refs #100
Author
Collaborator

Final PR #104 merged into main (53415d0); released as fenliu 3.0.0 (major: breaking posts-API rename of the AI verdict fields).

Delivered via three sub-PRs (#101, #102, #103) + review follow-ups on the final PR:

  • Configurable ai_text_prompt / ai_vision_prompt with settings-page textareas
  • Neutral columns: text_flagged / vision_accepted (migration c4e8a1f3b7d2)
  • Prompt-injection hardening: nonce-tagged delimiting, untrusted-data instructions (text + vision), fail-closed verdict validation on both stages
  • AI reject badge shows the text stage's reason (or vision category); docs updated

Closing — all scope from the design spec (.superpowers/specs/2026-08-21-generic-ai-classification-design.md) shipped.

Final PR **#104** merged into `main` (`53415d0`); released as **fenliu 3.0.0** (major: breaking posts-API rename of the AI verdict fields). Delivered via three sub-PRs (#101, #102, #103) + review follow-ups on the final PR: - Configurable `ai_text_prompt` / `ai_vision_prompt` with settings-page textareas - Neutral columns: `text_flagged` / `vision_accepted` (migration `c4e8a1f3b7d2`) - Prompt-injection hardening: nonce-tagged delimiting, untrusted-data instructions (text + vision), fail-closed verdict validation on both stages - AI reject badge shows the text stage's reason (or vision category); docs updated Closing — all scope from the design spec (`.superpowers/specs/2026-08-21-generic-ai-classification-design.md`) shipped.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
marvin8/dujiangyan#100
No description provided.