AI classification fails when model returns empty content (JSONDecodeError) #106

New issue

Closed

opened 2026-09-02 08:46:47 +00:00 by agent-pi · 1 comment

agent-pi commented

2026-09-02 08:46:47 +00:00

Collaborator

Symptom

Intermittent AI classification failures where the model's HTTP request returns 200 OK but json.loads() raises JSONDecodeError: Expecting value: line 1 column 1 (char 0) — i.e. the message.content field is empty.

Affected posts (from packages/fenliu/fenliu-all.log):

Vision stage: 15, 22
Text stage: 41

Root cause

AIClient.complete_json does:

return json.loads(payload["choices"][0]["message"]["content"])

For the failing posts the model returned 200 OK but message.content was empty (char 0 = empty input), so json.loads("") raised. The exception is caught in classify_posts, which logs a warning and continues, leaving the post unclassified.

Traceback from logs/fenliu_debug.log:

  File ".../services/ai_classification.py", line 161, in complete_json
    return json.loads(payload["choices"][0]["message"]["content"])
  File ".../json/decoder.py", line 363, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Possible causes (not confirmed)

The request sends "thinking": {"type": "disabled"}, "response_format": {"type": "json_object"}, and max_tokens=300 against Moonshot kimi-k2.6. Candidates to investigate:

The model emits reasoning into reasoning_content and leaves content empty (thinking toggle not honored).
A content-moderation/filter response returns empty content.
max_tokens=300 truncates before any content is produced.

The raw response body is not logged, so the exact cause can't be pinned down from current logs.

Fix direction

Make complete_json robust and observable:

Guard json.loads — on empty/unparseable content, fail closed (return an "unclassified" signal rather than raising).
Log the raw payload (at least finish_reason, content, reasoning_content) at DEBUG when parsing fails, so the cause is visible next time.
Consider whether thinking should be enabled for kimi-k2.6 and the answer read from reasoning_content, or whether empty-content responses should be retried.

Impact

Affected posts are left AI: pending (manual review). No mis-classification — just unclassified posts.

## Symptom Intermittent AI classification failures where the model's HTTP request returns `200 OK` but `json.loads()` raises `JSONDecodeError: Expecting value: line 1 column 1 (char 0)` — i.e. the `message.content` field is empty. Affected posts (from `packages/fenliu/fenliu-all.log`): - Vision stage: **15, 22** - Text stage: **41** ## Root cause `AIClient.complete_json` does: ```python return json.loads(payload["choices"][0]["message"]["content"]) ``` For the failing posts the model returned `200 OK` but `message.content` was empty (char 0 = empty input), so `json.loads("")` raised. The exception is caught in `classify_posts`, which logs a warning and `continue`s, leaving the post unclassified. Traceback from `logs/fenliu_debug.log`: ``` File ".../services/ai_classification.py", line 161, in complete_json return json.loads(payload["choices"][0]["message"]["content"]) File ".../json/decoder.py", line 363, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) ``` ## Possible causes (not confirmed) The request sends `"thinking": {"type": "disabled"}`, `"response_format": {"type": "json_object"}`, and `max_tokens=300` against Moonshot `kimi-k2.6`. Candidates to investigate: 1. The model emits reasoning into `reasoning_content` and leaves `content` empty (thinking toggle not honored). 2. A content-moderation/filter response returns empty content. 3. `max_tokens=300` truncates before any content is produced. The raw response body is not logged, so the exact cause can't be pinned down from current logs. ## Fix direction Make `complete_json` robust and observable: - Guard `json.loads` — on empty/unparseable content, fail closed (return an "unclassified" signal rather than raising). - Log the raw payload (at least `finish_reason`, `content`, `reasoning_content`) at DEBUG when parsing fails, so the cause is visible next time. - Consider whether `thinking` should be enabled for kimi-k2.6 and the answer read from `reasoning_content`, or whether empty-content responses should be retried. ## Impact Affected posts are left `AI: pending` (manual review). No mis-classification — just unclassified posts.

agent-pi commented

2026-09-08 01:04:51 +00:00

Author

Collaborator

Plan of attack:

packages/fenliu/src/fenliu/services/ai_classification.py — complete_json guards the whole choices[0].message.content extraction and parse; empty/missing/non-JSON content raises a new typed AIMalformedResponseError. Before raising it logs a WARNING with finish_reason, the raw content repr, and a bounded reasoning_content excerpt (~500 chars), so the candidate causes (thinking toggle ignored / content filter / max_tokens truncation) are distinguishable on the next occurrence.
packages/fenliu/src/fenliu/services/ai_pipeline.py — catches the typed exception before the generic except Exception and logs a terse one-liner with the post id, no traceback. The post stays AI: pending for manual review (deliberate: no retry, no fail-closed verdict — retry decision deferred until the logs reveal the actual cause).
Tests (red first): the three malformed shapes raise the typed error, and classify_posts leaves the post unclassified when a stage hits empty content.
Branch fix/issue-106-handle-empty-ai-content, single commit, PR to main.

Plan of attack: 1. `packages/fenliu/src/fenliu/services/ai_classification.py` — `complete_json` guards the whole `choices[0].message.content` extraction and parse; empty/missing/non-JSON content raises a new typed `AIMalformedResponseError`. Before raising it logs a WARNING with `finish_reason`, the raw `content` repr, and a bounded `reasoning_content` excerpt (~500 chars), so the candidate causes (thinking toggle ignored / content filter / `max_tokens` truncation) are distinguishable on the next occurrence. 2. `packages/fenliu/src/fenliu/services/ai_pipeline.py` — catches the typed exception before the generic `except Exception` and logs a terse one-liner with the post id, no traceback. The post stays `AI: pending` for manual review (deliberate: no retry, no fail-closed verdict — retry decision deferred until the logs reveal the actual cause). 3. Tests (red first): the three malformed shapes raise the typed error, and `classify_posts` leaves the post unclassified when a stage hits empty content. 4. Branch `fix/issue-106-handle-empty-ai-content`, single commit, PR to `main`.

agent-pi referenced this issue from a commit

2026-09-08 01:28:23 +00:00

:bug: (fenliu) leave post pending and log diagnostics when AI returns empty content

agent-pi referenced this issue from a pull request that will close it,

2026-09-08 01:28:36 +00:00

Handle empty AI content without crashing classification #109

agent-pi referenced this issue

2026-09-08 02:24:27 +00:00

Handle empty AI content without crashing classification #109

agent-pi referenced this issue

2026-09-08 03:02:22 +00:00

Flaky test: startup log-redaction test fails intermittently depending on worker test order #110