Handle empty AI content without crashing classification #109
Loading…
Reference in a new issue
No description provided.
Delete branch "fix/issue-106-handle-empty-ai-content"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Problem
Intermittently the model returns HTTP 200 but an empty
message.content, andAIClient.complete_jsonraised a bareJSONDecodeError: Expecting value: line 1 column 1 (char 0). The post stayedAI: pending(safe, but unexplained) and the logs gave no way to distinguish the candidate causes: thinking toggle not honoured (reasoning_contentpopulated,contentempty), a content filter, ormax_tokenstruncation.Fix
complete_jsonguards the wholechoices[0].message.contentextraction and parse. Empty / missing / non-JSON content raises a new typedAIMalformedResponseError— after logging a WARNING withfinish_reason, a boundedcontentexcerpt (200 chars), and a boundedreasoning_contentexcerpt (500 chars), all repr-escaped so untrusted model output can neither flood logs nor inject log lines. The cause is visible on the next occurrence.classify_posts(split into_classify_post/_text_verdict/_vision_verdictfor the complexity limit) catches the typed error before its genericexcept Exceptionand logs a terse one-liner per post, no traceback. The post staysAI: pendingfor manual review.Testing
choices[0]each raise the typed error (never a bareAttributeError/JSONDecodeError); diagnostics logging assertsfinish_reason, bounded content/reasoning excerpts, and the 500/200-char bounds; the raised exception message itself is bounded.ai_would_reject/text_flagged/ai_classified_atuntouched (pending) and never reaches the vision stage.follow_redirects=Truecoexist): ruff, ruff format, ty (0 diagnostics), complexipy, tryke.Closes #106
WuMing
Found 2 issue(s). See inline comments below.
@ -10,3 +10,4 @@---Occasionally the AI model answers with HTTP 200 but an empty message body, and the classifier died with a cryptic `JSONDecodeError`. Those posts now simply stay pending for manual review, accompanied by a clear warning; the model's finish reason, raw content, and a bounded reasoning excerpt are logged so the underlying cause is visible the next time it happens.docs [LOW]
The described failure is an empty
message.contentfield inside the model's JSON response, not an empty HTTP message body. Reword to something like 'HTTP 200 but an emptymessage.contentfield' so the release note matches the actual fix and doesn't mislead readers into thinking the HTTP response body was empty.@ -162,0 +201,4 @@except json.JSONDecodeError:passlogger.warning("Model reply has no parsable JSON content: %s", _malformed_reply_diagnostics(payload, reply_content)security [MEDIUM]
A09: The raw model reply content is written to WARNING logs via _malformed_reply_diagnostics. On a malformed response this content may echo or contain post text/PII from the user content sent to the classifier, leaking sensitive data into logs. Log only redacted/bounded metadata such as length, hash, finish_reason, or a safe sanitized excerpt.
WuMing
Found 2 issue(s). See inline comments below.
@ -5,3 +5,3 @@fenliu 3.0.0-- Improved: AI replies that arrive empty or unparseable now leave the post pending with a clear log line and full response diagnostics instead of a JSON traceback.docs [LOW]
'full response diagnostics' overstates what is logged: the reasoning excerpt is bounded to 500 characters, not the full response. Consider saying 'key response diagnostics' or listing the specific fields logged.
@ -57,0 +84,4 @@reasoning = str(message.get("reasoning_content") or "")excerpt = reasoning[:_REASONING_EXCERPT_LIMIT]return (f"finish_reason={finish_reason} content={content!r} "security [MEDIUM]
A09: The raw model reply content is written to WARNING logs via this diagnostics string. The reply is untrusted and may contain user-influenced or PII data; it is also not length-limited. Additionally, finish_reason is interpolated without validation or sanitisation, allowing log injection if an upstream or malicious response contains newlines or control characters. Remediate by redacting or truncating content, validating finish_reason against a known set, and emitting structured logs with control-character escaping.
WuMing review waves — responses
Release-Notes.md✅ Reworded in commit
4bcbb4f: the note now says 'an emptymessage.contentfield'.Release-Notes.md✅ Reworded in commit
4bcbb4f: the bullet now says 'key response diagnostics' and the detail post lists the bounded fields.ai_classification.py✅ Fixed in commit
4bcbb4f:contentis now excerpted to 200 chars,finish_reasonto 100,reasoning_contentto 500 — allrepr-escaped, so untrusted model output can neither flood logs with echoed post text nor inject log lines via control characters.finish_reason, presence of reasoning, and the bounded excerpts still distinguish the candidate causes.ai_classification.py✅ Fixed in commit
4bcbb4f: all three free-text fields are length-bounded andrepr-escaped (repr neutralises newlines/control characters, so injection is prevented at the formatting layer); the full reply body is never logged.WuMing
Found 3 issue(s). See inline comments below.
@ -10,3 +10,4 @@---Occasionally the AI model answers with HTTP 200 but an empty `message.content` field, and the classifier died with a cryptic `JSONDecodeError`. Those posts now simply stay pending for manual review, accompanied by a clear warning; the model's finish reason plus bounded excerpts of its content and reasoning are logged so the underlying cause is visible the next time it happens.docs [MEDIUM]
The release note says 'bounded excerpts of its content and reasoning are logged', but the described behavior logs the raw
contentrepr (not bounded) and a bounded 500-characterreasoning_contentexcerpt. Reword to avoid implying the content excerpt is also bounded, e.g. 'the model's finish reason plus the raw content and a bounded reasoning excerpt are logged'.@ -162,0 +205,4 @@except json.JSONDecodeError:passlogger.warning("Model reply has no parsable JSON content: %s", _malformed_reply_diagnostics(payload, reply_content)security [MEDIUM]
A09: The diagnostics log includes raw excerpts of the model's content and reasoning_content fields. These are untrusted model outputs that may echo user post text/PII or include chain-of-thought. While the values are repr-escaped and length-limited (which prevents log injection/line flooding), the underlying sensitive data is still written to logs. Log only non-sensitive diagnostics such as finish_reason, lengths, or a hash; avoid logging raw model output.
@ -52,0 +128,4 @@diagnostics = " ".join(record.getMessage() for record in collector.records)expect("finish_reason='content_filter'" in diagnostics, "finish_reason logged").to_be_truthy()expect("reasoning_content" in diagnostics, "reasoning_content noted").to_be_truthy()tests [LOW]
This assertion checks for the literal substring 'reasoning_content', but that word is a constant part of the diagnostics format and appears whether reasoning content is present or absent. It cannot distinguish the intended condition. Assert on 'reasoning_content=present' instead, or rely on the excerpt assertions already present.
WuMing wave 3 — responses
Release-Notes.md🔴 The premise no longer matches the code: commit
4bcbb4f(the same commit that reworded this note) boundscontentto 200 chars andfinish_reasonto 100, bothrepr-escaped. 'Bounded excerpts of its content and reasoning' is exactly what the current implementation logs, so the note is accurate as written.ai_classification.py🔴 Declining after explicit owner review. These are WARNING-level entries in the operator's private log, bounded (200/500 chars) and
repr-escaped so they cannot inject or flood log lines; hashes and lengths cannot answer why the model returned empty content, which was the purpose of the logging (see #106). The bounded excerpts are a deliberate traceability trade-off, re-examined twice — happy to revisit if a concrete leak path shows up.test_ai_classification.py✅ Fixed in commit
fcc4f7d: the test now assertsreasoning_content=present.WuMing
Found 3 issue(s). See inline comments below.
@ -57,0 +84,4 @@message = envelope.get("message") if isinstance(envelope, dict) else {}if not isinstance(message, dict):message = {}finish_reason = str(envelope.get("finish_reason", "unknown"))[:_FINISH_REASON_LIMIT]code [MEDIUM]
_malformed_reply_diagnostics assumes choices[0] is a dict, but line 83 can set envelope to a non-dict when choices[0] is e.g. a list. A malformed payload such as {"choices":} then raises AttributeError from this diagnostics helper instead of the intended AIMalformedResponseError, and the diagnostic warning is never logged. Guard with
if not isinstance(envelope, dict): envelope = {}before calling envelope.get.@ -162,0 +207,4 @@logger.warning("Model reply has no parsable JSON content: %s", _malformed_reply_diagnostics(payload, reply_content))raise AIMalformedResponseError(f"Model reply has no parsable JSON content: {reply_content!r}")code [MEDIUM]
Raising AIMalformedResponseError with the raw reply_content embeds the entire untrusted model output in the exception message. This can be very large and may leak post/model text if the exception is ever logged with a traceback. Reuse the bounded excerpt from _malformed_reply_diagnostics or slice reply_content to _CONTENT_EXCERPT_LIMIT before interpolating.
@ -162,0 +221,4 @@or not a string."""choices = payload.get("choices")code [LOW]
_message_content is documented as tolerating a malformed envelope, but it calls payload.get without first verifying payload is a dict. If response.json() returns a list, null, or another non-dict JSON value, complete_json raises AttributeError instead of AIMalformedResponseError and no diagnostics are logged. Add
if not isinstance(payload, dict): return Noneat the start.WuMing wave 4 — responses
ai_classification.py✅ Fixed in commit
9b0c3fb: the message now carries only the 200-char bounded excerpt, consistent with the log policy.ai_classification.py✅ Fixed in commit
9b0c3fb:_message_contentreturns None for non-dict bodies, and_malformed_reply_diagnosticsnormalises them too, so the typed error and diagnostics are produced in every malformed-envelope case (covered by new tests).ai_classification.py✅ Fixed in commit
9b0c3fb:envelopeis normalised to an empty dict before any.get, and a regression test pins{"choices":[[]]}→ typed error + diagnostics instead ofAttributeError.68f5d49585152b710322WuMing
Found 2 issue(s). See inline comments below.
@ -12,6 +13,8 @@ fenliu 3.0.0Some Mastodon attachments are served through a redirecting `media_proxy` URL rather than a direct file link. The AI vision stage's image downloader treated those redirects as errors, so posts with proxy-served media never received a vision verdict and had to be reviewed by hand. It now follows the redirect to the real file and infers the image type from the final URL's file extension, so proxied PNG/WebP/GIF attachments are labelled correctly instead of always falling back to `image/jpeg`.Occasionally the AI model answers with HTTP 200 but an empty `message.content` field, and the classifier died with a cryptic `JSONDecodeError`. Those posts now simply stay pending for manual review, accompanied by a clear warning; the model's finish reason plus bounded excerpts of its content and reasoning are logged so the underlying cause is visible the next time it happens.docs [LOW]
Mixed tense in the first sentence: 'answers' is present but 'died' is past. Make the historical description consistent, e.g. 'Occasionally the AI model would answer with HTTP 200 but an empty
message.contentfield, and the classifier died with a crypticJSONDecodeError.'@ -162,0 +209,4 @@except json.JSONDecodeError:passlogger.warning("Model reply has no parsable JSON content: %s", _malformed_reply_diagnostics(payload, reply_content)security [LOW]
A09: This warning logs raw model reply excerpts (content and reasoning_content) without redaction. Although length-bounded and repr-escaped against log injection, these excerpts can echo user post text/PII from the classification prompt. Log only non-sensitive metadata such as finish_reason, lengths, or hashes, or redact free-text model output.
WuMing
Found 1 issue(s). See inline comments below.
@ -12,6 +13,8 @@ fenliu 3.0.0Some Mastodon attachments are served through a redirecting `media_proxy` URL rather than a direct file link. The AI vision stage's image downloader treated those redirects as errors, so posts with proxy-served media never received a vision verdict and had to be reviewed by hand. It now follows the redirect to the real file and infers the image type from the final URL's file extension, so proxied PNG/WebP/GIF attachments are labelled correctly instead of always falling back to `image/jpeg`.Occasionally the AI model answers with HTTP 200 but an empty `message.content` field, and the classifier died with a cryptic `JSONDecodeError`. Those posts now simply stay pending for manual review, accompanied by a clear warning; the model's finish reason plus bounded excerpts of its content and reasoning are logged so the underlying cause is visible the next time it happens.docs [LOW]
The release-note sentence says “bounded excerpts of its content and reasoning are logged,” but the PR description says the raw
contentrepr is logged and only thereasoning_contentexcerpt is bounded to 500 characters. Revise the wording so it doesn’t imply both are bounded unless the implementation was changed to bound the content too.WuMing waves 209–210 (post-rebase) — responses
ai_classification.py🔴 Declining — settled with the owner during wave 3 (see the earlier consolidated reply). The excerpts are length-bounded (200/500 chars) and repr-escaped, live only in the operator's private log, and are the deliberate traceability trade-off for diagnosing why the model returns empty content; hashes/lengths cannot answer that. Re-raising on a fresh diff does not change the decision.
Release-Notes.md✅ Fixed: now reads 'Occasionally the AI model would answer with HTTP 200 but an empty
message.contentfield'.Release-Notes.md✅ The release note is accurate as written — the implementation WAS changed (commit
4bcbb4fboundscontentto 200 chars andfinish_reasonto 100, both repr-escaped). The stale text was in the PR description, which I have now corrected; no note change needed.WuMing
Found 2 issue(s). See inline comments below.
@ -12,6 +13,8 @@ fenliu 3.0.0Some Mastodon attachments are served through a redirecting `media_proxy` URL rather than a direct file link. The AI vision stage's image downloader treated those redirects as errors, so posts with proxy-served media never received a vision verdict and had to be reviewed by hand. It now follows the redirect to the real file and infers the image type from the final URL's file extension, so proxied PNG/WebP/GIF attachments are labelled correctly instead of always falling back to `image/jpeg`.Occasionally the AI model would answer with HTTP 200 but an empty `message.content` field, and the classifier died with a cryptic `JSONDecodeError`. Those posts now simply stay pending for manual review, accompanied by a clear warning; the model's finish reason plus bounded excerpts of its content and reasoning are logged so the underlying cause is visible the next time it happens.docs [LOW]
The fix logs the raw content repr and only bounds the reasoning_content excerpt to 500 chars, so describing both as 'bounded excerpts of its content and reasoning' is inaccurate. Consider wording it as 'the model's finish reason, the raw content repr, and a 500-character excerpt of its reasoning' to match the actual diagnostics.
@ -57,0 +92,4 @@reasoning = str(message.get("reasoning_content") or "")content_excerpt = (content or "")[:_CONTENT_EXCERPT_LIMIT]return (f"finish_reason={finish_reason!r} content={content_excerpt!r} "security [LOW]
A09: This diagnostic string includes raw model reply content and reasoning_content, which may contain user post text/PII echoed from the prompt. Bounding length and repr-escaping prevent log flooding/injection but do not prevent sensitive PII disclosure. Log only non-sensitive metadata such as finish_reason and content presence/length, or apply PII redaction before logging.
WuMing wave 211 — responses
Release-Notes.md🔴 The premise doesn't match the current code: since commit
4bcbb4f_malformed_reply_diagnosticsboundscontentto 200 chars (_CONTENT_EXCERPT_LIMIT) andfinish_reasonto 100, all repr-escaped — 'bounded excerpts of its content and reasoning' describes the implementation exactly. The finding appears to have been reviewed against a pre-4bcbb4fview of this file.ai_classification.py🔴 Declining — same finding as #issuecomment-2293/2230/2239, already reviewed and decided with the owner: the bounded, repr-escaped excerpts in the operator's private log are the deliberate traceability trade-off (see the wave-3 consolidated reply). No change.
WuMing
Found 3 issue(s). See inline comments below.
@ -5,6 +5,8 @@ visibility: publicfenliu 3.0.0- Fixed: AI vision now follows `media_proxy` redirects, so posts with proxy-served attachments get classified instead of remaining pending.- Improved: AI replies that arrive empty or unparseable now leave the post pending with a clear log line and key response diagnostics instead of a JSON traceback.docs [LOW]
The phrase "JSON traceback" is imprecise: the traceback is a Python stack trace ending in
json.JSONDecodeError, not a JSON-formatted traceback. Consider wording it as "instead of aJSONDecodeErrortraceback".@ -12,6 +14,10 @@ fenliu 3.0.0Some Mastodon attachments are served through a redirecting `media_proxy` URL rather than a direct file link. The AI vision stage's image downloader treated those redirects as errors, so posts with proxy-served media never received a vision verdict and had to be reviewed by hand. It now follows the redirect to the real file and infers the image type from the final URL's file extension, so proxied PNG/WebP/GIF attachments are labelled correctly instead of always falling back to `image/jpeg`.Occasionally the AI model would answer with HTTP 200 but an empty `message.content` field, and the classifier died with a cryptic `JSONDecodeError`. Those posts now simply stay pending for manual review, accompanied by a clear warning; the model's finish reason plus bounded excerpts of its content and reasoning are logged so the underlying cause is visible the next time it happens.docs [LOW]
The original behavior left the post as
AI: pending; the error was contained rather than killing the classifier process. People may read "classifier died" as a hard crash. Suggest "the classification request failed" or "classification raised a crypticJSONDecodeError".@ -37,1 +37,3 @@return f"{timestamp} - {context}:{record.lineno} - {record.levelname} - {record.getMessage()}"formatted = f"{timestamp} - {context}:{record.lineno} - {record.levelname} - {record.getMessage()}"if record.exc_info:formatted = f"{formatted}\n{self.formatException(record.exc_info)}"security [LOW]
A09: Appends raw exception tracebacks to log output whenever exc_info is set. Tracebacks can expose internal paths, library details, and exception messages that may contain sensitive context. Prefer sanitised/structured exception logging or redact sensitive data before writing tracebacks to production logs.
WuMing wave 212 — responses
Release-Notes.md✅ Fixed: now reads 'instead of a
JSONDecodeErrortraceback'.Release-Notes.md✅ Fixed: reworded to 'the classification request failed with a cryptic
JSONDecodeError'.logging.py🔴 Declining — this is the intended behaviour of the change (see the commit and release note): tracebacks in the operator's private console log are the standard diagnostic for unexpected failures, and stripping/redacting them would reintroduce exactly the 'classification failed with no why' blindness this commit fixes. No sensitive data is added by the traceback; it is the same exception the process already raised.