AI classification: support both Moonshot-direct and OpenRouter back ends #113

New issue

Open

opened 2026-09-15 21:28:44 +00:00 by agent-pi · 1 comment

agent-pi commented

2026-09-15 21:28:44 +00:00

Collaborator

Problem

FenLiu's AI classification (text + vision) is built around a single OpenAI-compatible endpoint, currently Moonshot direct (https://api.moonshot.ai/v1, model kimi-k2.6). Two assumptions in services/ai_classification.py are Moonshot-specific:

Every request sends "thinking": {"type": "disabled"}. That is a Moonshot parameter; OpenRouter does not advertise thinking as a supported parameter for moonshotai/kimi-k2.6 (its unified parameter is reasoning).
The failure-diagnostics path reads reasoning from message.reasoning_content, Moonshot's field name. OpenRouter returns reasoning under message.reasoning and message.reasoning_details and has no reasoning_content field at all.

Pointing FenLiu at OpenRouter is therefore not a drop-in configuration change. Requests may be forwarded with parameters an upstream provider does not support, and diagnostics that inspect reasoning content silently report it as absent even when the model reasoned.

OpenRouter's non-streaming envelope also differs structurally from Moonshot direct: it adds native_finish_reason and logprobs to each choice, provider/service_tier/system_fingerprint at top level, refusal on the message, and cost/cost_details/is_byok on usage.

A second wrinkle applies to OpenRouter's shared capacity (i.e. not BYOK). moonshotai/kimi-k2.6 is an open-weight model served by roughly twenty upstream providers, and their supported-parameter sets differ — at least one (Chutes) does not advertise response_format, which FenLiu relies on for {"type": "json_object"}. BYOK routes to the Moonshot AI endpoint specifically; shared capacity may land on any of the others. The response envelope itself is OpenRouter-normalized in both cases.

Desired outcome

FenLiu should work reliably against either backend — Moonshot direct (kimi-k2.6) or OpenRouter (moonshotai/kimi-k2.6) — selected purely by configuration, with classification verdicts unchanged.

Impact

Today an operator who wants to switch to OpenRouter (or fall back to it) cannot do so safely with configuration alone.

## Problem FenLiu's AI classification (text + vision) is built around a single OpenAI-compatible endpoint, currently Moonshot direct (`https://api.moonshot.ai/v1`, model `kimi-k2.6`). Two assumptions in `services/ai_classification.py` are Moonshot-specific: 1. Every request sends `"thinking": {"type": "disabled"}`. That is a Moonshot parameter; OpenRouter does not advertise `thinking` as a supported parameter for `moonshotai/kimi-k2.6` (its unified parameter is `reasoning`). 2. The failure-diagnostics path reads reasoning from `message.reasoning_content`, Moonshot's field name. OpenRouter returns reasoning under `message.reasoning` and `message.reasoning_details` and has no `reasoning_content` field at all. Pointing FenLiu at OpenRouter is therefore not a drop-in configuration change. Requests may be forwarded with parameters an upstream provider does not support, and diagnostics that inspect reasoning content silently report it as absent even when the model reasoned. OpenRouter's non-streaming envelope also differs structurally from Moonshot direct: it adds `native_finish_reason` and `logprobs` to each choice, `provider`/`service_tier`/`system_fingerprint` at top level, `refusal` on the message, and `cost`/`cost_details`/`is_byok` on `usage`. A second wrinkle applies to OpenRouter's shared capacity (i.e. not BYOK). `moonshotai/kimi-k2.6` is an open-weight model served by roughly twenty upstream providers, and their supported-parameter sets differ — at least one (Chutes) does not advertise `response_format`, which FenLiu relies on for `{"type": "json_object"}`. BYOK routes to the Moonshot AI endpoint specifically; shared capacity may land on any of the others. The response envelope itself is OpenRouter-normalized in both cases. ## Desired outcome FenLiu should work reliably against either backend — Moonshot direct (`kimi-k2.6`) or OpenRouter (`moonshotai/kimi-k2.6`) — selected purely by configuration, with classification verdicts unchanged. ## Impact Today an operator who wants to switch to OpenRouter (or fall back to it) cannot do so safely with configuration alone.

agent-pi commented

2026-09-15 21:28:58 +00:00

Author

Collaborator

Confirmation: the JSON really does differ

Verified live against both endpoints with the same trivial prompt (HTTP 200 on each). Side-by-side of the response envelopes:

	Moonshot direct (`api.moonshot.ai`, `kimi-k2.6`)	OpenRouter (`openrouter.ai`, `moonshotai/kimi-k2.6`)
Request reasoning param	`"thinking": {"type": "disabled"}`	`"reasoning": {...}` — `thinking` is not in the model's `supported_parameters`
Choice fields	`finish_reason`, `index`, `message`	`+ native_finish_reason`, `+ logprobs`
Message fields	`content`, `reasoning_content`, `role`	`content`, `reasoning`, `reasoning_details`, `refusal`, `role` — no `reasoning_content`
Top-level extra	—	`provider`, `service_tier`, `system_fingerprint`
Usage extra	—	`cost`, `cost_details`, `is_byok`, `prompt_tokens_details`

A reasoning-enabled OpenRouter call returned message.reasoning (a string) and message.reasoning_details (an array), with usage.completion_tokens_details.reasoning_tokens: 152. The Moonshot-direct call had no reasoning field at all — it uses reasoning_content. So the _malformed_reply_diagnostics helper from #106, which reads message.reasoning_content, will always report reasoning_content=absent for OpenRouter responses.

BYOK vs shared capacity (the "not BYOK" part)

The envelope is OpenRouter-normalized either way; BYOK does not change the shape. What "not BYOK" changes is which upstream provider serves the open-weight model. The model endpoints listing on OpenRouter shows ~20 providers for moonshotai/kimi-k2.6 (Baidu, DeepInfra, Fireworks, Moonshot AI, Chutes, …). Their capability sets differ: every one advertises reasoning, but none advertises thinking, and at least Chutes does not advertise response_format. A request that works BYOK (routed to the Moonshot AI endpoint) can therefore land somewhere with weaker parameter support on shared capacity.

Two more observations:

OpenRouter currently accepts and forwards FenLiu's "thinking" param without error (the routed provider understands it), so the request is not rejected today — but it is undocumented for this model and not the portable choice.
"response_format": {"type": "json_object"} works on the Moonshot AI endpoint and most shared providers, but is not universally advertised.

Plan of attack

Tests first (red): add client tests that feed both a Moonshot-shaped envelope (reasoning_content) and an OpenRouter-shaped envelope (reasoning/reasoning_details/native_finish_reason/cost/is_byok) and assert the diagnostic reports reasoning as present for each. Add a test asserting which reasoning-disable request parameter is sent per backend.
Response parsing: make _malformed_reply_diagnostics tolerate both field names (prefer reasoning, fall back to reasoning_content); keep message.content handling as-is since both use it.
Request parameters: select the reasoning-disable parameter by backend — keep thinking for Moonshot direct, send the unified reasoning for OpenRouter — derived from the configured base URL (possibly an explicit small provider enum rather than string-sniffing).
Optional robustness: if a shared-capacity provider lacks response_format, consider structured_outputs/prompt-enforced JSON or a retry; decide scope during review.
Docs: update docs/user-guide/ai-classification.md and the configuration table with both base-URL/model combinations.
Regenerate pylock.toml only if dependencies change (none expected).

Findings from live verification are above; no dependencies or schema changes needed.

## Confirmation: the JSON really does differ Verified live against both endpoints with the same trivial prompt (HTTP 200 on each). Side-by-side of the response envelopes: | | Moonshot direct (`api.moonshot.ai`, `kimi-k2.6`) | OpenRouter (`openrouter.ai`, `moonshotai/kimi-k2.6`) | |---|---|---| | Request reasoning param | `"thinking": {"type": "disabled"}` | `"reasoning": {...}` — `thinking` is **not** in the model's `supported_parameters` | | Choice fields | `finish_reason`, `index`, `message` | `+ native_finish_reason`, `+ logprobs` | | Message fields | `content`, `reasoning_content`, `role` | `content`, `reasoning`, `reasoning_details`, `refusal`, `role` — **no `reasoning_content`** | | Top-level extra | — | `provider`, `service_tier`, `system_fingerprint` | | Usage extra | — | `cost`, `cost_details`, `is_byok`, `prompt_tokens_details` | A reasoning-enabled OpenRouter call returned `message.reasoning` (a string) and `message.reasoning_details` (an array), with `usage.completion_tokens_details.reasoning_tokens: 152`. The Moonshot-direct call had no `reasoning` field at all — it uses `reasoning_content`. So the `_malformed_reply_diagnostics` helper from #106, which reads `message.reasoning_content`, will always report `reasoning_content=absent` for OpenRouter responses. ## BYOK vs shared capacity (the "not BYOK" part) The envelope is OpenRouter-normalized **either way**; BYOK does not change the shape. What "not BYOK" changes is which upstream provider serves the open-weight model. The model endpoints listing on OpenRouter shows ~20 providers for `moonshotai/kimi-k2.6` (Baidu, DeepInfra, Fireworks, Moonshot AI, Chutes, …). Their capability sets differ: every one advertises `reasoning`, but none advertises `thinking`, and at least Chutes does not advertise `response_format`. A request that works BYOK (routed to the Moonshot AI endpoint) can therefore land somewhere with weaker parameter support on shared capacity. Two more observations: - OpenRouter currently *accepts* and forwards FenLiu's `"thinking"` param without error (the routed provider understands it), so the request is not rejected today — but it is undocumented for this model and not the portable choice. - `"response_format": {"type": "json_object"}` works on the Moonshot AI endpoint and most shared providers, but is not universally advertised. ## Plan of attack 1. **Tests first (red):** add client tests that feed both a Moonshot-shaped envelope (`reasoning_content`) and an OpenRouter-shaped envelope (`reasoning`/`reasoning_details`/`native_finish_reason`/`cost`/`is_byok`) and assert the diagnostic reports reasoning as present for each. Add a test asserting which reasoning-disable request parameter is sent per backend. 2. **Response parsing:** make `_malformed_reply_diagnostics` tolerate both field names (prefer `reasoning`, fall back to `reasoning_content`); keep `message.content` handling as-is since both use it. 3. **Request parameters:** select the reasoning-disable parameter by backend — keep `thinking` for Moonshot direct, send the unified `reasoning` for OpenRouter — derived from the configured base URL (possibly an explicit small provider enum rather than string-sniffing). 4. **Optional robustness:** if a shared-capacity provider lacks `response_format`, consider `structured_outputs`/prompt-enforced JSON or a retry; decide scope during review. 5. **Docs:** update `docs/user-guide/ai-classification.md` and the configuration table with both base-URL/model combinations. 6. Regenerate `pylock.toml` only if dependencies change (none expected). Findings from live verification are above; no dependencies or schema changes needed.