AI vision stage fails on 302 redirects (media_proxy image URLs) #105

New issue

Closed

opened 2026-09-02 08:46:46 +00:00 by agent-pi · 1 comment

agent-pi commented

2026-09-02 08:46:46 +00:00

Collaborator

Symptom

The vision stage fails to download the first image for posts whose media is served through Mastodon's media_proxy URLs. The download raises, the post skips vision classification, and it stays AI: pending.

Affected posts (from packages/fenliu/fenliu-all.log): 42, 43, 44, 46.

Root cause

Mastodon serves some attachments via https://mastodon.social/media_proxy/<id>/small, which responds 302 Found redirecting to https://files.mastodon.social/cache/media_attachments/files/....

The httpx2 client that downloads images is created without follow_redirects=True:

# packages/fenliu/src/fenliu/services/ai_pipeline.py — classify_posts
async with httpx2.AsyncClient(timeout=180, headers=headers) as http:
    ...
    image, mime = await download_first_image(post, http)

Unlike stock httpx, httpx2's raise_for_status() raises on a 3xx redirect response. Traceback from logs/fenliu_debug.log:

  File ".../services/ai_vision.py", line 74, in download_first_image
    response.raise_for_status()
  File ".../httpx2/_models.py", line 827, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx2.HTTPStatusError: Redirect response '302 Found' for url 'https://mastodon.social/media_proxy/114776279186831253/small'
Redirect location: 'https://files.mastodon.social/cache/media_attachments/files/114/776/279/186/831/253/small/a3ca7f705aff6025.jpeg'

Fix direction

Create the download client with follow_redirects=True:

async with httpx2.AsyncClient(timeout=180, headers=headers, follow_redirects=True) as http:

The AI API client in ai_classification.py does not need this — it only calls api.moonshot.ai, which does not redirect.

Impact

Posts with media_proxy URLs never get a vision verdict (left AI: pending, reviewed manually). No mis-classification — just unclassified posts.

## Symptom The vision stage fails to download the first image for posts whose media is served through Mastodon's `media_proxy` URLs. The download raises, the post skips vision classification, and it stays `AI: pending`. Affected posts (from `packages/fenliu/fenliu-all.log`): **42, 43, 44, 46**. ## Root cause Mastodon serves some attachments via `https://mastodon.social/media_proxy/<id>/small`, which responds `302 Found` redirecting to `https://files.mastodon.social/cache/media_attachments/files/...`. The httpx2 client that downloads images is created without `follow_redirects=True`: ```python # packages/fenliu/src/fenliu/services/ai_pipeline.py — classify_posts async with httpx2.AsyncClient(timeout=180, headers=headers) as http: ... image, mime = await download_first_image(post, http) ``` Unlike stock httpx, httpx2's `raise_for_status()` raises on a 3xx redirect response. Traceback from `logs/fenliu_debug.log`: ``` File ".../services/ai_vision.py", line 74, in download_first_image response.raise_for_status() File ".../httpx2/_models.py", line 827, in raise_for_status raise HTTPStatusError(message, request=request, response=self) httpx2.HTTPStatusError: Redirect response '302 Found' for url 'https://mastodon.social/media_proxy/114776279186831253/small' Redirect location: 'https://files.mastodon.social/cache/media_attachments/files/114/776/279/186/831/253/small/a3ca7f705aff6025.jpeg' ``` ## Fix direction Create the download client with `follow_redirects=True`: ```python async with httpx2.AsyncClient(timeout=180, headers=headers, follow_redirects=True) as http: ``` The AI API client in `ai_classification.py` does not need this — it only calls `api.moonshot.ai`, which does not redirect. ## Impact Posts with `media_proxy` URLs never get a vision verdict (left `AI: pending`, reviewed manually). No mis-classification — just unclassified posts.

agent-pi commented

2026-09-08 01:04:51 +00:00

Author

Collaborator

Plan of attack:

packages/fenliu/src/fenliu/services/ai_pipeline.py — classify_posts creates the download client with follow_redirects=True so media_proxy 302s are followed.
packages/fenliu/src/fenliu/services/ai_vision.py — download_first_image guesses MIME from the post-redirect final URL (response.url) instead of the requested URL, so proxied non-jpeg attachments get their real MIME in the data URI sent to the vision API.
Tests (red first): end-to-end 302→200 through classify_posts via httpx2.MockTransport, and a direct MIME-from-final-URL test.
Branch fix/issue-105-follow-media-proxy-redirects, single commit, PR to main.

Plan of attack: 1. `packages/fenliu/src/fenliu/services/ai_pipeline.py` — `classify_posts` creates the download client with `follow_redirects=True` so `media_proxy` 302s are followed. 2. `packages/fenliu/src/fenliu/services/ai_vision.py` — `download_first_image` guesses MIME from the post-redirect final URL (`response.url`) instead of the requested URL, so proxied non-jpeg attachments get their real MIME in the data URI sent to the vision API. 3. Tests (red first): end-to-end 302→200 through `classify_posts` via `httpx2.MockTransport`, and a direct MIME-from-final-URL test. 4. Branch `fix/issue-105-follow-media-proxy-redirects`, single commit, PR to `main`.

agent-pi referenced this issue from a commit

2026-09-08 01:09:34 +00:00

:bug: (fenliu) follow redirects when downloading media_proxy images

agent-pi referenced this issue from a pull request that will close it,

2026-09-08 01:09:45 +00:00

Follow redirects when downloading AI vision images #107

agent-pi referenced this issue

2026-09-08 01:20:10 +00:00

SSRF hardening for server-side media fetches in the AI vision stage #108

agent-pi referenced this issue

2026-09-08 01:20:26 +00:00

Follow redirects when downloading AI vision images #107