Severity floor and a valid no-findings outcome #83

New issue

Closed

opened 2026-10-15 19:45:31 +00:00 by agent-pi · 1 comment

agent-pi commented

2026-10-15 19:45:31 +00:00

Collaborator

On #75 the maintainer declined several MEDIUM findings and most LOW findings — a revealed preference for a higher floor than the default prompt produces.

A configurable minimum severity (e.g. WUMING_MIN_SEVERITY) plus explicit agent-prompt permission to return no findings would let repos tune strictness and cut nit volume.

Reviewers have no strictness knob: LOW wording nits arrive with the same weight as HIGH security findings, and there is implicit pressure to produce findings on every run rather than report "nothing worth flagging". On #75 the maintainer declined several MEDIUM findings and most LOW findings — a revealed preference for a higher floor than the default prompt produces. A configurable minimum severity (e.g. `WUMING_MIN_SEVERITY`) plus explicit agent-prompt permission to return no findings would let repos tune strictness and cut nit volume.

agent-pi added this to the Review quality & convergence milestone

2026-10-15 19:45:31 +00:00

agent-pi added the

Kind/Enhancement

Priority

Low

labels

2026-10-15 19:45:31 +00:00

agent-pi commented

2026-10-16 08:38:56 +00:00

Author

Collaborator

Plan of attack (design agreed in session; spec: wave-1 review quality)

Exploration findings:

Severity is already first-class: parsed from agent JSON, four levels (low|medium|high|critical), counted in the summary table with an unrecognised-severity catch-all. The floor is a rank-compare filter, not new plumbing.
All 7 agent prompts already say If no issues are found, return an empty array: [] and the pipeline handles [] cleanly — the gap is prompt pressure (no signal that a clean report is valued) and calibration, not mechanics.

Approach:

comments.py: severity_rank() (case-insensitive, None for unknown) + passes_severity_floor() — unrecognised severity fails closed when a floor is set.
config.py: WUMING_MIN_SEVERITY → Config.min_severity; invalid values raise ValueError naming the variable.
main.py: filter applied right after router.review(), before position resolution (cheapest filter first).
All 7 agent prompts: severity calibration block + explicit no-findings permission.
README/CLAUDE.md docs in the same PR.

Branch: feat/issue-83-min-severity
Commits: :sparkles: rank+parser+helper → :sparkles: pipeline wiring → :sparkles: prompt calibration → :memo: docs
PR: targets main, Closes #83

**Plan of attack** (design agreed in session; spec: wave-1 review quality) **Exploration findings:** - Severity is already first-class: parsed from agent JSON, four levels (`low|medium|high|critical`), counted in the summary table with an unrecognised-severity catch-all. The floor is a rank-compare filter, not new plumbing. - All 7 agent prompts already say `If no issues are found, return an empty array: []` and the pipeline handles `[]` cleanly — the gap is prompt *pressure* (no signal that a clean report is valued) and calibration, not mechanics. **Approach:** 1. `comments.py`: `severity_rank()` (case-insensitive, `None` for unknown) + `passes_severity_floor()` — unrecognised severity fails closed when a floor is set. 2. `config.py`: `WUMING_MIN_SEVERITY` → `Config.min_severity`; invalid values raise `ValueError` naming the variable. 3. `main.py`: filter applied right after `router.review()`, before position resolution (cheapest filter first). 4. All 7 agent prompts: severity calibration block + explicit no-findings permission. 5. README/CLAUDE.md docs in the same PR. **Branch:** `feat/issue-83-min-severity` **Commits:** `:sparkles:` rank+parser+helper → `:sparkles:` pipeline wiring → `:sparkles:` prompt calibration → `:memo:` docs **PR:** targets `main`, `Closes #83`

agent-pi referenced this issue

2026-10-16 21:20:42 +00:00

Add WUMING_MIN_SEVERITY severity floor #104

agent-pi closed this issue

2026-10-16 22:05:35 +00:00