marvin8/cang
Watch
1
0
Fork
You've already forked cang
 0
Code Issues Pull requests Releases 35 Activity Actions

Fix progressive web UI slowdown from WAL bloat and missing index #133

Manually merged
marvin8 merged 8 commits from fix/issue-132-wal-checkpoint-and-index into main 2026-08-11 23:49:45 +00:00
Conversation 6 Commits 8 Files changed 10 +130 -22
agent-pi commented 2026-08-11 20:51:09 +00:00
Collaborator
Copy link

Closes #132

Changes

WAL checkpoint after periodic scan

Adds PRAGMA wal_checkpoint(TRUNCATE) after each periodic scan cycle to prevent SQLite WAL file unbounded growth. The default PASSIVE auto-checkpoint silently fails when readers are active (which is always the case in a web server with HTMX polling). TRUNCATE waits for readers to finish and zeroes the WAL.

Also adds PRAGMA busy_timeout=5000 at startup so readers don't wait indefinitely if a checkpoint briefly holds the lock.

Index on snapshots(recorded_at, motion) (V5 migration)

The calendar view's GROUP BY day query was doing a full table scan on every page load. The new composite index turns it into a single-pass index-only scan, and also speeds up unfiltered day-range queries.

Tests

  • V5 index existence
  • busy_timeout pragma set to 5000
  • WAL checkpoint truncates the WAL file after writes
Closes #132 ## Changes ### WAL checkpoint after periodic scan Adds `PRAGMA wal_checkpoint(TRUNCATE)` after each periodic scan cycle to prevent SQLite WAL file unbounded growth. The default PASSIVE auto-checkpoint silently fails when readers are active (which is always the case in a web server with HTMX polling). TRUNCATE waits for readers to finish and zeroes the WAL. Also adds `PRAGMA busy_timeout=5000` at startup so readers don't wait indefinitely if a checkpoint briefly holds the lock. ### Index on snapshots(recorded_at, motion) (V5 migration) The calendar view's `GROUP BY day` query was doing a full table scan on every page load. The new composite index turns it into a single-pass index-only scan, and also speeds up unfiltered day-range queries. ### Tests - V5 index existence - busy_timeout pragma set to 5000 - WAL checkpoint truncates the WAL file after writes
⚡ Add WAL checkpoint after periodic scan, busy_timeout, and recorded_at index
Some checks failed
/ pr-review (pull_request) Failing after 3s
Details
/ gitleaks (pull_request) Successful in 16s
Details
/ checks (pull_request) Successful in 1m56s
Details
/ publish (pull_request) Has been skipped
Details
/ deploy-docs (pull_request) Has been skipped
Details
/ publish-container (pull_request) Has been skipped
Details
47f82c6c4c
 
- Add PRAGMA busy_timeout=5000 in open_db to prevent indefinite waits
- Add V5 migration: index on snapshots(recorded_at, motion) for calendar
  GROUP BY query — turns full table scan into index-only scan
- Add _checkpoint_wal(db) that runs PRAGMA wal_checkpoint(TRUNCATE)
  after each periodic scan cycle — prevents WAL unbounded growth from
  PASSIVE checkpoints silently failing when readers are active
- Add tests: V5 index existence, busy_timeout, WAL checkpoint truncation
- Update Release-Notes.md
🔧 Finalize forge.marvin8.zone migration: update wuming and zaojun URLs
All checks were successful
/ gitleaks (pull_request) Successful in 37s
Details
/ pr-review (pull_request) Successful in 42s
Details
/ checks (pull_request) Successful in 2m5s
Details
/ publish (pull_request) Has been skipped
Details
/ deploy-docs (pull_request) Has been skipped
Details
/ publish-container (pull_request) Has been skipped
Details
a74d3f1f41
 
- Update wuming action URL from codeberg.org to forge.marvin8.zone
- Update zaojun pre-commit hook repo URL to forge.marvin8.zone
- Remove dead codeberg.page redirect from docs (cang was never on Codeberg Pages)
forgejo-actions reviewed 2026-08-11 21:01:28 +00:00
forgejo-actions left a comment
Copy link

WuMing

Found 1 issue(s). See inline comments below.

## WuMing Found **1** issue(s). See inline comments below.
.forgejo/workflows/ci.yml Outdated
@ -13,3 +13,3 @@
if: github.event_name == 'pull_request'
steps:
- uses: https://codeberg.org/marvin8/wuming@main
- uses: https://forge.marvin8.zone/marvin8/wuming@main
forgejo-actions commented 2026-08-11 21:01:28 +00:00
Copy link

security [MEDIUM]

The workflow action is referenced with a mutable branch '@main' rather than a pinned commit SHA, making it vulnerable to supply chain attacks. An attacker who compromises the branch can inject malicious code and exfiltrate secrets such as 'forge.token'. OWASP A08: Software and Data Integrity Failures. Remediate by pinning to a specific, verified commit hash.

**security** [MEDIUM] The workflow action is referenced with a mutable branch '@main' rather than a pinned commit SHA, making it vulnerable to supply chain attacks. An attacker who compromises the branch can inject malicious code and exfiltrate secrets such as 'forge.token'. OWASP A08: Software and Data Integrity Failures. Remediate by pinning to a specific, verified commit hash. <!-- wuming:sha256:4717384c43089eea254f9cffdb6e73e15cc4fb8571083b07ca8a73b416117375 -->
agent-pi commented 2026-08-11 21:04:35 +00:00
Author
Collaborator
Copy link

line 15

The workflow action is referenced with a mutable branch @main rather than a pinned commit SHA

✅ Pinning to commit d4ef1c6 — the current HEAD of wuming's main branch.

[ line 15](https://forge.marvin8.zone/marvin8/cang/pulls/133#issuecomment-1216) > The workflow action is referenced with a mutable branch `@main` rather than a pinned commit SHA ✅ Pinning to commit `d4ef1c6` — the current HEAD of wuming's main branch.
agent-pi commented 2026-08-11 21:04:40 +00:00
Author
Collaborator
Copy link

forgejo/workflows/ci.yml line 15

The workflow action is referenced with a mutable branch @main rather than a pinned commit SHA

✅ Pinning to commit d4ef1c6 — the current HEAD of wuming main branch.

[_forgejo/workflows/ci.yml_ line 15](https://forge.marvin8.zone/marvin8/cang/pulls/133#issuecomment-1216) > The workflow action is referenced with a mutable branch `@main` rather than a pinned commit SHA ✅ Pinning to commit `d4ef1c6` — the current HEAD of wuming main branch.
🔒 Pin wuming action to commit SHA instead of branch ref
All checks were successful
/ gitleaks (pull_request) Successful in 16s
Details
/ pr-review (pull_request) Successful in 39s
Details
/ checks (pull_request) Successful in 1m48s
Details
/ publish (pull_request) Has been skipped
Details
/ deploy-docs (pull_request) Has been skipped
Details
/ publish-container (pull_request) Has been skipped
Details
e717c33239
agent-pi commented 2026-08-11 21:42:31 +00:00
Author
Collaborator
Copy link

forgejo/workflows/ci.yml line 15

The workflow action is referenced with a mutable branch @main rather than a pinned commit SHA

🔴 Declined — both the cang and wuming repos live on the same self-hosted forge instance. The generic supply-chain threat model (external attacker compromising a third-party action branch) does not apply here. Pinned-SHA maintenance churn is not warranted for this trust boundary.

[_forgejo/workflows/ci.yml_ line 15](https://forge.marvin8.zone/marvin8/cang/pulls/133#issuecomment-1216) > The workflow action is referenced with a mutable branch `@main` rather than a pinned commit SHA 🔴 Declined — both the cang and wuming repos live on the same self-hosted forge instance. The generic supply-chain threat model (external attacker compromising a third-party action branch) does not apply here. Pinned-SHA maintenance churn is not warranted for this trust boundary.
⏪ Revert wuming action pin — back to @main branch ref
All checks were successful
/ gitleaks (pull_request) Successful in 14s
Details
/ pr-review (pull_request) Successful in 34s
Details
/ checks (pull_request) Successful in 1m48s
Details
/ publish (pull_request) Has been skipped
Details
/ deploy-docs (pull_request) Has been skipped
Details
/ publish-container (pull_request) Has been skipped
Details
af7ef0d999
forgejo-actions reviewed 2026-08-11 21:43:17 +00:00
forgejo-actions left a comment
Copy link

WuMing

Found 1 issue(s). See inline comments below.

## WuMing Found **1** issue(s). See inline comments below.
.pre-commit-config.yaml
@ -42,3 +42,3 @@
- id: gitleaks
- repo: https://codeberg.org/marvin8/zaojun
- repo: https://forge.marvin8.zone/marvin8/zaojun
forgejo-actions commented 2026-08-11 21:43:17 +00:00
Copy link

security [MEDIUM]

A08: The pre-commit hook is fetched from a mutable branch (no pinned revision) without integrity verification. If the remote repository or its domain is hijacked, arbitrary code could run on developer machines. Use a specific commit SHA in the repo URL or verify signatures to prevent tampering.

**security** [MEDIUM] A08: The pre-commit hook is fetched from a mutable branch (no pinned revision) without integrity verification. If the remote repository or its domain is hijacked, arbitrary code could run on developer machines. Use a specific commit SHA in the repo URL or verify signatures to prevent tampering. <!-- wuming:sha256:32fe13e0a049ae728ff6ab1a10aa6fce4902e81a6367374c506f8f7c1ea3c076 -->
🔧 Fix bumpversion config, remove dead redirect.js, fix template sed
All checks were successful
/ gitleaks (pull_request) Successful in 16s
Details
/ pr-review (pull_request) Successful in 38s
Details
/ checks (pull_request) Successful in 1m42s
Details
/ publish (pull_request) Has been skipped
Details
/ deploy-docs (pull_request) Has been skipped
Details
/ publish-container (pull_request) Has been skipped
Details
a95218ee14
 
- Add bumpversion rule for docker pull line (cang:{version}) — was
  missed, so the docker pull command in Release-Notes.md never got
  updated on version bumps
- Fix Release-Notes.md.template: use YYYY.MM.DD placeholder matching
  the sed pattern in reset-release-notes
- Fix current docker pull version in Release-Notes.md (2026.7.25 → 2026.8.4)
- Remove docs/javascripts/redirect.js and its mkdocs reference —
  no redirects needed, cang was never on Codeberg Pages
🔧 Complete fix of bumpversion to also update version in docs link
All checks were successful
/ gitleaks (pull_request) Successful in 17s
Details
/ pr-review (pull_request) Successful in 46s
Details
/ checks (pull_request) Successful in 1m44s
Details
/ publish (pull_request) Has been skipped
Details
/ deploy-docs (pull_request) Has been skipped
Details
/ publish-container (pull_request) Has been skipped
Details
c60cf81c3d
🔧 fix order of posts in Release Notes
All checks were successful
/ gitleaks (pull_request) Successful in 16s
Details
/ pr-review (pull_request) Successful in 41s
Details
/ checks (pull_request) Successful in 1m40s
Details
/ publish (pull_request) Has been skipped
Details
/ deploy-docs (pull_request) Has been skipped
Details
/ publish-container (pull_request) Has been skipped
Details
4e2ea1005b
🔧 also update the release notes template to include version in
All checks were successful
/ gitleaks (pull_request) Successful in 16s
Details
/ pr-review (pull_request) Successful in 41s
Details
/ checks (pull_request) Successful in 1m42s
Details
/ publish (pull_request) Has been skipped
Details
/ deploy-docs (pull_request) Has been skipped
Details
/ publish-container (pull_request) Has been skipped
Details
7f16fa448c
 
doc url
marvin8 approved these changes 2026-08-11 23:48:15 +00:00
marvin8 manually merged commit 020108cca9 into main 2026-08-11 23:49:45 +00:00
marvin8 deleted branch fix/issue-132-wal-checkpoint-and-index 2026-08-11 23:51:04 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
marvin8
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Assignees
Clear assignees
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
marvin8/cang!133
No description provided.