Periodic snapshot scan finds 0 files after prune_empty_dirs deletes and camera recreates date directory #121

New issue

Closed

opened 2026-07-16 06:28:03 +00:00 by coding-agent-marvin8 · 2 comments

coding-agent-marvin8 commented

2026-07-16 06:28:03 +00:00

(Migrated from codeberg.org)

Shed Inside camera stopped being ingested after ~07:20 today. The periodic scan consistently reports "0 new, 0 already known" even though 816 jpg source files exist on disk.

What happens

The snapshot scanner processes all snapshots for a date, deletes source files, then calls prune_empty_dirs on the camera root
prune_empty_dirs deletes the now-empty YYYY-MM-DD/ directory
The camera writes the next snapshot, recreating the YYYY-MM-DD/ directory tree
On the next scan, root.iterdir() sees the recreated directory but its iterdir() returns [] — a stale kernel dcache entry
The scanner finds 0 snapshots, the cycle repeats, and the camera is never ingested again

Evidence

Diagnostic run inside the container confirms the stale cache:

Carpark avoids the bug because its motion-only snapshot schedule means the directory stays deleted between prunes — no stale cache to hit.

Shed Inside camera stopped being ingested after ~07:20 today. The periodic scan consistently reports "0 new, 0 already known" even though 816 jpg source files exist on disk. ## What happens 1. The snapshot scanner processes all snapshots for a date, deletes source files, then calls prune_empty_dirs on the camera root 2. prune_empty_dirs deletes the now-empty YYYY-MM-DD/ directory 3. The camera writes the next snapshot, recreating the YYYY-MM-DD/ directory tree 4. On the next scan, root.iterdir() sees the recreated directory but its iterdir() returns [] — a stale kernel dcache entry 5. The scanner finds 0 snapshots, the cycle repeats, and the camera is never ingested again ## Evidence Diagnostic run inside the container confirms the stale cache: Carpark avoids the bug because its motion-only snapshot schedule means the directory stays deleted between prunes — no stale cache to hit.

coding-agent-marvin8 commented

2026-07-16 06:28:21 +00:00

(Migrated from codeberg.org)

Root cause confirmed

The kernel dcache caches directory entries by (parent_ino, name). When prune_empty_dirs rmdirs the 2026-07-16/ directory and the camera immediately recreates it under the same parent, the kernel returns the cached empty entry list for the new directory — even through a fresh file descriptor. This only manifests on cameras with frequent writes (Shed Inside does 1 snapshot/minute; Carpark only writes on motion, so the deleted directory stays gone).

Plan

Fix the race, not the symptom. The correct fix is to stop deleting directories at a level where the dcache race can occur — specifically, never prune the date directory or channel directory from within the snapshot scan. Only prune leaf directories (jpg/HH/MM/ and jpg/HH/) that were actually emptied by the scan.

Specific changes

_scan_snapshots_for_camera in lifespan.py: Replace the blanket prune_empty_dirs(cam.root) call with targeted pruning of only the hour and minute subdirectories that the scan emptied. Collect affected directories from the paths of deleted source files and rmdir them bottom-up.
Ensure date-level cleanup still happens: Date and channel directories that become truly empty (no new files for the whole day) will be cleaned up by:
- _run_cleanup which already prunes per-camera output dirs
- Or add a lightweight prune in _run_cleanup that handles empty camera source directories at a safe cadence (not racing with writes)

Commit plan

:bug: Restrict snapshot-scan pruning to hour/minute directories below the date level
Branch: fix/issue-121-stale-dcache-after-prune
PR against main

## Root cause confirmed The kernel dcache caches directory entries by (parent_ino, name). When prune_empty_dirs rmdirs the `2026-07-16/` directory and the camera immediately recreates it under the same parent, the kernel returns the cached empty entry list for the new directory — even through a fresh file descriptor. This only manifests on cameras with frequent writes (Shed Inside does 1 snapshot/minute; Carpark only writes on motion, so the deleted directory stays gone). ## Plan **Fix the race, not the symptom.** The correct fix is to stop deleting directories at a level where the dcache race can occur — specifically, never prune the date directory or channel directory from within the snapshot scan. Only prune leaf directories (`jpg/HH/MM/` and `jpg/HH/`) that were actually emptied by the scan. ### Specific changes 1. **`_scan_snapshots_for_camera` in `lifespan.py`**: Replace the blanket `prune_empty_dirs(cam.root)` call with targeted pruning of only the hour and minute subdirectories that the scan emptied. Collect affected directories from the paths of deleted source files and rmdir them bottom-up. 2. **Ensure date-level cleanup still happens**: Date and channel directories that become truly empty (no new files for the whole day) will be cleaned up by: - `_run_cleanup` which already prunes per-camera output dirs - Or add a lightweight prune in `_run_cleanup` that handles empty camera source directories at a safe cadence (not racing with writes) ### Commit plan - `:bug: Restrict snapshot-scan pruning to hour/minute directories below the date level` - Branch: `fix/issue-121-stale-dcache-after-prune` - PR against `main`

marvin8 commented

2026-07-16 08:02:58 +00:00

(Migrated from codeberg.org)

Fixed in release 0.10.1

No labels

No milestone

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

marvin8/cang#121

No description provided.

Rows
Columns