Replace custom template syntax with Jinja2 #89
No reviewers
Labels
No labels
bug
contribution welcome
duplicate
enhancement
good first issue
help wanted
invalid
question
upstream
No milestone
No assignees
1 participant
Notifications
Due date
No due date set.
Reference
marvin8/feed2fedi!89
Loading…
Reference in a new issue
No description provided.
Delete branch "refs/pull/89/head"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
{var}/{[prefix]var[suffix]}template engine with Jinja2.j2files referenced viabot_post_template_file/post_template_filein configfeed2fedi-migrate-templatesCLI command to automatically convert existing configs and templates\npreprocessing workaround — template files use real newlinesbot_post_template/post_templateconfig fields raise a clear error pointing to the migration toolfeed2fedi-convert-configutility (INI-to-JSON migration, long obsolete)Breaking change
Users must migrate their templates. Run:
Test plan
nox -s pytest)ruff check .passesty check .passescomplexipy .passesfeed2fedi-migrate-templates --config-file config.json(dry run) against a config with old template fields--applyand verify.j2files are created and config is updatedCloses #88
Closes #90
src/feed2fedi/control.pyline 410 · line 420 — @wuming✅ Will wrap both `read_text()` calls in `load_config` with a `try/except FileNotFoundError` that re-raises as `Feed2FediError` with a descriptive message including the expected path.
src/feed2fedi/control.pyline 410 · line 420 — @wuming🔴 Not actioning. These paths come from the user's own `config.json`, which only they can write. An attacker who can modify `config.json` already has full control of the machine. Restricting path traversal here would prevent legitimate use cases like `../shared-templates/post.j2` across multiple configs. The threat model for a CLI tool run by its own operator does not include a malicious config file.
src/feed2fedi/control.pyline 400 — @wuming✅ Confirmed by design. The hard error is intentional — it ensures users notice the breaking change rather than silently losing their custom template. The `feed2fedi-migrate-templates` tool handles the one-time conversion.
src/feed2fedi/migrate.pyline 105 — @wuming✅ Valid point — the migration tool reads raw JSON without schema validation. Will add a check that raises a `ValueError` with a descriptive message if a feed entry is missing `url`.
src/feed2fedi/migrate.pyline 117 — @wuming✅ The current message already says: "If you have already migrated, this config has no templates." — so this is addressed.
src/feed2fedi/migrate.pyline 88 — @wuming🔴 Not actioning. This scenario cannot arise in practice: `load_config` raises a `Feed2FediError` if both are present in the JSON, so any valid working config will never have both simultaneously. The migration tool is a one-time conversion utility; it's not expected to be run on already-migrated configs.
src/feed2fedi/migrate.pyline 83 — @wuming🔴 Not actioning.
config.jsonis a local file owned by the operator running feed2fedi. An attacker who can write to it already has full local access. Adding checksums would give a false sense of security without changing the threat model.Release-Notes.md line 6 — @wuming
✅ Already covered. The prose section of the release notes (line 16 onward) explicitly states: "The old inline
bot_post_templateandpost_templateconfig fields are no longer accepted — feed2fedi will error immediately on startup if they are present in your config."Release-Notes.md line 12 · line 19 — @wuming
✅ Will split the prose block into two shorter paragraphs and remove the duplicate migration step reference.
docs/Config-File-Explained.mdline 250 — @wuming🔴 Not actioning — this is correct Markdown. Inside a GFM table cell, a literal
|must be escaped as\|to avoid breaking the table column delimiter. The rendered output is the correct Jinja2 syntax{{ description | default(title, true) }}with a proper pipe character.src/feed2fedi/publish.pyline 39 · line 37 — @wuming🔴 Not actioning. SSTI requires an attacker to control the template string itself, not just the interpolated values. Here, the template is a
.j2file written by the operator; RSS feed data only ever reaches the template as variable values viarender(**params), which Jinja2 evaluates as data, not as template code. Enablingautoescapewould HTML-escape every value in plain-text Fediverse posts (turning&into&etc.), which is incorrect.SandboxedEnvironmentwould restrict legitimate Jinja2 features operators may want to use. Neither change improves the actual security posture for this use case.src/feed2fedi/migrate.pyline 97 · line 112 — @wuming🔴 Not actioning, but worth explaining:
derive_template_filenamerunsre.sub(r'[^a-zA-Z0-9._-]', '_', netloc + path)on the URL, which replaces every/and\with_. The resulting filename contains no path separators, soconfig_dir / filenamecannot escapeconfig_dir—..as two dots in a filename is inert without a separator. That said, this tool only runs on URLs the operator already trusts (they are in their own config.json), so the threat model does not include a malicious feed URL.src/feed2fedi/control.pyline 116 — @wuming🔴 Not actioning. The field is retained only so
msgspeccan detect it in a JSON config and raise a clear error (seeload_config). Direct programmatic construction ofConfigurationis trusted internal code — the field docstring already marks it asLegacy field — raises error if set. Removing it would silently discard the value from deserialized JSON, which is the worse outcome.src/feed2fedi/publish.py— @wuming✅ Fixed.
render_templatenow wrapsfrom_string().render()intry/except TemplateErrorand re-raises asFeed2FediError. Test added (test_template_syntax_error_raises_feed2fedi_error).tests/unit/test_cli_apps.py— @wuming✅ Test added:
test_load_config_legacy_bot_post_template_raisesintest_cli_apps.py.src/feed2fedi/control.py— @wuming🔴 Not actioning — same reasoning as the equivalent comment in the first review round. These paths come from the user's own
config.json. An attacker who can write it already has full local access; restricting paths would prevent legitimate uses like../shared/post.j2.src/feed2fedi/migrate.py— @wuming🔴 Not actioning. feed2fedi is a local CLI tool — it has no concept of a web-accessible directory. The concern does not apply to this threat model.
src/feed2fedi/migrate.py— @wuming🔴 Not actioning. The migration tool converts from the old feed2fedi template syntax to Jinja2. In the old syntax, every
{var}was a placeholder — there was no escape mechanism for literal braces. Any{var}in an old template is intentionally converted to{{ var }}. The behavioral change (missing vars →""instead of leaving{var}intact) is intentional and documented: silently dropping an unknown variable is better than leaking raw{title}text into a Fediverse post.src/feed2fedi/migrate.py— @wuming🔴 Not actioning — this was also raised in the first review round. The scenario cannot arise in practice:
load_configraisesFeed2FediErrorif bothbot_post_templateandbot_post_template_fileare present in the JSON simultaneously. A valid working config can never have both.src/feed2fedi/migrate.py— @wuming🔴 Not actioning — same reasoning as above. The per-feed equivalent also cannot arise, as
load_configrejects any config that has bothpost_templateandpost_template_fileon the same feed.src/feed2fedi/control.py— @wuming✅ Fixed. Extracted
_read_template_filehelper that catches bothFileNotFoundErrorandUnicodeDecodeError, re-raising each asFeed2FediErrorwith a descriptive message. Applied to both the bot and feed template file reads. Test added:test_load_config_unicode_error_bot_template_file.src/feed2fedi/migrate.py— @wuming🔴 Not actioning. Python dicts preserve insertion order since 3.7, and
json.dumpsrespects that order. The only cosmetic change is thatbot_post_templateis removed andbot_post_template_fileis appended at the end — a one-time migration tool moving a key to the end of the object is acceptable.src/feed2fedi/control.py— @wuming🔴 Not actioning. Python's
pathlibcorrectly handles this:Path('/config/dir') / '/abs/path.j2'evaluates toPath('/abs/path.j2')— the absolute right-hand side replaces the left. This is documented Python behavior and means absolute paths in the config work correctly. No special-casing is needed.CLAUDE.md— @wuming🔴 Not actioning. CLAUDE.md is a project-level instruction file for Claude agents; the diff reflects edits made outside this PR's scope. The file is correct as it stands:
ruff.toml,noxfile.py,CHANGELOG.md,changelog.d/,versioninpyproject.toml, and version bump/release tasks are all still off-limits.CLAUDE.md— @wuming🔴 Not actioning. The replacement is accurate —
mypywas removed from this project and replaced withty(Astral's type checker). The diff correctly reflects the current state of the toolchain.Release-Notes.md— @wuming🔴 Not actioning — the link already uses https. The current file reads:
[Jinja2](https://jinja.palletsprojects.com/).docs/Config-File-Explained.md— @wuming🔴 Not actioning. GitHub Flavored Markdown (and Codeberg's renderer) automatically generates anchor IDs by lowercasing the heading text and replacing spaces with hyphens.
## Template Syntaxproduces the anchor#template-syntax, which is exactly what the link uses — no mismatch.tests/unit/test_feed_template_delay.py— @wuming✅ Already covered.
test_only_bot_templateintest_feed_template_delay.pyexplicitly tests that_determine_post_template(feed_template=None)and_determine_post_template(feed_template="")both return the bot template content. The removedtest_no_templatewas testing old behavior that no longer exists.src/feed2fedi/control.py— @wuming🔴 Not a bug. The code on that line is
isinstance(new_config.bot_post_visibility, str)— the correct form. The reviewer appears to have misread the code.src/feed2fedi/migrate.py— @wuming✅ Fixed.
start_migrate_shimnow catchesValueErrorfrommigrate_configand printsError: <message>before returning, giving a clean user-facing message. Test added:test_migrate_app_no_templates_prints_clean_message.src/feed2fedi/control.py— @wuming🔴 Not actioning. As noted in response to the equivalent earlier comment: Python pathlib handles this correctly —
Path(dir) / '/abs/path'→Path('/abs/path'). Absolute paths work as expected without any special-casing.src/feed2fedi/control.py— @wuming🔴 Not actioning — same reasoning. Python pathlib correctly resolves absolute right-hand sides.
src/feed2fedi/publish.py— @wuming🔴 Not actioning. This change is intentional. Leaving raw
{var}text in a Fediverse post (the old behavior) is confusing for the user's followers and was considered a bug, not a feature. Rendering missing variables as""is the better default — the{{ var | default("fallback") }}filter is available for cases where a fallback is needed.StrictUndefinedwould make templates unusable with sparse feed data (many RSS feeds omit optional fields likeauthororcontent_html).