fix(posting_guard): count invisible characters inside hidden HTML and clean the hidden-block excerpts

#702 removes, counts and flags Unicode tag characters in the visible
text, but guard() discarded the counts from cleaning the hidden text and
returned the hidden_blocks excerpts raw. A tag-spelled instruction inside
a display:none block therefore got the clean verdict with invisible_chars
empty, and its 28 tag characters travelled on in the findings block that
posting-fetcher forwards verbatim to the main agent.

The hidden text's invisible-character counts and flags now merge into the
findings, so they raise the same alarm as visible ones, and each excerpt
is cleaned before it is cut to EXCERPT characters, so a run of invisible
characters at the front of a block cannot leave an empty excerpt. Three
new tests, all failing before this change; the proof of concept from the
report now reads "28 tag characters (text hidden from the reader)" with
the excerpt reduced to "note".

Reported privately by balhyo-younjisang (GHSA-v7p7-p2vp-82vg).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Mads LorentzenandClaude Fable 5.1 authored and Mads Lorentzen committed 2026-10-27 16:07:33 +01:00
1 parent 07a77acbb0
commit ae6c5aded5
3 files changed
+48 -3

No files matched your search

+12 -3
View File
@@ -300,7 +300,11 @@ def clean_html(markup: str):
hidden_buf.clear()
hidden_at = None
if block:
hidden_blocks.append(block[:EXCERPT])
# The excerpt is reported verbatim to the agent, so it carries no
# invisible characters; the raw block goes to hidden_text, where
# guard() counts and decodes them. Clean before cutting, or a run of
# invisible characters at the front would leave an empty excerpt.
hidden_blocks.append(clean_text(block)[0][:EXCERPT])
hidden_parts.append(block)
def pop_to(index):
@@ -439,8 +443,13 @@ def guard(raw: str, mode: str):
text, hidden_text, hidden_blocks = raw, "", []
text, tf = clean_text(text)
hidden_text, hidden_tf = clean_text(hidden_text)
# What invisible characters spell is hidden content: scanned and reported,
# never part of the cleaned text.
# Invisible characters inside hidden content count and alarm like visible
# ones: a tag-spelled instruction in a display:none block is the same
# payload, only better hidden. What they spell is hidden content: scanned
# and reported, never part of the cleaned text.
for key, n in hidden_tf["invisible_chars"].items():
tf["invisible_chars"][key] = tf["invisible_chars"].get(key, 0) + n
tf["invisible_flagged"] += hidden_tf["invisible_flagged"]
tf["invisible_text"] += hidden_tf["invisible_text"]
hidden_text = "\n".join([hidden_text, *tf["invisible_text"]])
findings = {"hidden_blocks": hidden_blocks, **tf, **scan(text, hidden_text)}