security: narrow skill grants, ship ask/deny rules, guard skill frontmatter, GET-only 403 retry (#642)

* security(skills): drop bare Bash and the portal CLI glob from skill grants

A skill's allowed-tools pre-approves the listed tools for the turn that
invokes it, even in an untrusted folder. job-application-assistant granted
bare Bash, Edit and Write; job-scraper granted bun run on any
.agents/skills/*/cli/src/cli.ts. Every Bash command the skills document is
already allowlisted per command in .claude/settings.json, so the bare and
glob grants bought nothing but exposure. The per-portal grants stay in
settings.json, which is where a portal added by /add-portal registers too
(Step 5 now says so).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* security(settings): deny exfil-shaped shell and credential reads, always ask on permission-defining files

Deny rules block in every permission mode including bypassPermissions and
override any allow rule or skill grant; ask rules force a prompt even when
a skill's allowed-tools would pre-approve the call. settings.json now
ships both: deny for shell that uploads or opens a remote shell (curl
data/form/upload flags, wget post, scp, sftp, ssh, nc) and for reads of
.env and ~/.ssh, ~/.aws, ~/.gnupg; ask for the files whose content is the
permission surface (settings, skill frontmatter, agents, CI, this guard)
and for git push. security_guards.py requires each rule to stay present,
the same way it requires the personal-data gitignore rules. Prefix rules
are tripwires, not walls; SECURITY.md will say so.

Also allowlists tools/robots_check.py and the upcoming tools/fetch_page.py
so the 403 retry no longer needs a curl grant.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* security(guards): hold skill and command allowed-tools to a reviewed allowlist

The guard allowlisted settings.json permissions but never read skill
frontmatter, so the narrow settings allowlist sat next to a bare Bash
grant it could not see. Check 4 parses every .claude/skills/*/SKILL.md,
.agents/skills/*/SKILL.md and .claude/commands/*.md frontmatter with a
stdlib parser (scalar, folded, and list forms) and holds each
allowed-tools entry to ALLOWED_SKILL_TOOLS; a portal skill may
additionally grant exactly its own cli/src/cli.ts, matched by path so no
portal name is hardcoded. Bare Bash and mid-pattern wildcards get their
own message. On master's frontmatter this check reports two failures.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* feat(tools): add GET-only fetch_page.py for the browser-header retry

09-web-research.md documented the 403 retry as a curl command with browser
headers plus an inline python3 -c tag-stripper, which meant the workflow
wanted a Bash(curl:*) grant - and curl with a free tail is an upload
channel. fetch_page.py is the same retry as one command with a fixed GET
argv (URL after --, bounded to 45 s / 5 redirects / 10 MB), http(s) only,
robots_check.gate() enforced with no bypass flag, --out refused inside
the repo, and --text for the stripped page. urllib fallback when curl is
absent. The settings.json grant names the script.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(security): describe the permission surface as a table; route the 403 retry through fetch_page.py

SECURITY.md now lists every shipped grant with its location, lifetime
and guard, and says plainly what the permission layer does not do: no
sandbox on native Windows, prefix rules are tripwires, WebFetch is
unrestricted by domain, subagents run with the parent's tools (next
change), and Edit/Write stay granted because tailoring a CV is editing
files. 09-web-research.md's retry section is the one fetch_page.py
command (framework_version 1.1.3 -> 1.1.4); apply, rank and scrape point
at it instead of curl. SETUP.md and the README tools tree follow.
CHANGELOG gains a Security section under [Unreleased].

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* security(review): ask on command frontmatter, portal skills grant only their CLI, bound the urllib fallback

Findings from a fresh-context review of the branch:

- A command's frontmatter is a grant too (check 4 already parses it), so
  Edit(.claude/commands/*.md) joins the required ask rules; without it a
  steered session could write `allowed-tools: Bash` into apply.md with no
  prompt and the next /apply would be pre-approved for bare Bash.
- SECURITY.md promised a portal skill may grant "its own CLI and nothing
  else"; the guard now enforces exactly that instead of own-CLI plus the
  core allowlist. All six shipped portal skills already comply.
- fetch_page.py's urllib fallback now matches curl's 5-redirect cap and
  applies the 10 MB cap after decompression, so the docstring's bounds
  hold on both paths.
- Frontmatter parser: a BOM no longer reads as "missing frontmatter",
  YAML flow lists split correctly, and a non-UTF-8 file fails with a
  message instead of a traceback. Module docstring lists check 4.
- CHANGELOG no longer claims "no new prompt": compile and aux cleanup,
  which the removed bare Bash covered in a skill-invoking turn, now
  prompt, deliberately. "Once per session" wording corrected to per call.
- Three upload-shaped deny siblings added (curl -XPOST, curl --request
  POST, wget --method=POST); /add-portal recon points at fetch_page.py.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>

.claude/commands/add-portal.md

@@ -31,7 +31,7 @@ Ask the user (skip anything already answered by `$ARGUMENTS`):
 
 ## Step 2: Investigate the Portal
 
-Do reconnaissance before writing any code. Use WebFetch (or `curl` via Bash) on the portal:
+Do reconnaissance before writing any code. Use WebFetch (or `python3 tools/fetch_page.py '<URL>' --text` via Bash when WebFetch is refused) on the portal:
 
 1. **Find the search URL pattern.** Load the portal's search page, run a search in the URL bar mentally or via fetch, and identify: the search endpoint, the query parameter, and any parameters for location, posting age, and pagination. Prefer a JSON API if one backs the site (check for `/api/` XHR endpoints in the page source); otherwise plan to parse the HTML results page.
 2. **Fetch one search-results response** for the test query and identify the per-result fields: **id, title, company, location, posting date, and URL**. For HTML, note the class names / attributes that anchor each field. For JSON, note the field paths.
@@ -123,6 +123,7 @@ Do not proceed to Step 5 until search, detail, and tests all pass.
 
 1. Ask whether the user wants the new portal added to their `/scrape` search strategy. If yes:
    - The portal CLI itself is already picked up automatically by `/scrape` (it discovers `.agents/skills/*/SKILL.md`) — no further wiring is needed for CLI search/detail.
+   - **Pre-approval is per portal and deliberate.** `/scrape` runs the CLI through Bash, and the only standing grants are the per-portal entries in `.claude/settings.json`. Add `Bash(bun run .agents/skills/<name>/cli/src/cli.ts:*)` to `permissions.allow` there **and** to `ALLOWED_PERMISSIONS` in `tools/security_guards.py` in the same commit - the guard fails the fork's CI on a settings entry it does not know, which is the review step working as intended. Until both are in place, `/scrape` prompts on each call of this portal's CLI; the generated `SKILL.md`'s own `allowed-tools` line covers only turns that invoke that skill directly.
    - Optionally add WebSearch/`site:` placeholder queries for that board in `.claude/skills/job-scraper/search-queries.md` (use the `[YOUR_JOB_BOARD]` style placeholders already there) so the fallback path still covers the board if the CLI is unavailable.
 2. Remind the user to add the install line for their own records if they maintain a fork README:
    ```bash