security: narrow skill grants, ship ask/deny rules, guard skill frontmatter, GET-only 403 retry (#642)

* security(skills): drop bare Bash and the portal CLI glob from skill grants

A skill's allowed-tools pre-approves the listed tools for the turn that
invokes it, even in an untrusted folder. job-application-assistant granted
bare Bash, Edit and Write; job-scraper granted bun run on any
.agents/skills/*/cli/src/cli.ts. Every Bash command the skills document is
already allowlisted per command in .claude/settings.json, so the bare and
glob grants bought nothing but exposure. The per-portal grants stay in
settings.json, which is where a portal added by /add-portal registers too
(Step 5 now says so).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* security(settings): deny exfil-shaped shell and credential reads, always ask on permission-defining files

Deny rules block in every permission mode including bypassPermissions and
override any allow rule or skill grant; ask rules force a prompt even when
a skill's allowed-tools would pre-approve the call. settings.json now
ships both: deny for shell that uploads or opens a remote shell (curl
data/form/upload flags, wget post, scp, sftp, ssh, nc) and for reads of
.env and ~/.ssh, ~/.aws, ~/.gnupg; ask for the files whose content is the
permission surface (settings, skill frontmatter, agents, CI, this guard)
and for git push. security_guards.py requires each rule to stay present,
the same way it requires the personal-data gitignore rules. Prefix rules
are tripwires, not walls; SECURITY.md will say so.

Also allowlists tools/robots_check.py and the upcoming tools/fetch_page.py
so the 403 retry no longer needs a curl grant.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* security(guards): hold skill and command allowed-tools to a reviewed allowlist

The guard allowlisted settings.json permissions but never read skill
frontmatter, so the narrow settings allowlist sat next to a bare Bash
grant it could not see. Check 4 parses every .claude/skills/*/SKILL.md,
.agents/skills/*/SKILL.md and .claude/commands/*.md frontmatter with a
stdlib parser (scalar, folded, and list forms) and holds each
allowed-tools entry to ALLOWED_SKILL_TOOLS; a portal skill may
additionally grant exactly its own cli/src/cli.ts, matched by path so no
portal name is hardcoded. Bare Bash and mid-pattern wildcards get their
own message. On master's frontmatter this check reports two failures.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* feat(tools): add GET-only fetch_page.py for the browser-header retry

09-web-research.md documented the 403 retry as a curl command with browser
headers plus an inline python3 -c tag-stripper, which meant the workflow
wanted a Bash(curl:*) grant - and curl with a free tail is an upload
channel. fetch_page.py is the same retry as one command with a fixed GET
argv (URL after --, bounded to 45 s / 5 redirects / 10 MB), http(s) only,
robots_check.gate() enforced with no bypass flag, --out refused inside
the repo, and --text for the stripped page. urllib fallback when curl is
absent. The settings.json grant names the script.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(security): describe the permission surface as a table; route the 403 retry through fetch_page.py

SECURITY.md now lists every shipped grant with its location, lifetime
and guard, and says plainly what the permission layer does not do: no
sandbox on native Windows, prefix rules are tripwires, WebFetch is
unrestricted by domain, subagents run with the parent's tools (next
change), and Edit/Write stay granted because tailoring a CV is editing
files. 09-web-research.md's retry section is the one fetch_page.py
command (framework_version 1.1.3 -> 1.1.4); apply, rank and scrape point
at it instead of curl. SETUP.md and the README tools tree follow.
CHANGELOG gains a Security section under [Unreleased].

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* security(review): ask on command frontmatter, portal skills grant only their CLI, bound the urllib fallback

Findings from a fresh-context review of the branch:

- A command's frontmatter is a grant too (check 4 already parses it), so
  Edit(.claude/commands/*.md) joins the required ask rules; without it a
  steered session could write `allowed-tools: Bash` into apply.md with no
  prompt and the next /apply would be pre-approved for bare Bash.
- SECURITY.md promised a portal skill may grant "its own CLI and nothing
  else"; the guard now enforces exactly that instead of own-CLI plus the
  core allowlist. All six shipped portal skills already comply.
- fetch_page.py's urllib fallback now matches curl's 5-redirect cap and
  applies the 10 MB cap after decompression, so the docstring's bounds
  hold on both paths.
- Frontmatter parser: a BOM no longer reads as "missing frontmatter",
  YAML flow lists split correctly, and a non-UTF-8 file fails with a
  message instead of a traceback. Module docstring lists check 4.
- CHANGELOG no longer claims "no new prompt": compile and aux cleanup,
  which the removed bare Bash covered in a skill-invoking turn, now
  prompt, deliberately. "Once per session" wording corrected to per call.
- Three upload-shaped deny siblings added (curl -XPOST, curl --request
  POST, wget --method=POST); /add-portal recon points at fetch_page.py.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>

.claude/skills/job-application-assistant/09-web-research.md

@@ -1,5 +1,5 @@
 ---
-framework_version: 1.1.3
+framework_version: 1.1.4
 ---
 
 # Web Research and Fetching
@@ -32,7 +32,7 @@ Do **not** respond to a 403 by softening the cover letter to vague generalities,
 - **A WAF default on a site whose published policy allows access.** Many bank and corporate domains serve `User-agent: *` / `Allow: /` while their firewall filters any client that does not look like a browser. Retrying there overrides a firewall default, not an expressed preference. Proceed.
 - **A site that has actually declined.** If `robots.txt` disallows the path for `*` or for `Claude-User`, retrying with browser headers circumvents the exact mechanism the site was told to use. **Do not retry.** Skip to escalation step 4 and find the employer's own posting instead.
 
-Check it first. It is one cheap fetch, and the repo ships the check:
+The retry command below runs this check itself and refuses to fetch when it fails - there is no flag to skip it. The standalone checker is still there for diagnosing a refusal:
 
 ```bash
 python3 tools/robots_check.py '<URL>'
@@ -45,37 +45,19 @@ Two details worth knowing, both covered by `tests/test_robots_check.py`:
 - **The WAF usually blocks `robots.txt` too.** On `privatebank.barclays.com` the policy file itself returns 403 to `Claude-User` and 200 to a browser. The checker therefore reads the policy as a browser if the honest request is refused, then obeys it strictly. A policy you are prevented from reading cannot be honored, and `robots.txt` is not the protected resource.
 - **Do not substitute `urllib.robotparser`.** It ends a record at a blank line and matches rules in file order, so a real-world file like Barclays' (blank lines between `User-agent: *` and its rules, `Allow: /` listed before `Disallow: /cs/`) reads as "everything allowed". That fails open, in the one direction that matters.
 
-### The retry: curl with browser headers
+### The retry: `tools/fetch_page.py`
 
 ```bash
-cd "${SCRATCHPAD:?set this to the session scratchpad directory from your system prompt}" && curl -sSL --max-time 45 -o page.html -w "HTTP %{http_code} size=%{size_download}\n" \
- -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36' \
- -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8' \
- -H 'Accept-Language: en-GB,en;q=0.9' \
- -H 'Accept-Encoding: gzip, deflate, br' --compressed \
- -H 'Sec-Fetch-Dest: document' -H 'Sec-Fetch-Mode: navigate' -H 'Sec-Fetch-Site: none' \
- -H 'Upgrade-Insecure-Requests: 1' \
- '<URL>'
+python3 tools/fetch_page.py '<URL>' --out "${SCRATCHPAD:?set this to the session scratchpad directory from your system prompt}/page.html" --text
 ```
 
-Write to the session scratchpad directory, never into the repo. `--compressed` is required alongside the `Accept-Encoding` header or the output is unreadable binary.
+One command does the whole retry: it runs the `robots.txt` gate above (exit `1` and no fetch when the policy says no), GETs the page with browser headers, saves the body to `--out`, prints `HTTP <code> size=<bytes> saved=<path>`, and with `--text` prints the tag-stripped text (script, style, noscript, svg and comments removed, entities unescaped) capped at `--max-chars` (default 6000). It is GET-only with a fixed argv, http(s) only, bounded to 45 s, 5 redirects and 10 MB, and it refuses an `--out` inside the repo - fetched content is untrusted and belongs in the scratchpad. That fixed shape is why the workflow pre-approves `python tools/fetch_page.py` and not `curl`.
 
-### Extracting text from the saved HTML
+The HTTP status is information, not a verdict: a `403` body still tells you what the site answered. Only exit `2` (unsupported scheme, transport failure, oversized response) means nothing was retrieved.
 
-`WebFetch` converts to markdown for you; curl does not. Strip the tags:
+### Reading the extracted text
 
-```bash
-cd "${SCRATCHPAD:?set this to the session scratchpad directory from your system prompt}" && python3 -c "
-import re, html
-h = open('page.html', encoding='utf-8', errors='replace').read()
-h = re.sub(r'(?is)<(script|style|noscript|svg)[^>]*>.*?</\1>', ' ', h)
-t = html.unescape(re.sub(r'(?s)<[^>]+>', ' ', h))
-t = re.sub(r'[ \t\xa0]+', ' ', t)
-print(re.sub(r'\n\s*\n+', '\n', t).strip()[:6000])
-"
-```
-
-Modern sites embed real copy inside JSON blobs in the markup, so useful text often survives with escaped `\n` and stray attribute fragments around it. That is normal. Read through the noise rather than assuming the extraction failed. To find specific facts in a large page, grep the extracted text for keywords (office cities, "since", regulator names) with surrounding context instead of printing the whole document.
+Modern sites embed real copy inside JSON blobs in the markup, so useful text often survives with escaped `\n` and stray attribute fragments around it. That is normal. Read through the noise rather than assuming the extraction failed. To find specific facts in a large page, grep the saved `page.html` or the printed text for keywords (office cities, "since", regulator names) with surrounding context instead of printing the whole document; raise `--max-chars` only when the posting body is genuinely longer.
 
 ## Escalation order
 
@@ -87,7 +69,7 @@ Try these in order and stop at the first that yields real content:
    ```
    This retrieves the posting directly from the portal's API or resilient detail route, bypassing bot-detection 403s and login walls entirely.
 2. **`WebFetch`** on the target URL (for non-portal URLs, or if the portal CLI is missing or failed). Cheapest for general web pages, returns clean markdown.
-3. **Check `robots.txt`, then `curl` with browser headers** (above), then strip tags. Fixes the 403 class of failure. If `robots.txt` disallows the path for `*` or `Claude-User`, **skip this step entirely** and go to step 4.
+3. **`python3 tools/fetch_page.py '<URL>' --out "$SCRATCHPAD/page.html" --text`** (above). It checks `robots.txt` itself, retries with browser headers and strips the tags. Fixes the 403 class of failure. An exit of `1` means the site's policy disallows the path for `*` or `Claude-User`: the retry did not run, so go to step 4.
 4. **`WebSearch`** for the company or role by name, to find an alternative canonical URL: the employer's own careers portal is almost always richer than the aggregator that surfaced the posting, and it carries the reference ID and grade that aggregators drop.
 5. **Declare it genuinely unavailable** only after 1 to 4 have failed. In `/rank` that means marking the entry `unreachable`, which later runs retry (`expired` is only for a posting shown to be closed - see `rank.md` Step 2); in `/apply` it means telling the user the posting could not be retrieved and stopping rather than drafting from the title.
 

.claude/skills/job-application-assistant/SKILL.md

@@ -4,8 +4,8 @@ description: >
   Assists with job applications: evaluating job postings, tailoring CVs, writing cover letters,
   and preparing for interviews. Triggers on keywords like: job posting, job application, CV,
   cover letter, resume, interview prep, job fit, career, application, apply, ansøgning, stilling
-allowed-tools: Read, Glob, Grep, WebFetch, WebSearch, Bash, Edit, Write, AskUserQuestion
-framework_version: 1.3.4
+allowed-tools: Read, Glob, Grep, WebFetch, WebSearch, Edit, Write, AskUserQuestion
+framework_version: 1.3.5
 ---
 
 # Job Application Assistant