* fix(web-research): stop treating a WebFetch 403 as a dead posting
WebFetch sends a bot user agent, and many bank and corporate sites answer
with HTTP 403 while serving the same page to a browser normally. Every
command treated that as "page unavailable" and degraded silently rather
than failing loudly:
- /rank marked live postings `expired`
- /apply fell back to search snippets, or to vague cover-letter prose
- /scrape stored listing-page `#fragment` URLs, which fetch fine and
return unrelated jobs, so every later /rank and /apply run on that
entry failed
Adds 09-web-research.md as the single reference: the trust boundary, a
curl browser-header retry with a tag-stripping extractor, a four-step
escalation order, the login-wall case, why the employer's own careers
posting beats an aggregator listing (the requisition ID and the grade
survive there), and the rule that a search-result snippet is a lead
rather than a source.
Wires it into /apply, /rank, /interview, /outcome, /notion-sync, the
job-scraper skill, and writing-style rule 5. Bumps 03-writing-style.md
to 1.2.0; 09-web-research.md starts at 1.0.0.
Aggregator examples are given generically (LinkedIn, Indeed, national
job boards) so the guidance holds in any market.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(web-research): gate the browser-header retry on robots.txt
Addresses review feedback on #277.
WebFetch identifies itself as Claude-User and honors robots.txt, so a 403 has
two very different causes and they must not be treated the same: a WAF default
on a site whose published policy allows access, or a site that has actually
declined. Retrying with browser headers in the second case circumvents the very
opt-out mechanism site owners are told they can rely on, and the core framework
cannot hold a looser standard than it asks of community forks.
The escalation now runs tools/robots_check.py before the retry. A disallow for
"*" or for "Claude-User" skips the retry entirely and goes to step 3 (find the
employer's own posting). The rule is stated plainly in 09-web-research.md so
later edits do not erode it: the retry exists to get past bot-filtering
firewalls on sites whose robots.txt permits access; it is never used to
override a site that has said no.
Two findings from testing the gate against live sites, both pinned by
tests/test_robots_check.py (15 offline cases):
- The WAF usually blocks robots.txt too. privatebank.barclays.com returns 403
on the policy file to Claude-User and 200 to a browser, so a naive gate would
block the retry on exactly the sites the retry is for. The checker reads the
policy as a browser when the honest request is refused, then obeys it
strictly - a policy you are prevented from reading cannot be honored, and
robots.txt is not the protected resource.
- urllib.robotparser cannot be used. It ends a record at a blank line and
matches rules in file order, so Barclays' real file (blank lines between
"User-agent: *" and its rules, "Allow: /" before "Disallow: /cs/") reads as
everything-allowed. That fails open, in the one direction that matters. The
checker implements RFC 9309 longest-match instead, with ties resolved to
Disallow rather than Allow.
Verified live: barclays /careers/ allowed and /cs/ blocked, ubs.com allowed,
jobup.ch /api/ blocked while /en/jobs/ stays allowed. 09-web-research.md
1.0.0 to 1.1.0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: kgb <kevingblackman@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* feat(08): add application-form fields as a third /apply artifact
/apply produces a CV and a cover letter. Many applications need a third
thing: free-text typed into a portal. Graduate programs, large-employer
ATS systems and startup forms ask for self-introductions, structured
project entries, motivation questions and pitches under a hard character
limit - none of which either document covers, and all of which the
interviewer reads alongside the CV.
Governing rule: a form field selects from what is already true and
arranges it for the question asked. It never introduces a new claim.
All accuracy rules from 03 and 05 apply unchanged.
Covers three field types (self-introduction, structured project entries,
hard character limits), the output format (a plain .txt the candidate
pastes from, with counts stated and internal NOTE TO SELF blocks marked
as not-for-pasting), and a verification checklist.
Two places where form fields are stricter than a CV, because both are
easy to get wrong:
- Project entries carry a name and a role, so they read as ownership of
the whole project in a way a terse CV bullet does not. Contributory
work has to be scoped inside the description.
- Project dates are the dates of the project, not of the employment.
Narrowing them is more accurate where the candidate can say when the
project started - but never invent a boundary to improve the ratio.
Registers the file in SKILL.md (framework_version 1.0.1 -> 1.1.0) and in
the FRAMEWORK_FILES list in tools/check_upstream_updates.py, so it is
covered by the update check like every other framework file.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fixup(08,apply): wire as optional /apply offer; align grounding to three-source union
- apply.md Step 6: offer the third artifact after CV/cover letter are
produced, mirroring the /outcome house pattern for optional capabilities
(offer, act only on yes, default output unchanged).
- 08-application-forms.md: ground claims against the framework's
01-candidate-profile.md + master CV + CLAUDE.md union (per #185)
instead of only 01, in both the governing rule and the checklist.
Per MadsLorentzen review on PR #212. Rebase onto merged #210 (the
tenure-check reference this file cites) still pending — #210 hasn't
landed on upstream/master yet.
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
CVs from /apply were named cv/main_<company>.tex, so a second role at the same company overwrote the first (cover letters already carried the role). Aligns CV naming to main_<company>_<role>.tex across apply, add-template, the CV template guide, CLAUDE.md, and SETUP.md; /outcome and /interview fallbacks glob main_<company>*.tex to match both legacy and new names. framework_version bumped on both touched framework files.
* Add framework_version markers to assistant skills and implement CI version guard and update checker
* Address review feedback: update ci.yml based on latest upstream, gate CI version guard to upstream repo, and remove non-ASCII characters from check_upstream_updates.py
