mirror of https://github.com/MadsLorentzen/ai-job-search.git synced 2026-09-17 08:36:25 +00:00

Files

T

fcefb8150f fix(web-research): stop treating a WebFetch 403 as a dead posting (#277 )

* fix(web-research): stop treating a WebFetch 403 as a dead posting

WebFetch sends a bot user agent, and many bank and corporate sites answer
with HTTP 403 while serving the same page to a browser normally. Every
command treated that as "page unavailable" and degraded silently rather
than failing loudly:

- /rank marked live postings `expired`
- /apply fell back to search snippets, or to vague cover-letter prose
- /scrape stored listing-page `#fragment` URLs, which fetch fine and
  return unrelated jobs, so every later /rank and /apply run on that
  entry failed

Adds 09-web-research.md as the single reference: the trust boundary, a
curl browser-header retry with a tag-stripping extractor, a four-step
escalation order, the login-wall case, why the employer's own careers
posting beats an aggregator listing (the requisition ID and the grade
survive there), and the rule that a search-result snippet is a lead
rather than a source.

Wires it into /apply, /rank, /interview, /outcome, /notion-sync, the
job-scraper skill, and writing-style rule 5. Bumps 03-writing-style.md
to 1.2.0; 09-web-research.md starts at 1.0.0.

Aggregator examples are given generically (LinkedIn, Indeed, national
job boards) so the guidance holds in any market.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web-research): gate the browser-header retry on robots.txt

Addresses review feedback on #277.

WebFetch identifies itself as Claude-User and honors robots.txt, so a 403 has
two very different causes and they must not be treated the same: a WAF default
on a site whose published policy allows access, or a site that has actually
declined. Retrying with browser headers in the second case circumvents the very
opt-out mechanism site owners are told they can rely on, and the core framework
cannot hold a looser standard than it asks of community forks.

The escalation now runs tools/robots_check.py before the retry. A disallow for
"*" or for "Claude-User" skips the retry entirely and goes to step 3 (find the
employer's own posting). The rule is stated plainly in 09-web-research.md so
later edits do not erode it: the retry exists to get past bot-filtering
firewalls on sites whose robots.txt permits access; it is never used to
override a site that has said no.

Two findings from testing the gate against live sites, both pinned by
tests/test_robots_check.py (15 offline cases):

- The WAF usually blocks robots.txt too. privatebank.barclays.com returns 403
  on the policy file to Claude-User and 200 to a browser, so a naive gate would
  block the retry on exactly the sites the retry is for. The checker reads the
  policy as a browser when the honest request is refused, then obeys it
  strictly - a policy you are prevented from reading cannot be honored, and
  robots.txt is not the protected resource.
- urllib.robotparser cannot be used. It ends a record at a blank line and
  matches rules in file order, so Barclays' real file (blank lines between
  "User-agent: *" and its rules, "Allow: /" before "Disallow: /cs/") reads as
  everything-allowed. That fails open, in the one direction that matters. The
  checker implements RFC 9309 longest-match instead, with ties resolved to
  Disallow rather than Allow.

Verified live: barclays /careers/ allowed and /cs/ blocked, ubs.com allowed,
jobup.ch /api/ blocked while /en/jobs/ stays allowed. 09-web-research.md
1.0.0 to 1.1.0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: kgb <kevingblackman@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

2026-08-04 20:36:55 +02:00

9.6 KiB

Raw Blame History

/interview - Prepare for an Interview on a Tracked Application

You are preparing the user for a real, scheduled interview on one of their applications. The frameworks for this already exist - 07-interview-prep.md (STAR examples, tough questions, questions to ask, roleplay protocol) and the Company Research Checklist in 04-job-evaluation.md - and the /outcome archive records which stage the user is at and what earlier stages surfaced. This command wires them together into a stage-specific prep pack and an optional mock interview.

/apply optimizes what the company reads; /interview optimizes what the company hears. The bridge between them is consistency: the interviewer has read the submitted CV and cover letter, so everything prepared here must match what those documents claim.

Follow these steps in order.

Step 0: Parse Input

$ARGUMENTS may contain a company name (optionally with a role), e.g. /interview acme.

With an argument: match against job_search_tracker.csv rows (case-insensitive on company, then role). One match → proceed. Several → list and ask. None → this application isn't tracked; suggest /outcome <company> to register it first, or accept the posting and role details directly if the user wants to prep anyway.
Without an argument: list tracker rows whose status suggests a live process (interview, offer, or recently applied) and ask which one. If the tracker is empty, ask for the company, role, and posting.

v1 preps for a specific application. Generic no-target practice is out of scope - if asked, prep against a real tracked application instead.

Step 1: Load the Application Context

The archive (maintained by /outcome): documents/applications/<company>_<role>/
- job_posting.md - the exact posting the user applied to
- cv_draft.tex and cover_letter.tex - what was actually submitted. These are what the interviewer read; every talking point must be consistent with their claims.
- outcome.md - the stage reached so far and any recorded feedback from earlier stages. Feedback from stage N is the highest-value input for stage N+1 prep.
Fallbacks (the application may predate /outcome): posting via WebFetch on the tracker row's source URL, or ask the user to paste it; CV via cv/main_<company>*.tex and cover letter via cover_letters/cover_<company>_*.tex. State plainly which context is missing rather than guessing - and suggest /outcome <company> to build the archive for next time.
Ask the user what this interview is (skip anything outcome.md already records): stage (phone screen / technical / case / final round), date, format (phone, video, onsite), and who is interviewing (names and titles, if known).
Read the frameworks once - do not re-read them in later steps:
- .claude/skills/job-application-assistant/07-interview-prep.md
- .claude/skills/job-application-assistant/01-candidate-profile.md
- .claude/skills/job-application-assistant/02-behavioral-profile.md
- .claude/skills/job-application-assistant/04-job-evaluation.md

Step 2: Research the Company (Interview-Focused)

Execute the Company Research Checklist that 04-job-evaluation.md defines: company website (mission, values, recent news), review sites, LinkedIn (team size, recent hires), and media coverage (growth, restructuring, workplace issues).

Additions for interview purposes:

Interviewer angle: if interviewer names are known (from Step 1 or the tracker's contact_person), look up their public professional profile. A hiring manager probes team fit and motivation; a senior engineer probes technical depth; HR probes the CV timeline. Note the likely angle per interviewer - do not speculate beyond public information.
Conversation hooks: 2-3 recent, verifiable company specifics (a product launch, a stated strategic priority) the user can reference naturally in answers and in the "why this company" moment.

Verify before using: every company claim that will appear in the prep pack must be independently confirmed via WebFetch/WebSearch - same rule the repo applies to cover-letter claims. An unverified "fact" delivered confidently in an interview is worse than no fact. On a 403, retry with browser headers per .claude/skills/job-application-assistant/09-web-research.md rather than dropping to search snippets; a snippet is a lead, not a source.

Step 3: Build the Prep Pack

Assemble a stage-appropriate prep document with these sections:

1. Likely questions

Derive from four sources, in priority order:

Recorded feedback from earlier stages (outcome.md) - anything flagged, doubted, or left unresolved will come back
The fit evaluation's gaps - the requirements where the profile is weakest are the likeliest probes. For each, prepare an honest bridge answer per 07's "You don't have [X]" pattern: acknowledge, connect adjacent experience, show the learning path. Never prepare an answer that invents experience.
The posting's stated requirements - competency by competency
The stage type - phone screens get motivation and timeline questions; technical rounds get the posting's stack; final rounds get values, salary, and "any reservations" questions

2. STAR answer mapping

Match the ready-made STAR examples in 07-interview-prep.md to the likely questions using their "Use for" tags. Then:

For likely questions no existing STAR example covers, draft a new STAR answer grounded strictly in facts from 01-candidate-profile.md - profile facts arranged into S/T/A/R, not embellished. Include these drafts in the prep pack; offer to append them to 07-interview-prep.md only if the user explicitly approves.
If /setup left incomplete STAR stubs relevant to this role, surface them: the user should fill in the details before the interview.

3. Consistency brief

A short list of the specific claims the submitted CV and cover letter make (achievements, numbers, skills emphasized) that the interviewer is most likely to probe. The rule stated plainly: no claim in the room that isn't on the paper, and every claim on the paper must be defensible in depth.

4. Tough questions, customized

The relevant entries from 07's tough-question list with per-application answers - "Why this company specifically?" must use the verified hooks from Step 2, never a generic line.

5. Questions to ask

Pick 4-6 from 07's categories, customized to the research and the stage: role and team questions at screens, tech and growth questions at technical rounds, culture and leadership questions by the final round (that is the last chance to detect a deal-breaker). Cut any question the research already answers publicly - asking it signals you didn't look.

6. Logistics

The phone/video tips from 07 when the format calls for them, plus date and interviewer names as a header.

Save the pack to documents/applications/<company>_<role>/interview_prep_<stage>.md (create the folder if this application predates /outcome). The folder is gitignored, so the pack stays personal; one file per stage, so earlier packs remain as history. Present the pack in chat as well - the file is the artifact, the conversation is the delivery.

Step 4: Offer a Mock Interview

Ask if the user wants to practice. If yes, run the roleplay in this conversation following the Roleplay Guidelines in 07-interview-prep.md exactly: warm-up first, then role-specific technical questions, 1-2 behavioral questions tied to the posting's competencies, and one tough question or curveball. After each answer, give brief feedback - what worked, what to sharpen, and which STAR example from the pack would have served better.

Calibrate feedback against 02-behavioral-profile.md: coach toward the user's natural register, not a generic ideal - the same voice-consistency rule the /apply reviewer applies to cover letters.

Step 5: Close the Loop

End with:

Good luck. After the interview, run /outcome <company> to log the stage and any feedback - it sharpens the prep for the next round, and once the process resolves it feeds your fit-framework calibration via /setup.

If Step 3 drafted new STAR answers the user approved for keeps, remind them those were appended to 07-interview-prep.md (or offer again if they deferred).

Important Rules

Consistency with the submitted documents. The interviewer read the archived CV and cover letter; prep must never contradict them or coach claims beyond them.
Honesty on gaps. Weak matches get bridge answers (acknowledge → adjacent experience → learning path), never invented experience. Same rule as everywhere else in this repo.
Verified research only. Company specifics go in the pack only after independent confirmation. Interviewer notes stick to public professional information.
Stage-appropriate prep. A phone screen pack and a final-round pack are different documents; recorded feedback from earlier stages takes priority over generic question lists.
Write only to the application archive — with one exception. The prep pack lands in documents/applications/<company>_<role>/; framework files are not edited, except appending user-approved STAR examples to 07-interview-prep.md on explicit request.

The exception is 01-candidate-profile.md. Interview prep is where new facts surface most often: the user recalls a metric, corrects a scope, or fills in a STAR stub. When that happens, write the fact into the profile, as well as putting it in the prep pack. A fact recorded only in prep material reads as unsupported to a later drafting session and gets stripped from CVs as a fabrication. Prep files are not a substitute for the profile.

9.6 KiB Raw Blame History