fix(scrape): make seen_jobs.json keys a pure function of the posting (#441)

* fix(scraper): make seen_jobs.json keys a pure function of the posting, and clean up the drift

The dedup key was prose only ("<url_or_company_title_key>"), so different
/scrape runs slugified company+title differently and the state file
accumulated two failures: keys carrying "&", "/", "," and ":" that break
the archive-folder path /apply and /outcome derive from company+role
(documents/README.md's subfolder rule exists because of exactly this),
and the same posting stored twice under two different truncations of a
long title (two Deloitte entries, one job, one URL).

tools/job_key.py makes the key a pure, deterministic function of
company+title+url: a strict allowlist slug, length-capped with a hash of
the full slug so truncation never collides across runs, and a fallback
to the portal's numeric job id when a non-Latin title slugifies to
nothing (a real prior entry, "securion_", would have collided with
every future non-Latin posting from that company).

--audit finds both failure classes in an existing seen_jobs.json without
guessing at a fix: malformed keys (real damage), a legacy three-part
company_title_location shape (harmless but not what the current rule
produces, so it silently re-duplicates on the next scrape), and
duplicate URLs. Ran it against this workspace's file and re-keyed the 15
entries it found - 7 malformed, 8 legacy-shape - verified byte-for-byte
against the pre-cleanup copy that no entry's data changed, only its key.

tests/test_job_key.py (16 tests) covers the slugify rules, the
truncation-hash behavior, both non-Latin fallback paths, and the audit
CLI's exit codes.

* fix(scrape): call the key helper from Step 4 instead of slugifying ad hoc

The helper added in the previous commit is only load-bearing if the spec
calls it. Step 4 described the key as prose ("<url_or_company_title_key>"),
which is what let each run slugify its own way. Step 4 now names the
command, and the schema shows the key's provenance.

Renumbers the trailing list item; no other behaviour in the step changes.

* docs(changelog): record the job-key rule under Unreleased

* fix(scrape): preserve dedup continuity across key rule

* changelog: note that existing seen_jobs.json entries need no migration (#441)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013fqqLgQSnwgWkv98twQhHi

---------

Co-authored-by: nox <nox@Mac.home>
Co-authored-by: Mads Lorentzen <madslorentzen_17@hotmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>

.claude/skills/job-scraper/SKILL.md

@@ -5,7 +5,7 @@ description: >
   (LinkedIn, local job boards, and any skills added with /add-portal). Deduplicates
   across runs. Triggers on: job scrape, find jobs, search jobs, new jobs, job search,
   scrape jobs, /scrape
-allowed-tools: Read, Write, Edit, Glob, Grep, Bash(bun --version), Bash(bun run .agents/skills/*/cli/src/cli.ts *), WebFetch, WebSearch, Agent, AskUserQuestion
+allowed-tools: Read, Write, Edit, Glob, Grep, Bash(bun --version), Bash(bun run .agents/skills/*/cli/src/cli.ts *), Bash(python tools/job_key.py:*), Bash(python3 tools/job_key.py:*), WebFetch, WebSearch, Agent, AskUserQuestion
 ---
 
 # Job Scraper
@@ -117,7 +117,10 @@ site for the role and store that URL instead, or drop the candidate rather than
 fragment link.
 
 For every candidate:
-- Skip if the URL or company+title combo already exists in `seen_jobs.json`
+- Skip if the URL matches any existing `seen_jobs.json` entry, regardless of
+  that entry's key. This preserves dedup continuity for postings stored under
+  the pre-helper key rule while new entries use the canonical key from Step 4.
+- Otherwise, skip if the company+title combo already exists in `seen_jobs.json`
 - Skip if the company+role already appears in `job_search_tracker.csv`
 
 ### Step 2.5: Mass-Posting Detection (within this run)
@@ -138,11 +141,19 @@ For each new job, do a rapid fit check (NOT the full evaluation from `04-job-eva
 
 ### Step 4: Deduplicate & Store
 
-1. Add ALL fetched jobs (new and skipped) to `seen_jobs.json` with structure:
+1. Derive each entry's key with the helper, never by slugifying in the moment:
+
+```bash
+python3 tools/job_key.py --company "<company>" --title "<title>" --url "<url>"
+```
+
+It prints one line: the canonical key for that posting. The key must be a pure function of the posting, because two runs that slugify differently store the same job twice and defeat the dedup this step exists to provide. The helper also length-caps long titles and disambiguates the cap with a hash of the full slug, so a truncated title is stable across runs and two different long titles never collide. `python3 tools/job_key.py --audit` reports entries in an existing state file that predate this rule; it only reports, and never rewrites keys, since a rewritten key breaks the tracker's own company+role matching.
+
+2. Add ALL fetched jobs (new and skipped) to `seen_jobs.json` with structure:
 ```json
 {
   "seen": {
-    "<url_or_company_title_key>": {
+    "<key from tools/job_key.py>": {
       "title": "...",
       "company": "...",
       "url": "...",
@@ -168,7 +179,8 @@ The `source` field records which mechanism produced the entry: `cli` for Step 1b
 
 `posted_date` is the posting's own publication date, taken from the `date` field Step 2's contract already guarantees on every portal CLI's search output. Step 1b uses that date to scope the run to the last 14 days and then drops it, so nothing downstream can distinguish a posting published yesterday from one published two years ago - `first_seen` is when this scraper first saw the entry, not when the employer posted it. Persisting it makes Step 1b's window auditable after the run and gives `/rank` a freshness signal to weigh, instead of rediscovering the date and recording it in prose that nothing reads. That gap landed for real: a freehire-search posting dated 2024-05-13 was scraped and ranked Strong Fit at position 1 of 133, its own scoring note observing the listing "may be long stale" with nothing able to act on it. `null` means the portal returned no date for that result (the CLIs emit `date: null` when a listing omits it); a missing key means the entry predates this field - **never infer a posting date** from either, and never backfill by guessing.
 
-2. Only present jobs NOT already in the seen list or tracker.
+3. Only present jobs NOT already in the seen list (matched by URL or
+   company+title) or tracker.
 
 ### Step 4.5: Generate Referral Contact Links (High & Medium Fit Only)