* feat(scrape,rank): record and surface a posting's employment type
Nullable employment_type (full_time|part_time|contract|freelance) in the
seen_jobs.json entry, read from posting text at Step 3; /rank shows it on
the shortlist line and flags a mismatch with a stated profile preference
(never a veto). No pay normalisation, no new profile field.
* fix(scrape,rank): address review on employment type
- scope the never-a-veto test to the Employment type paragraph instead of
matching the phrase anywhere in rank.md
- count portal `detail` output as a source next to the posting text
- bracket the optional ` · <employment type>` so a null type leaves no
dangling separator
- drop the CHANGELOG.md hunk: the file is frozen since #800 and release notes
come from the PR title
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mv8AZjCnuGt98qsM9zuCFZ
---------
Co-authored-by: Claude <noreply@anthropic.com>
* fix(scrape): send one plain term per CLI call and make portals state their query syntax (#536)
search-queries.md is written in Google syntax, and Step 1b said to
translate its terms into portal flags without saying that OR and quotes
do not survive the translation.
- job-scraper Step 1b: one plain term per CLI call, never OR or a
combined string; results merge across calls (Step 2 dedups on URL).
Quoting follows the portal's own SKILL.md.
- Portal SKILL.md files carry a one-line "Query syntax:" note;
/add-portal asks the generator to write it for new portals.
- search-queries.md: site: blocks are labelled WebSearch-only and each
priority gains a plain-terms block for the CLIs.
- tests/test_scrape_contract.py: every installed portal SKILL.md must
carry the Query syntax line; the portal list is derived from the
skills tree.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* fix(scrape): fall back to site: terms for placeholder blocks; update freehire query syntax
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* chore: drop the CHANGELOG hunk, frozen by #800 (release notes come from the PR title)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* fix(scrape): dedup the pool by URL before Step 2 and give each portal's literal quote form
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* fix(add-portal): compare the portal's reported total, not capped rows, when probing query syntax
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
* fix(scrape): use the full job_scraper/seen_jobs.json path in every step of the spec
Step 0 reads job_scraper/seen_jobs.json but Steps 1-4 and the rules used the bare
filename. Step 4 is where the file is written, so an agent following it literally
can create seen_jobs.json in the repo root, outside the **/job_scraper/seen_jobs.json
ignore rule, where it gets committed with the user's scrape history.
Adds a spec test that rejects bare references; relaxes the existing dedup-continuity
regex to accept the qualified path.
* docs(changelog): entry for #608
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Mads Lorentzen <madslorentzen_17@hotmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* fix(scraper): make seen_jobs.json keys a pure function of the posting, and clean up the drift
The dedup key was prose only ("<url_or_company_title_key>"), so different
/scrape runs slugified company+title differently and the state file
accumulated two failures: keys carrying "&", "/", "," and ":" that break
the archive-folder path /apply and /outcome derive from company+role
(documents/README.md's subfolder rule exists because of exactly this),
and the same posting stored twice under two different truncations of a
long title (two Deloitte entries, one job, one URL).
tools/job_key.py makes the key a pure, deterministic function of
company+title+url: a strict allowlist slug, length-capped with a hash of
the full slug so truncation never collides across runs, and a fallback
to the portal's numeric job id when a non-Latin title slugifies to
nothing (a real prior entry, "securion_", would have collided with
every future non-Latin posting from that company).
--audit finds both failure classes in an existing seen_jobs.json without
guessing at a fix: malformed keys (real damage), a legacy three-part
company_title_location shape (harmless but not what the current rule
produces, so it silently re-duplicates on the next scrape), and
duplicate URLs. Ran it against this workspace's file and re-keyed the 15
entries it found - 7 malformed, 8 legacy-shape - verified byte-for-byte
against the pre-cleanup copy that no entry's data changed, only its key.
tests/test_job_key.py (16 tests) covers the slugify rules, the
truncation-hash behavior, both non-Latin fallback paths, and the audit
CLI's exit codes.
* fix(scrape): call the key helper from Step 4 instead of slugifying ad hoc
The helper added in the previous commit is only load-bearing if the spec
calls it. Step 4 described the key as prose ("<url_or_company_title_key>"),
which is what let each run slugify its own way. Step 4 now names the
command, and the schema shows the key's provenance.
Renumbers the trailing list item; no other behaviour in the step changes.
* docs(changelog): record the job-key rule under Unreleased
* fix(scrape): preserve dedup continuity across key rule
* changelog: note that existing seen_jobs.json entries need no migration (#441)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013fqqLgQSnwgWkv98twQhHi
---------
Co-authored-by: nox <nox@Mac.home>
Co-authored-by: Mads Lorentzen <madslorentzen_17@hotmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* fix(scrape): persist each posting's publication date in seen_jobs.json (#390)
Step 2's contract guarantees a `date` on every portal CLI's search output and
CI enforces it in test_scrape_contract.py; Step 3 uses that date to scope a run
to the last 14 days. Step 4's storage schema then dropped it, so a posting's age
was unrecoverable the moment the run ended - `first_seen` records when the
scraper saw an entry, not when the employer posted it. /rank reads the stored
entry rather than the run, so it had no age signal to weigh.
A freehire-search posting dated 2024-05-13 was scraped 27 months later and
ranked Strong Fit at position 1 of 133. The scoring note observed the listing
"may be long stale" in prose nothing reads, and an /apply run drafted a tailored
CV and cover letter against it.
The schema gains `posted_date` (null when the portal returned no date, never
inferred or backfilled), documented alongside `deadline` with the same
never-backfill rule. Three new cases, each verified to fail on the unfixed spec.
Closes#390
* fix(scrape): correct the 14-day scoping cross-reference, restore EOF newline
Review follow-up on #391.
The 14-day scoping is Step 1b's list item 3, not Step 3 - Step 3 is Quick Fit
Assessment and never touches dates. The "3." list item had been promoted to a
step number. Corrected in the new SKILL.md paragraph (both occurrences), the
CHANGELOG entry, and the test class docstring; a wrong pointer in a file agents
execute as instructions actively misleads.
Also restores the trailing newline on tests/test_scrape_contract.py (the nit
left for a future touch in #344) and adds the (#390) ref to the CHANGELOG entry
to match its siblings.
The Step 2 contract ('Search output already includes title, company,
location, date, and URL') had no cross-portal regression net: a CLI that
quietly drops a contract field flags the portal as degraded on every
/scrape run while CI stays green. That failure class landed for real
(jobnet/jobdanmark/jobbank, fixed in #339/#340/#342). The contract fields
are derived from the SKILL.md sentence itself (never hardcoded), compared
against the real search output of every .agents/skills/*-search CLI, and
detail.ts is deliberately excluded - the contract is about what /scrape
consumes. Fails against pre-fix master on exactly the three portals the
fixes cover; passes with them applied.
