feat(scraper): record whether each seen job came from a CLI or the WebSearch fallback (#338)

Adds an additive source field (cli/websearch) to the seen_jobs Step 4 schema, Step 1c tagging at collection time, and a 'fallback (websearch):' Step 5 summary line - so future ghost-job reports (#331) self-triage from stored state. Same additive-field contract as portal/deadline: never backfilled.

Co-authored-by: yshraj <87583119+yshraj@users.noreply.github.com>
This commit is contained in:
Yash Rajeshbhai Darji
2026-08-18 21:06:16 +02:00
committed by GitHub
parent 0cf2ce0d36
commit f136b534de
3 changed files with 109 additions and 2 deletions
+14
View File
@@ -13,6 +13,20 @@ per-file diff commands.
## [Unreleased]
### Added
- **`seen_jobs.json` entries record which mechanism produced them** - a new additive `source`
field (`cli` for Step 1b portal-CLI output, `websearch` for the Step 1c fallback), a Step 1c
rule tagging fallback results at collection time, and a `fallback (websearch):` line in the
Step 5 run summary naming the portals that ran on the fallback. Motivated by the
ghost-LinkedIn-jobs report (#331): when a stored job later turns out not to exist at its URL,
triage hinges on whether the entry came from live CLI output or a search index that can be
weeks stale - evidence that previously lived only in the run's scrollback. An entry that is
missing `source` predates the field and is never backfilled; a presented job with no
`seen_jobs.json` entry at all points at fabrication, which the scraper's Rule 1 forbids.
Pinned by `tests/test_scrape_provenance.py`. `job-scraper/SKILL.md` sits outside the
`framework_version`-marked set, so no version bump applies.
### Changed
- **Job matching reframed around function, not title** (`framework_version` 1.2.2 -> 1.2.3 in