feat(scraper): record whether each seen job came from a CLI or the WebSearch fallback (#338)

Adds an additive source field (cli/websearch) to the seen_jobs Step 4 schema, Step 1c tagging at collection time, and a 'fallback (websearch):' Step 5 summary line - so future ghost-job reports (#331) self-triage from stored state. Same additive-field contract as portal/deadline: never backfilled.

Co-authored-by: yshraj <87583119+yshraj@users.noreply.github.com>

.claude/skills/job-scraper/SKILL.md

@@ -83,6 +83,8 @@ Use `WebSearch` for:
 
 Use the site-specific query strings from `search-queries.md` directly as WebSearch queries for these portals.
 
+Tag each fallback result as WebSearch-sourced, keeping the portal tag when the fallback stands in for an installed portal whose CLI failed. Step 4 persists this as the entry's `source`, and Step 5 reports which portals ran on the fallback this run.
+
 ### Step 2: Fetch & Parse
 
 For each promising result from Step 1:
@@ -138,7 +140,8 @@ For each new job, do a rapid fit check (NOT the full evaluation from `04-job-eva
       "deadline": "YYYY-MM-DD" | null,
       "fit": "high/medium/low",
       "status": "new/skipped/ranked/expired",
-      "portal": "<source portal skill, e.g. jobindex-search>"
+      "portal": "<source portal skill, e.g. jobindex-search>",
+      "source": "cli/websearch"
     }
   }
 }
@@ -146,6 +149,8 @@ For each new job, do a rapid fit check (NOT the full evaluation from `04-job-eva
 
 The `portal` field records which CLI skill produced the job (results are already tagged per portal in Step 1b - persist that tag here). Entries written before this field existed lack it; the health check (Step 4.75) attributes those by matching the URL's domain against each portal's base URL, so do not backfill.
 
+The `source` field records which mechanism produced the entry: `cli` for Step 1b portal-CLI output, `websearch` for the Step 1c fallback. This is what keeps a ghost-job report diagnosable after the run's summary is gone: a stored entry whose URL later resolves to nothing (or to a different job) reads very differently depending on whether it came from live CLI output or from a search index that can be weeks stale - and a presented job with no entry here at all points at fabrication, which Rule 1 forbids. Entries written before this field existed lack it; never backfill it - the mechanism was not recorded.
+
 `/rank` extends this schema additively: ranked entries also carry `rank_score` (0–100 overall score), `rank_verdict` (fit band, e.g. "strong fit"), `rank_date` (ISO date of ranking), and `strengths`/`gaps` (1-3 verbatim bullets each, copied from the scoring agent's findings). The `status` field is set to `"ranked"`. Do not drop any of these fields when re-writing entries. Entries ranked before `strengths`/`gaps` existed simply lack them; readers tolerate their absence and never backfill by guessing.
 
 `deadline` is a base field rather than a `/rank` extension: Step 2's detail fetch already extracts the application deadline, so it is written when the job is first seen and refreshed by `/rank` Step 4 when a scoring agent returns a different value. `null` means the posting states no deadline; a missing key means the entry predates this field - **never infer a deadline** from either, and never backfill by guessing.
@@ -196,7 +201,11 @@ Scraper-based portal CLIs rot silently: when a portal changes its markup, the pa
 Present new jobs in a table sorted by fit (high first). When Step 1b skipped
 portals (`enabled: false`), report them with the `skipped (disabled):` line below
 so opting one out stays visible rather than silent; omit the line when nothing
-was skipped. When Step 4.75 found a portal degraded, broken, or inconclusive,
+was skipped. When any portal's results came from the Step 1c fallback this run
+(bun unavailable, or its CLI failed at runtime), report it with the
+`fallback (websearch):` line - fallback results come from a search index that
+can be stale, so the reader should know which rows carry that caveat; omit the
+line when every portal ran its CLI. When Step 4.75 found a portal degraded, broken, or inconclusive,
 add one `health:` line per suspect portal (healthy portals get no line); after
 the report, offer to set that portal's `enabled: false` so `/scrape` stops
 running it (and covers it via the Step 1c fallback) until it is fixed - only
@@ -210,6 +219,8 @@ Found X new positions (Y high, Z medium, W low match).
 
 skipped (disabled): <portal-name>, <portal-name>
 
+fallback (websearch): <portal-name>, <portal-name>
+
 health: <portal-name> - degraded (company null on all 12 results); parsing anchors in .agents/skills/<portal-name>/url-reference.md
 health: <portal-name> - broken (0 results for the SKILL.md test query and a broader retry); parsing anchors in .agents/skills/<portal-name>/url-reference.md
 

CHANGELOG.md

@@ -13,6 +13,20 @@ per-file diff commands.
 
 ## [Unreleased]
 
+### Added
+
+- **`seen_jobs.json` entries record which mechanism produced them** - a new additive `source`
+  field (`cli` for Step 1b portal-CLI output, `websearch` for the Step 1c fallback), a Step 1c
+  rule tagging fallback results at collection time, and a `fallback (websearch):` line in the
+  Step 5 run summary naming the portals that ran on the fallback. Motivated by the
+  ghost-LinkedIn-jobs report (#331): when a stored job later turns out not to exist at its URL,
+  triage hinges on whether the entry came from live CLI output or a search index that can be
+  weeks stale - evidence that previously lived only in the run's scrollback. An entry that is
+  missing `source` predates the field and is never backfilled; a presented job with no
+  `seen_jobs.json` entry at all points at fabrication, which the scraper's Rule 1 forbids.
+  Pinned by `tests/test_scrape_provenance.py`. `job-scraper/SKILL.md` sits outside the
+  `framework_version`-marked set, so no version bump applies.
+
 ### Changed
 
 - **Job matching reframed around function, not title** (`framework_version` 1.2.2 -> 1.2.3 in

tests/test_scrape_provenance.py

@@ -0,0 +1,82 @@
+"""Guards for /scrape's result-provenance recording.
+
+The scraper is a markdown spec (the spec IS the implementation), so these
+tests pin the invariants that would break silently: seen_jobs.json entries
+record whether they came from a portal CLI or the WebSearch fallback
+(`source`), and the Step 5 summary names the portals that ran on the
+fallback. Together these keep a ghost-job report diagnosable days after
+the run's scrollback is gone (#331): a stale-index entry, a live-CLI
+entry, and a job with no entry at all each point at a different mechanism.
+"""
+import unittest
+from pathlib import Path
+
+REPO = Path(__file__).resolve().parent.parent
+SKILL = REPO / ".claude" / "skills" / "job-scraper" / "SKILL.md"
+
+
+def _steps(text: str) -> dict[str, str]:
+    """Split the skill spec into {heading: body} by '###' step headers.
+
+    Splitting this way lets a fork's extra steps sit between the ones
+    under test without shifting which text a given assertion sees.
+    """
+    result = {}
+    for part in text.split("\n### ")[1:]:
+        heading, _, body = part.partition("\n")
+        result[heading.strip()] = body
+    return result
+
+
+class ScrapeProvenanceSpec(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls):
+        cls.text = SKILL.read_text(encoding="utf-8")
+        cls.steps = _steps(cls.text)
+
+    def test_schema_block_carries_source_field(self):
+        step4 = self.steps.get("Step 4: Deduplicate & Store", "")
+        self.assertIn(
+            '"source": "cli/websearch"',
+            step4,
+            "the seen_jobs.json schema block lost the source (provenance) field",
+        )
+
+    def test_source_field_is_additive_and_never_backfilled(self):
+        step4 = self.steps.get("Step 4: Deduplicate & Store", "")
+        self.assertIn(
+            "`cli` for Step 1b portal-CLI output, `websearch` for the Step 1c fallback",
+            step4,
+            "Step 4 must define which mechanism each source value names",
+        )
+        self.assertIn(
+            "the mechanism was not recorded",
+            step4,
+            "Step 4 must forbid back-filling source on entries that predate the field",
+        )
+
+    def test_fallback_results_are_tagged_at_the_source(self):
+        step1 = self.steps.get("Step 1: Search", "")
+        fallback = step1.partition("#### 1c. WebSearch fallback")[2]
+        self.assertIn(
+            "Step 4 persists this as the entry's `source`",
+            fallback,
+            "Step 1c must tag fallback results so Step 4 has a provenance value to store",
+        )
+
+    def test_step5_summary_names_fallback_portals(self):
+        step5 = self.steps.get("Step 5: Present Results", "")
+        self.assertIn(
+            "fallback (websearch):",
+            step5,
+            "Step 5 must surface which portals ran on the WebSearch fallback this run",
+        )
+        self.assertIn(
+            "omit the line when every portal ran its CLI",
+            " ".join(step5.split()),
+            "the fallback line must be omitted when every portal ran its CLI, not printed empty",
+        )
+
+
+if __name__ == "__main__":
+    unittest.main()