From a5de9384926fc19d9c623c7e609403cc13f4d36b Mon Sep 17 00:00:00 2001 From: Ayobami Adegoke Date: Mon, 20 Jul 2026 20:53:51 +0100 Subject: [PATCH] feat(scrape): portal health check to catch silent scraper rot (#193) Adds Step 4.75 to /scrape: detects the failure mode where a portal changes markup and its parser exits 0 with zero or garbled results (invisible to the Step 1c fallback, which only fires on non-zero exit). Free pass over this run's results (degraded scan) plus seen_jobs.json yield history; bounded escalation on suspicion only (the portal's own SKILL.md test query, one broader retry, rate-limit never treated as evidence); health: lines in the Step 5 summary with a confirmation-gated enabled:false quarantine offer; healthy portals stay silent. Adds a /scrape health [portal] probe-only mode. Persists a portal field in seen_jobs.json additively (per the /rank precedent), with read-time URL-domain attribution for pre-field entries so no migration is needed. Folded into /scrape rather than a standalone /doctor command - the detection lives where the evidence (run results + yield history) already is, and a routine command catches rot a user would otherwise notice weeks late. By @ayobamiseun. --- .claude/skills/job-scraper/SKILL.md | 32 +++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/.claude/skills/job-scraper/SKILL.md b/.claude/skills/job-scraper/SKILL.md index 8df3197..7f210eb 100644 --- a/.claude/skills/job-scraper/SKILL.md +++ b/.claude/skills/job-scraper/SKILL.md @@ -30,6 +30,7 @@ The user triggers this skill by saying things like: Optional arguments: - A focus area, e.g. "/scrape data science" or "/scrape geophysics" - "broad" to run all search categories, e.g. "/scrape broad" +- "health" to run the portal health check only (Step 4.75), without searching, deduplicating, or presenting jobs - e.g. "/scrape health", or "/scrape health jobnet" to probe one portal even if disabled --- @@ -118,12 +119,15 @@ For each new job, do a rapid fit check (NOT the full evaluation from `04-job-eva "url": "...", "first_seen": "YYYY-MM-DD", "fit": "high/medium/low", - "status": "new/skipped/evaluated/ranked/expired" + "status": "new/skipped/evaluated/ranked/expired", + "portal": "" } } } ``` +The `portal` field records which CLI skill produced the job (results are already tagged per portal in Step 1b - persist that tag here). Entries written before this field existed lack it; the health check (Step 4.75) attributes those by matching the URL's domain against each portal's base URL, so do not backfill. + `/rank` extends this schema additively: ranked entries also carry `rank_score` (0–100 overall score), `rank_verdict` (fit band, e.g. "strong fit"), and `rank_date` (ISO date of ranking). The `status` field is set to `"ranked"`. Do not drop any of these fields when re-writing entries. 2. Only present jobs NOT already in the seen list or tracker. @@ -152,12 +156,32 @@ Both links are for the user to open and browse themselves - never fetch or scrap LinkedIn people-search result pages programmatically. Never fabricate contacts or claim a specific person was found; these are search links, not results. +### Step 4.75: Portal Health Check + +Scraper-based portal CLIs rot silently: when a portal changes its markup, the parser usually exits 0 with zero results or with null/garbled fields, and the Step 1c fallback never fires because it only triggers on hard failure. This step catches that from evidence the run already holds. + +**Free pass (no extra requests).** For each enabled portal that ran in Step 1b: + +- **Degraded scan:** inspect the results it returned this run. Flags: `company` null or empty on every result, empty titles, undecoded entities (`&`) or HTML fragments in titles, URLs that do not point at the portal. Any of these means the parser is half-working and `/scrape` is silently collecting junk. +- **Yield history:** if the portal returned zero results across all of this run's queries, check whether `seen_jobs.json` holds prior entries from it (via the `portal` field, or by matching URL domains for entries predating the field). A portal that produced jobs on earlier runs and produces nothing now is suspect - the same queries worked before. + +**Escalation (bounded, on suspicion only).** A suspect portal gets **one** sentinel probe: run its documented `search` with the example query from its own SKILL.md (that query provably worked when the skill was registered), the portal's limit flag capped at 3, `--format json`. If that returns nothing, retry **once** with a single common word. Only then is the verdict **broken**. A 429 or block page is **never** evidence of breakage - record the portal as **inconclusive (rate-limited)**, back off, and do not retry. + +**Verdicts.** Healthy portals get silence - no table, no line. Anything else surfaces in the Step 5 summary as a health line. + +**Probe-only mode (`/scrape health`).** Skip Steps 1-4 and this step's free pass (there is no fresh run to scan); instead probe every installed portal directly - enabled ones by default, a disabled one only when named explicitly (e.g. `/scrape health jobnet`). Each portal gets the sentinel probe above, the degraded criteria applied to whatever it returns, and - since the user explicitly asked for diagnosis - one `detail` fetch on the first result of each healthy portal (description must be readable decoded text; a failure downgrades to degraded). Report all statuses in this mode, including healthy. Volume stays bounded: one search, at most one retry, at most one detail per portal. + ### Step 5: Present Results Present new jobs in a table sorted by fit (high first). When Step 1b skipped portals (`enabled: false`), report them with the `skipped (disabled):` line below so opting one out stays visible rather than silent; omit the line when nothing -was skipped. +was skipped. When Step 4.75 found a portal degraded, broken, or inconclusive, +add one `health:` line per suspect portal (healthy portals get no line); after +the report, offer to set that portal's `enabled: false` so `/scrape` stops +running it (and covers it via the Step 1c fallback) until it is fixed - only +edit the toggle with the user's confirmation, and never edit anything else in +the skill. ``` ## New Job Matches - YYYY-MM-DD @@ -166,6 +190,9 @@ Found X new positions (Y high, Z medium, W low match). skipped (disabled): , +health: - degraded (company null on all 12 results); parsing anchors in .agents/skills//url-reference.md +health: - broken (0 results for the SKILL.md test query and a broader retry); parsing anchors in .agents/skills//url-reference.md + | # | Fit | Title | Company | Location | Deadline | URL | |---|-----|-------|---------|----------|----------|-----| | 1 | High | ... | ... | ... | ... | [Link](...) | @@ -205,3 +232,4 @@ If the user decides to apply to any job, add a row to `job_search_tracker.csv`. 5. **Be efficient with detail fetches.** Don't run `detail` or WebFetch on every search hit — pre-filter by title/snippet, then fetch only promising matches. 6. **Parallel searches.** Run portal CLI searches in parallel; use WebSearch only for gaps the CLIs don't cover. 7. **No automated people lookups.** Referral contacts (Step 4.5) are LinkedIn search links only - never fetch or scrape LinkedIn people-search result pages programmatically. +8. **Health checks are bounded and honest.** Step 4.75 spends at most one probe, one retry, and (in `health` mode) one detail fetch per portal - a diagnosis, not a crawl. A rate-limit is never evidence of breakage. Health verdicts come only from observed CLI output; a portal that could not be tested is reported as inconclusive, never guessed. The `enabled` toggle is the only thing the health check may edit, and only with confirmation.