mirror of
https://github.com/MadsLorentzen/ai-job-search.git
synced 2026-09-17 08:36:25 +00:00
fix(scrape): add a client-side recency fallback for flagless portals
Step 1b.3's "scope to 14 days using the portal's recency flag" was unsatisfiable on jobdanmark, which has no date filter or sort - the agent either silently skipped the scoping or invented a flag, and the CLIs now reject invented flags loudly. Every portal emits date, so the instruction now filters client-side after the call, and stops presenting --order (a sort) as interchangeable with a filter. Review finding F32 (2026-08-19). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
2c3d2d8558
commit
9a69309749
@@ -66,7 +66,7 @@ For each **enabled** portal skill:
|
||||
|
||||
1. Read its `SKILL.md` to find the correct `bun run …` invocation and supported flags.
|
||||
2. Translate the query terms from `search-queries.md` into that portal's flag format (e.g. `--key`, `--search-string`, `--query`, filter codes — whatever the portal's SKILL.md specifies).
|
||||
3. Scope to the last 14 days using the portal's supported recency flag (`--jobage`, `--since <YYYY-MM-DD>`, `--order PublicationDate`, etc. — as documented per portal).
|
||||
3. Scope to the last 14 days using the portal's supported recency **filter** flag (`--jobage`, `--since <YYYY-MM-DD>`, etc. — as documented per portal). A portal with **no recency flag** (jobdanmark offers none) still gets scoped: every portal's search output carries a `date` field, so filter client-side — drop results whose `date` is older than 14 days after the call returns, and never invent a flag the portal's SKILL.md does not document (the CLIs reject unknown flags). `--order PublicationDate` is a sort, and a sort is not a filter — pairing it with a `--limit` is a defensible approximation on a portal that offers nothing better (jobnet), but apply the client-side date filter on top all the same.
|
||||
4. Cap results to ~20 per call using the portal's limit flag.
|
||||
5. Use `--format json` for machine-readable output.
|
||||
|
||||
|
||||
@@ -120,6 +120,13 @@ per-file diff commands.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **`/scrape` gains a recency fallback for portals with no recency flag** - Step 1b.3
|
||||
told every portal to scope to 14 days "using the portal's supported recency flag", but
|
||||
jobdanmark has none, leaving the instruction unsatisfiable there: the agent either
|
||||
silently skipped the scoping or invented a flag (which the CLIs now reject). Every
|
||||
portal emits a `date` field, so the instruction now says to filter client-side after
|
||||
the call, and stops presenting `--order` (a sort) as interchangeable with a filter.
|
||||
Pinned in `tests/test_scrape_provenance.py`.
|
||||
- **`/html-report`'s funnel counts stages from history; the rejection rate stops
|
||||
counting non-rejections** - the funnel was computed from current status, which is a
|
||||
state, not a history: an application that interviewed and was then rejected never
|
||||
|
||||
@@ -78,5 +78,33 @@ class ScrapeProvenanceSpec(unittest.TestCase):
|
||||
)
|
||||
|
||||
|
||||
class TestRecencyFallback(unittest.TestCase):
|
||||
"""Step 1b.3 says to scope to the last 14 days via the portal's recency
|
||||
flag - but not every portal has one (jobdanmark offers no date filter or
|
||||
sort at all), which left the instruction unsatisfiable there: the agent
|
||||
either silently skipped the scoping or invented a flag, and inventing a
|
||||
flag is exactly what the UNKNOWN_FLAG rejection now errors on (review
|
||||
finding F32, 2026-08-19). Every portal emits a `date` field, so
|
||||
client-side filtering is always available as the fallback."""
|
||||
|
||||
def test_step1b_names_a_client_side_fallback_for_flagless_portals(self):
|
||||
text = SKILL.read_text(encoding="utf-8")
|
||||
self.assertIn(
|
||||
"no recency flag",
|
||||
text,
|
||||
"Step 1b.3 must say what to do when a portal offers no recency flag",
|
||||
)
|
||||
self.assertIn(
|
||||
"filter client-side",
|
||||
text,
|
||||
"the fallback is filtering results by their `date` field after the call",
|
||||
)
|
||||
self.assertIn(
|
||||
"a sort is not a filter",
|
||||
text,
|
||||
"the instruction must stop presenting --order (a sort) as interchangeable with a filter",
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
Reference in New Issue
Block a user