py/ai-job-search
1
0 Fork 0
mirror of https://github.com/MadsLorentzen/ai-job-search.git synced 2026-11-13 18:37:39 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(scrape): send one plain term per CLI call and make portals state their query syntax (#536) (#653)

Browse Source 
* fix(scrape): send one plain term per CLI call and make portals state their query syntax (#536)

search-queries.md is written in Google syntax, and Step 1b said to
translate its terms into portal flags without saying that OR and quotes
do not survive the translation.

- job-scraper Step 1b: one plain term per CLI call, never OR or a
  combined string; results merge across calls (Step 2 dedups on URL).
  Quoting follows the portal's own SKILL.md.
- Portal SKILL.md files carry a one-line "Query syntax:" note;
  /add-portal asks the generator to write it for new portals.
- search-queries.md: site: blocks are labelled WebSearch-only and each
  priority gains a plain-terms block for the CLIs.
- tests/test_scrape_contract.py: every installed portal SKILL.md must
  carry the Query syntax line; the portal list is derived from the
  skills tree.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(scrape): fall back to site: terms for placeholder blocks; update freehire query syntax

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore: drop the CHANGELOG hunk, frozen by #800 (release notes come from the PR title)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(scrape): dedup the pool by URL before Step 2 and give each portal's literal quote form

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(add-portal): compare the portal's reported total, not capped rows, when probing query syntax

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Thomas Christian VangandClaude Sonnet 5.5 authored and GitHub committed 2026-11-09 20:51:36 +01:00
1 parent 2b3f613fda
commit 7432228340
11 files changed
+88 -6
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.agents/skills/jobindex-search/url-reference.md
+3 -3
View File
@@ -59,7 +59,7 @@ Category uses a two-part path: `{group}/{slug}`:
| Parameter | Description | Examples |
|-----------|-------------|----------|
| `q` | Search query | `q=data+engineer` (broad), `q=%27data+engineer%27` (exact match) |
| `q` | Search query | `q=data+engineer` (broad), `q=%22data+engineer%22` (exact match) |
| `jobage` | Max age in days | `jobage=1` (today), `jobage=3`, `jobage=7`, `jobage=30` |
| `page` | Page number (1-indexed) | `page=1`, `page=2` |
@@ -69,8 +69,8 @@ Category uses a two-part path: `{group}/{slug}`:
# Basic keyword search
playwright-cli goto "https://www.jobindex.dk/jobsoegning?q=python+developer"
# Exact match search (single quotes around query)
playwright-cli goto "https://www.jobindex.dk/jobsoegning?q=%27python+developer%27"
# Exact match search (double quotes around query)
playwright-cli goto "https://www.jobindex.dk/jobsoegning?q=%22python+developer%22"
# Search in Storkøbenhavn area
playwright-cli goto "https://www.jobindex.dk/jobsoegning/storkoebenhavn?q=python+developer"