0 Fork 0

mirror of https://github.com/MadsLorentzen/ai-job-search.git synced 2026-11-11 17:37:59 +00:00

Files

T

Thomas Christian VangandClaude Sonnet 5.5 7432228340 fix(scrape): send one plain term per CLI call and make portals state their query syntax (#536 ) (#653 )

* fix(scrape): send one plain term per CLI call and make portals state their query syntax (#536)

search-queries.md is written in Google syntax, and Step 1b said to
translate its terms into portal flags without saying that OR and quotes
do not survive the translation.

- job-scraper Step 1b: one plain term per CLI call, never OR or a
  combined string; results merge across calls (Step 2 dedups on URL).
  Quoting follows the portal's own SKILL.md.
- Portal SKILL.md files carry a one-line "Query syntax:" note;
  /add-portal asks the generator to write it for new portals.
- search-queries.md: site: blocks are labelled WebSearch-only and each
  priority gains a plain-terms block for the CLIs.
- tests/test_scrape_contract.py: every installed portal SKILL.md must
  carry the Query syntax line; the portal list is derived from the
  skills tree.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(scrape): fall back to site: terms for placeholder blocks; update freehire query syntax

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore: drop the CHANGELOG hunk, frozen by #800 (release notes come from the PR title)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(scrape): dedup the pool by URL before Step 2 and give each portal's literal quote form

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(add-portal): compare the portal's reported total, not capped rows, when probing query syntax

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>

2026-11-09 20:51:36 +01:00

3.3 KiB

Raw Blame History

Jobindex URL Reference

Base URL

https://www.jobindex.dk/jobsoegning

Full URL Pattern

https://www.jobindex.dk/jobsoegning/{category-group}/{category-slug}/{area-slug}?q={query}&jobage={days}&page={num}

All path segments and query params are optional. Category MUST come before area in the path.

Path Segments

Areas (geography)

Area slug goes at the end of the path:

Area	Slug
Storkøbenhavn	`storkoebenhavn`
Københavnsområdet	`storkoebenhavn`
Nordsjælland	`nordsjaelland`
Sjælland	`sjaelland`
Fyn	`fyn`
Nordjylland	`nordjylland`
Midtjylland	`midtjylland`
Sydjylland	`sydjylland`
Bornholm	`bornholm`

Note: If the exact slug is unknown, use the Geografi filter UI instead:

Click the "Geografi" button
Click "Tilføj område"
Type city/region name in the textbox
Select from autocomplete (treeitem)
Click "Vis X job" button
Note the URL path that results

Category	Path
IT-drift og support	`it/itdrift`

Query Parameters

Parameter	Description	Examples
`q`	Search query	`q=data+engineer` (broad), `q=%22data+engineer%22` (exact match)
`jobage`	Max age in days	`jobage=1` (today), `jobage=3`, `jobage=7`, `jobage=30`
`page`	Page number (1-indexed)	`page=1`, `page=2`

Examples

# Basic keyword search
playwright-cli goto "https://www.jobindex.dk/jobsoegning?q=python+developer"

# Exact match search (double quotes around query)
playwright-cli goto "https://www.jobindex.dk/jobsoegning?q=%22python+developer%22"

# Search in Storkøbenhavn area
playwright-cli goto "https://www.jobindex.dk/jobsoegning/storkoebenhavn?q=python+developer"

# Search with category + area
playwright-cli goto "https://www.jobindex.dk/jobsoegning/it/itdrift/storkoebenhavn?q=data+engineer"

# Last 7 days only
playwright-cli goto "https://www.jobindex.dk/jobsoegning?q=data+engineer&jobage=7"

# Page 2 of results
playwright-cli goto "https://www.jobindex.dk/jobsoegning?q=data+engineer&page=2"

# Everything combined
playwright-cli goto "https://www.jobindex.dk/jobsoegning/storkoebenhavn?q=data+engineer&jobage=7&page=1"

Filters Available via UI Only

These filters require clicking through the filter panel (not URL-constructable):

Ansættelsestype (Employment type): Fastansættelse, Tidsbegrænset, Studiejob, Graduate/trainee, Freelance, etc.

Arbejdstid (Working hours): Fuldtid, Deltid

Hjemmearbejde (Remote work): Muligt, Tilbydes ikke, 100% hjemmearbejde

To use these: click "Filtre" button → check desired options → click "Vis X job".

Job Card Extraction

CSS selectors for job cards: div.PaidJob, div.jix_robotjob

Each card contains:

Title: h4 a (text + href)
Company: first a link text
Location: div after h4 (clean by removing "Se rejsetid")
Posted date: time element (format: DD-MM-YYYY)
Description: p elements
Job URL: h4 a href (may be external or jobindex-hosted)

~20 results per page.

3.3 KiB Raw Blame History