docs(jobindex-search): align search examples with the actual CLI contract (#579)

* docs(jobindex-search): align search examples with the actual CLI contract

The posted-today skill example omits the query that the shipped CLI requires, so copying it exits MISSING_REQUIRED. Supply a keyword and make the required flag explicit. Replace stale JSON-endpoint/card-parser search documentation with the implemented HTML Stash route and emitted field behavior.

Run the example read directly from SKILL.md through a real CLI subprocess with loopback HTTP fixtures. Keep a separate query/page routing control; no live portal requests or production-code changes.

* test(jobindex-search): cover Windows documentation newlines
This commit is contained in:
Rudy Mizrahi Celekli authored and GitHub committed 2026-10-18 07:31:12 +02:00
1 parent 894e3917ba
commit 3b16f80595
4 files changed
+115 -38

No files matched your search

+5 -4
View File
@@ -41,11 +41,11 @@ Invoke this skill when the user wants to:
### Search job listings ### Search job listings
```bash ```bash
bun run .agents/skills/jobindex-search/cli/src/cli.ts search [flags] bun run .agents/skills/jobindex-search/cli/src/cli.ts search --query "<text>" [flags]
``` ```
Key flags: Key flags:
- `--query <text>` / `-q <text>` — keyword search (job title, skill, company, city). **Required** for meaningful results. - `--query <text>` / `-q <text>` — keyword search (job title, skill, company, city). **Required.** Searches without a query exit with `MISSING_REQUIRED`.
- `--jobage <days>` — filter by posting age: `1` (today), `7`, `14`, `30`, or `9999` (all, default) - `--jobage <days>` — filter by posting age: `1` (today), `7`, `14`, `30`, or `9999` (all, default)
- `--sort <order>` — `score` (relevance, default) or `date` (newest first) - `--sort <order>` — `score` (relevance, default) or `date` (newest first)
- `--page <n>` — page number (1-indexed, 20 results per page, fixed) - `--page <n>` — page number (1-indexed, 20 results per page, fixed)
@@ -121,10 +121,11 @@ bun run .agents/skills/jobindex-search/cli/src/cli.ts search \
--format json --format json
``` ```
### Jobs posted today across all sectors ### Jobs posted today for a keyword
```bash ```bash
bun run .agents/skills/jobindex-search/cli/src/cli.ts search \ bun run .agents/skills/jobindex-search/cli/src/cli.ts search \
--query python \
--jobage 1 \ --jobage 1 \
--sort date \ --sort date \
--limit 20 \ --limit 20 \
@@ -167,5 +168,5 @@ All errors are written to **stderr** as `{ "error": "...", "code": "..." }` and
- Page size is fixed at 20 results per page (Jobindex API limitation). - Page size is fixed at 20 results per page (Jobindex API limitation).
- Area/region filtering via API params does not work — include city names in `--query` instead. - Area/region filtering via API params does not work — include city names in `--query` instead.
- `--jobage 9999` is the default and includes all postings regardless of age. - `--jobage 9999` is the default and includes all postings regardless of age.
- Total count in `meta.total` uses Danish dot-thousands notation internally (e.g. `18.903`) — the CLI normalizes this to a plain integer. - Total count in `meta.total` comes from the numeric `searchResponse.hitcount` embedded in the search page.
- Job IDs are string-prefixed (e.g. `h1647303`) — pass them as-is to `detail`. - Job IDs are string-prefixed (e.g. `h1647303`) — pass them as-is to `detail`.
+36 -34
View File
@@ -4,7 +4,7 @@ CLI for searching jobs on [Jobindex.dk](https://www.jobindex.dk).
**Base URL**: `https://www.jobindex.dk/` **Base URL**: `https://www.jobindex.dk/`
**Authentication**: None required. **Authentication**: None required.
**Format**: The API returns JSON with embedded HTML blobs. The CLI parses the HTML internally and emits clean JSON. **Format**: The search HTML embeds a JSON `Stash` payload. The CLI extracts its structured search results and emits JSON, table, or plain output.
--- ---
@@ -31,19 +31,19 @@ All errors are written to **stderr** as `{ "error": "...", "code": "..." }` and
## `search` — Search for job listings ## `search` — Search for job listings
**Endpoint**: `GET https://www.jobindex.dk/jobsoegning.json` **Endpoint**: `GET https://www.jobindex.dk/jobsoegning`
```bash ```bash
bun run src/cli.ts search [flags] bun run src/cli.ts search --query "<text>" [flags]
``` ```
The API always returns 20 results per page (fixed — no `--per-page` flag). The CLI parses the `result_list_box_html` HTML blob from the response to extract structured job records. The portal returns 20 results per page (fixed — no `--per-page` flag). The CLI reads `searchResponse.results` from the page's `var Stash = {...}` script. `--query` is required; a query-free search exits with `MISSING_REQUIRED` before fetching.
### Flags ### Flags
| Flag | Type | Default | Description | | Flag | Type | Default | Description |
|------|------|---------|-------------| |------|------|---------|-------------|
| `--query` / `-q` | string | — | Keyword search query (e.g. `python`, `grafisk designer`) | | `--query` / `-q` | string | — | **Required.** Keyword search query (e.g. `python`, `grafisk designer`) |
| `--page` | number | `1` | Page number (1-indexed) | | `--page` | number | `1` | Page number (1-indexed) |
| `--jobage` | number | `9999` | Max age of posting in days: `1`, `7`, `14`, `30`, or `9999` (all) | | `--jobage` | number | `9999` | Max age of posting in days: `1`, `7`, `14`, `30`, or `9999` (all) |
| `--sort` | string | `score` | Sort order: `score` (relevance) or `date` (newest first) | | `--sort` | string | `score` | Sort order: `score` (relevance) or `date` (newest first) |
@@ -97,8 +97,9 @@ bun run src/cli.ts search --query "data engineer" --page 2 --format table
"companyUrl": "https://www.gavefabrikken.dk/", "companyUrl": "https://www.gavefabrikken.dk/",
"location": "Valby", "location": "Valby",
"date": "2026-03-12", "date": "2026-03-12",
"url": "https://www.jobindex.dk/jobannonce/h1647303/data-engineer-til-opbygning-af-gavefabrikkens-dataplatform", "deadline": "2026-04-01",
"description": "Vi søger en dygtig Data Engineer til at opbygge og vedligeholde vores dataplatform..." "url": "https://www.jobindex.dk/jobannonce/h1647303",
"description": null
} }
] ]
} }
@@ -109,10 +110,11 @@ bun run src/cli.ts search --query "data engineer" --page 2 --format table
- `company` — company name; may be `null` for some aggregated listings. - `company` — company name; may be `null` for some aggregated listings.
- `companyUrl` — company homepage URL; may be `null` if not present. - `companyUrl` — company homepage URL; may be `null` if not present.
- `location` — city or area; may be `null` if not listed. - `location` — city or area; may be `null` if not listed.
- `date` — ISO date string (`YYYY-MM-DD`) from the `datetime` attribute on the `<time>` element; may be `null`. - `date` — the posting's `firstdate` value; may be `null`.
- `description` — short excerpt from the listing; may be `null` or empty. - `deadline` — the first ten characters of `apply_deadline`, falling back to `lastdate`; `null` when `apply_deadline_asap` is set or no date is present.
- `url` — full Jobindex.dk URL for the listing. - `description` — currently always `null` in search results. Use `detail` for the full ad.
- `total` in `meta` — parsed from `hitcount_html` (Danish thousands separator `.` is stripped before parsing, e.g. `18.903` → `18903`). - `url` — canonical `https://www.jobindex.dk/jobannonce/{id}` URL; a result without an ID falls back to its `share_url` or `url`.
- `total` in `meta` — numeric `searchResponse.hitcount`, falling back to the number of returned results if the count is absent.
> **Note on area filtering**: The Jobindex API does not reliably support area/region filtering via query parameters. `area` and `geoareaid` params are silently ignored. To filter by location, use `--query` with a city name (e.g. `--query "python aarhus"`) or apply `--limit` and filter the JSON output externally. > **Note on area filtering**: The Jobindex API does not reliably support area/region filtering via query parameters. `area` and `geoareaid` params are silently ignored. To filter by location, use `--query` with a city name (e.g. `--query "python aarhus"`) or apply `--limit` and filter the JSON output externally.
@@ -213,35 +215,35 @@ All errors are written to **stderr** in JSON format and exit with code `1`:
Job detail pages on jobindex.dk: Job detail pages on jobindex.dk:
- `https://www.jobindex.dk/jobannonce/{id}/{slug}` - `https://www.jobindex.dk/jobannonce/{id}/{slug}`
The slug is part of the `url` returned by `search`. When calling `detail` with just an ID, the CLI fetches `https://www.jobindex.dk/jobannonce/{id}` which redirects to the full URL. Search emits the canonical URL without a slug. When calling `detail` with an ID or full posting URL, the CLI rebuilds `https://www.jobindex.dk/jobannonce/{id}` and follows redirects.
--- ---
## Parsing notes ## Parsing notes
### Total count from `hitcount_html` ### Search results from `Stash`
The API returns pagination info as an HTML string like: The former `/jobsoegning.json` endpoint no longer provides the search payload.
The HTML search page embeds structured data in a `var Stash = {...}` script.
The CLI locates a nested `searchResponse` object with a `results` array, for
example under `jobsearch.result_app.storeData.searchResponse`:
```html ```json
<div class="jix_pagination_total"><strong>1</strong> til <strong>20</strong> af <strong>18.903</strong> resultater.</div> {
"hitcount": 237,
"results": [
{
"tid": "h1647303",
"headline": "Data Engineer",
"company": { "name": "Gavefabrikken", "homeurl": "https://www.gavefabrikken.dk/" },
"area": "Valby",
"firstdate": "2026-03-12",
"apply_deadline": "2026-04-01T00:00:00"
}
]
}
``` ```
Parse total with: `/af <strong>([\d.]+)<\/strong>/` and strip `.` before converting to integer. The active search path maps these fields directly; it does not parse job-card
HTML or `hitcount_html`. `extractStash` and `parseSearchPage` in `src/helpers.ts`
### Job card selectors implement this path. The legacy `parseJobCards` helper is not used by `search`.
Each job card is wrapped in `[data-beacon-tid]`. Inside, select:
| Field | Selector |
|-------|----------|
| `id` | `[data-beacon-tid]` attribute value |
| `title` | `h4 > a` text content |
| `url` | `h4 > a[href]` |
| `company` | `.jix-toolbar-top__company a` text |
| `companyUrl` | `.jix-toolbar-top__company a[href]` |
| `location` | `span.jix_robotjob--area` text |
| `date` | `time[datetime]` attribute value |
| `description` | `p` text content (first `<p>` in card) |
Two card types exist: `div.PaidJob` (sponsored) and `div.jix_robotjob` (aggregated). Both use the same selector pattern.
@@ -0,0 +1,72 @@
import { expect, test } from "bun:test";
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
async function search(args: string[]): Promise<URL[]> {
const requests: URL[] = [];
const server = Bun.serve({
hostname: "127.0.0.1",
port: 0,
fetch(request) {
requests.push(new URL(request.url));
// The test checks the documented invocation and request route, not live
// portal markup. This is the existing Stash shape with no result items.
return new Response('<script>var Stash = {"jobsearch":{"result_app":{"storeData":{"searchResponse":{"hitcount":0,"results":[]}}}}};</script>');
},
});
const dir = mkdtempSync(join(tmpdir(), "jobindex-doc-search-"));
const preload = join(dir, "fetch.ts");
writeFileSync(preload, `
const originalFetch = globalThis.fetch;
globalThis.fetch = (input, init) => {
const url = new URL(input instanceof Request ? input.url : String(input));
if (url.protocol === "data:") return originalFetch(input, init);
if (url.origin !== "https://www.jobindex.dk" || url.pathname !== "/jobsoegning") {
throw new Error("Unexpected documented search endpoint");
}
return originalFetch(${JSON.stringify(server.url.origin)} + url.pathname + url.search, init);
};
`);
try {
const proc = Bun.spawn([
process.execPath, "run", "--preload", preload,
join(import.meta.dir, "../src/cli.ts"), "search", ...args,
], { stdout: "pipe", stderr: "pipe" });
const [stdout, stderr, exitCode] = await Promise.all([
new Response(proc.stdout).text(), new Response(proc.stderr).text(), proc.exited,
]);
expect(stderr).toBe("");
expect(exitCode).toBe(0);
expect(stdout).not.toBe("");
expect(requests).toHaveLength(1);
return requests;
} finally {
server.stop(true);
rmSync(dir, { recursive: true, force: true });
}
}
test.each(["LF", "CRLF"])("the skill's posted-today example runs through the real search CLI with %s newlines", async (lineEnding) => {
const source = await Bun.file(join(import.meta.dir, "../../SKILL.md")).text();
const checkout = source.replace(/\r\n/g, "\n").replace(/\n/g, lineEnding === "CRLF" ? "\r\n" : "\n");
expect(checkout.includes("\r\n")).toBe(lineEnding === "CRLF");
const skill = checkout.replace(/\r\n/g, "\n");
const example = skill.match(/### Jobs posted today[^\n]*\n\n```bash\n([\s\S]*?)```/)?.[1];
expect(example).toBeDefined();
// These concrete documentation examples contain only ordinary quoted or
// unquoted shell words. No shell is invoked and no profile data is loaded.
const words = Array.from(example!.replace(/\\\n/g, " ").matchAll(/"([^"]*)"|'([^']*)'|(\S+)/g),
(match) => match[1] ?? match[2] ?? match[3]);
const args = words.slice(words.indexOf("search") + 1);
const [url] = await search(args);
expect(url.searchParams.get("q")).toBeTruthy();
expect(url.searchParams.get("jobage")).toBe("1");
});
test("a required query and page reach the current HTML endpoint", async () => {
const [url] = await search(["--query", "python", "--page", "2"]);
expect(url.pathname).toBe("/jobsoegning");
expect(url.searchParams.get("q")).toBe("python");
expect(url.searchParams.get("page")).toBe("2");
});
+2
View File
@@ -68,6 +68,8 @@ per-file diff commands.
### Fixed ### Fixed
- **`jobindex-search` examples follow the search CLI contract** - the posted-today example includes a required query, unsupported flags are removed, and regression checks accept LF and CRLF checkouts (#579).
- **`convert_salary_excel.py` converts the IDA workbook again** (#615, `tools/convert_salary_excel.py`, - **`convert_salary_excel.py` converts the IDA workbook again** (#615, `tools/convert_salary_excel.py`,
`tests/test_convert_salary_excel.py`) - the workbook heads its company column "Firmanavn og by", `tests/test_convert_salary_excel.py`) - the workbook heads its company column "Firmanavn og by",
and since #75 made header matching whole-token only, "firma" no longer matched inside "Firmanavn", and since #75 made header matching whole-token only, "firma" no longer matched inside "Firmanavn",