mirror of
https://github.com/prdlk/jobyssey.git
synced 2026-11-10 08:57:41 +00:00
feat: add skills
This commit is contained in:
1 parent
6984212b17
commit
c50aa92862
150 files changed
+12702
No files matched your search
Executable
+247
@@ -0,0 +1,247 @@
|
||||
# jobindex-cli
|
||||
|
||||
CLI for searching jobs on [Jobindex.dk](https://www.jobindex.dk).
|
||||
|
||||
**Base URL**: `https://www.jobindex.dk/`
|
||||
**Authentication**: None required.
|
||||
**Format**: The API returns JSON with embedded HTML blobs. The CLI parses the HTML internally and emits clean JSON.
|
||||
|
||||
---
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
cd skills/jobindex-search/cli
|
||||
bun install
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Commands
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `search` | Search for job listings |
|
||||
| `detail` | Fetch full detail for a single job listing |
|
||||
|
||||
All commands accept `--format json|table|plain` (default: `json`).
|
||||
All errors are written to **stderr** as `{ "error": "...", "code": "..." }` and the process exits with code `1`.
|
||||
|
||||
---
|
||||
|
||||
## `search` — Search for job listings
|
||||
|
||||
**Endpoint**: `GET https://www.jobindex.dk/jobsoegning.json`
|
||||
|
||||
```bash
|
||||
bun run src/cli.ts search [flags]
|
||||
```
|
||||
|
||||
The API always returns 20 results per page (fixed — no `--per-page` flag). The CLI parses the `result_list_box_html` HTML blob from the response to extract structured job records.
|
||||
|
||||
### Flags
|
||||
|
||||
| Flag | Type | Default | Description |
|
||||
|------|------|---------|-------------|
|
||||
| `--query` / `-q` | string | — | Keyword search query (e.g. `python`, `grafisk designer`) |
|
||||
| `--page` | number | `1` | Page number (1-indexed) |
|
||||
| `--jobage` | number | `9999` | Max age of posting in days: `1`, `7`, `14`, `30`, or `9999` (all) |
|
||||
| `--sort` | string | `score` | Sort order: `score` (relevance) or `date` (newest first) |
|
||||
| `--limit` | number | — | Cap total results returned by the CLI (client-side) |
|
||||
| `--format` | string | `json` | Output format: `json`, `table`, `plain` |
|
||||
|
||||
### Sort options
|
||||
|
||||
| Value | Description |
|
||||
|-------|-------------|
|
||||
| `score` | Relevance / best match (default) |
|
||||
| `date` | Newest postings first |
|
||||
|
||||
### jobage options
|
||||
|
||||
| Value | Description |
|
||||
|-------|-------------|
|
||||
| `1` | Posted today |
|
||||
| `7` | Last 7 days |
|
||||
| `14` | Last 14 days |
|
||||
| `30` | Last 30 days |
|
||||
| `9999` | All time (default) |
|
||||
|
||||
### Example
|
||||
|
||||
```bash
|
||||
# Search for Python jobs posted in the last 7 days, sorted by date
|
||||
bun run src/cli.ts search --query python --jobage 7 --sort date
|
||||
|
||||
# Search for "grafisk designer" jobs — show first 5 results
|
||||
bun run src/cli.ts search --query "grafisk designer" --limit 5
|
||||
|
||||
# Page 2 of results for data engineer
|
||||
bun run src/cli.ts search --query "data engineer" --page 2 --format table
|
||||
```
|
||||
|
||||
### Response shape
|
||||
|
||||
```json
|
||||
{
|
||||
"meta": {
|
||||
"total": 237,
|
||||
"page": 1,
|
||||
"perPage": 20
|
||||
},
|
||||
"results": [
|
||||
{
|
||||
"id": "h1647303",
|
||||
"title": "Data Engineer til opbygning af Gavefabrikkens dataplatform",
|
||||
"company": "Gavefabrikken",
|
||||
"companyUrl": "https://www.gavefabrikken.dk/",
|
||||
"location": "Valby",
|
||||
"date": "2026-03-12",
|
||||
"url": "https://www.jobindex.dk/jobannonce/h1647303/data-engineer-til-opbygning-af-gavefabrikkens-dataplatform",
|
||||
"description": "Vi søger en dygtig Data Engineer til at opbygge og vedligeholde vores dataplatform..."
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Field notes:**
|
||||
- `id` — string ID prefixed with `h` (e.g. `h1647303`). Use this with the `detail` command.
|
||||
- `company` — company name; may be `null` for some aggregated listings.
|
||||
- `companyUrl` — company homepage URL; may be `null` if not present.
|
||||
- `location` — city or area; may be `null` if not listed.
|
||||
- `date` — ISO date string (`YYYY-MM-DD`) from the `datetime` attribute on the `<time>` element; may be `null`.
|
||||
- `description` — short excerpt from the listing; may be `null` or empty.
|
||||
- `url` — full Jobindex.dk URL for the listing.
|
||||
- `total` in `meta` — parsed from `hitcount_html` (Danish thousands separator `.` is stripped before parsing, e.g. `18.903` → `18903`).
|
||||
|
||||
> **Note on area filtering**: The Jobindex API does not reliably support area/region filtering via query parameters. `area` and `geoareaid` params are silently ignored. To filter by location, use `--query` with a city name (e.g. `--query "python aarhus"`) or apply `--limit` and filter the JSON output externally.
|
||||
|
||||
---
|
||||
|
||||
## `detail` — Fetch full job listing detail
|
||||
|
||||
**URL**: `https://www.jobindex.dk/jobannonce/{id}/{slug}`
|
||||
|
||||
```bash
|
||||
bun run src/cli.ts detail <id> [--format json|plain]
|
||||
```
|
||||
|
||||
The `id` is the job ID from `search` results (e.g. `h1647303`). The slug is optional — the CLI fetches the canonical URL by first constructing `https://www.jobindex.dk/jobannonce/{id}` and following any redirect, or by using the full URL from the `url` field in `search` results.
|
||||
|
||||
You may also pass the full URL directly as the `id` argument.
|
||||
|
||||
### Flags
|
||||
|
||||
| Flag | Type | Default | Description |
|
||||
|------|------|---------|-------------|
|
||||
| `--format` | string | `json` | Output format: `json`, `plain` |
|
||||
|
||||
### Example
|
||||
|
||||
```bash
|
||||
# Using ID from search results
|
||||
bun run src/cli.ts detail h1647303
|
||||
|
||||
# Using full URL
|
||||
bun run src/cli.ts detail "https://www.jobindex.dk/jobannonce/h1647303/data-engineer-til-opbygning-af-gavefabrikkens-dataplatform"
|
||||
|
||||
# Plain text output
|
||||
bun run src/cli.ts detail h1647303 --format plain
|
||||
```
|
||||
|
||||
### Response shape
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "h1647303",
|
||||
"title": "Data Engineer til opbygning af Gavefabrikkens dataplatform",
|
||||
"company": "Gavefabrikken",
|
||||
"companyUrl": "https://www.gavefabrikken.dk/",
|
||||
"location": "Valby, København",
|
||||
"date": "2026-03-12",
|
||||
"deadline": "2026-04-01",
|
||||
"employmentType": "Fastansættelse",
|
||||
"hours": "Fuldtid",
|
||||
"applyUrl": "https://www.gavefabrikken.dk/jobs/apply/123",
|
||||
"url": "https://www.jobindex.dk/jobannonce/h1647303/data-engineer-til-opbygning-af-gavefabrikkens-dataplatform",
|
||||
"description": "Full job description text here..."
|
||||
}
|
||||
```
|
||||
|
||||
**Field notes:**
|
||||
|
||||
Jobindex serves detail pages in two shapes, and field availability differs:
|
||||
a **jobindex-native** page (recognisable by its `jd-*` facts blocks) carries
|
||||
company, location, an ISO deadline, employment type and hours; an **external
|
||||
ATS passthrough** (the employer's hosted ad, e.g. hr-manager/Talentech, served
|
||||
through jobindex) has no reliable company anchor, so `company` is `null` there
|
||||
rather than the ATS brand, and location/deadline come from the ad's own
|
||||
widgets when present.
|
||||
|
||||
- `id` / `url` — always the jobindex id and its `jobannonce` URL, never the
|
||||
page's `og:url`/canonical (on passthrough pages those point at the external
|
||||
ATS, not the posting).
|
||||
- `deadline` — `YYYY-MM-DD` or `null`; Danish long dates ("13. september
|
||||
2026") and `DD-MM-YYYY` widget dates are converted.
|
||||
- `employmentType` / `hours` — from the native facts blocks; `null` on
|
||||
passthrough pages.
|
||||
- `companyUrl` — currently always `null`; no page shape carries a usable
|
||||
company link.
|
||||
- `applyUrl` — the Jobindex redirect link (`/c?t=...`) when present; `null`
|
||||
otherwise.
|
||||
- `description` — plain text of the ad body (HTML stripped), falling back to
|
||||
the page's meta description when the body is empty.
|
||||
- All fields except `id`, `title`, and `url` may be `null`.
|
||||
|
||||
---
|
||||
|
||||
## Error handling
|
||||
|
||||
All errors are written to **stderr** in JSON format and exit with code `1`:
|
||||
|
||||
```json
|
||||
{ "error": "Job not found", "code": "NOT_FOUND" }
|
||||
{ "error": "API request failed: 500 Internal Server Error", "code": "API_ERROR" }
|
||||
{ "error": "Failed to parse job listing HTML", "code": "PARSE_ERROR" }
|
||||
{ "error": "--query is required", "code": "MISSING_REQUIRED" }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## URL construction
|
||||
|
||||
Job detail pages on jobindex.dk:
|
||||
- `https://www.jobindex.dk/jobannonce/{id}/{slug}`
|
||||
|
||||
The slug is part of the `url` returned by `search`. When calling `detail` with just an ID, the CLI fetches `https://www.jobindex.dk/jobannonce/{id}` which redirects to the full URL.
|
||||
|
||||
---
|
||||
|
||||
## Parsing notes
|
||||
|
||||
### Total count from `hitcount_html`
|
||||
|
||||
The API returns pagination info as an HTML string like:
|
||||
|
||||
```html
|
||||
<div class="jix_pagination_total"><strong>1</strong> til <strong>20</strong> af <strong>18.903</strong> resultater.</div>
|
||||
```
|
||||
|
||||
Parse total with: `/af <strong>([\d.]+)<\/strong>/` and strip `.` before converting to integer.
|
||||
|
||||
### Job card selectors
|
||||
|
||||
Each job card is wrapped in `[data-beacon-tid]`. Inside, select:
|
||||
|
||||
| Field | Selector |
|
||||
|-------|----------|
|
||||
| `id` | `[data-beacon-tid]` attribute value |
|
||||
| `title` | `h4 > a` text content |
|
||||
| `url` | `h4 > a[href]` |
|
||||
| `company` | `.jix-toolbar-top__company a` text |
|
||||
| `companyUrl` | `.jix-toolbar-top__company a[href]` |
|
||||
| `location` | `span.jix_robotjob--area` text |
|
||||
| `date` | `time[datetime]` attribute value |
|
||||
| `description` | `p` text content (first `<p>` in card) |
|
||||
|
||||
Two card types exist: `div.PaidJob` (sponsored) and `div.jix_robotjob` (aggregated). Both use the same selector pattern.
|
||||
+25
@@ -0,0 +1,25 @@
|
||||
{
|
||||
"name": "jobindex-cli",
|
||||
"version": "1.0.0",
|
||||
"description": "CLI for searching jobs on Jobindex.dk",
|
||||
"type": "module",
|
||||
"main": "src/cli.ts",
|
||||
"bin": {
|
||||
"jobindex": "src/cli.ts"
|
||||
},
|
||||
"scripts": {
|
||||
"start": "bun run src/cli.ts",
|
||||
"test": "bun test --timeout 30000",
|
||||
"typecheck": "tsc --noEmit"
|
||||
},
|
||||
"dependencies": {
|
||||
"@bunli/core": "0.9.1",
|
||||
"@bunli/utils": "0.6.0",
|
||||
"node-html-parser": "^6.1.13",
|
||||
"zod": "^3.23.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"typescript": "^5.4.0",
|
||||
"@types/bun": "1.3.14"
|
||||
}
|
||||
}
|
||||
+45
@@ -0,0 +1,45 @@
|
||||
import { createCLI } from "@bunli/core"
|
||||
import { writeError } from "./helpers.js"
|
||||
import { search } from "./commands/search.js"
|
||||
import { detail } from "./commands/detail.js"
|
||||
|
||||
const cli = await createCLI({
|
||||
name: "jobindex-cli",
|
||||
version: "0.1.0",
|
||||
description: "CLI for searching jobs on Jobindex.dk",
|
||||
})
|
||||
|
||||
const commands = [search, detail]
|
||||
for (const command of commands) {
|
||||
cli.command(command)
|
||||
}
|
||||
|
||||
// Reject unknown --flags before dispatch. bunli silently discards them, and a
|
||||
// silently discarded filter changes what the search returns without any error
|
||||
// (a wrong flag name once returned an entire portal's database as if it
|
||||
// matched the query). add-portal.md's contract requires a bogus flag to exit 1
|
||||
// with a JSON error on stderr; this enforces it for the reference CLIs too.
|
||||
const argv = process.argv.slice(2)
|
||||
const invoked = commands.find((c) => (c as { name?: string }).name === argv[0])
|
||||
if (invoked) {
|
||||
const known = new Set([
|
||||
...Object.keys((invoked as { options?: Record<string, unknown> }).options ?? {}),
|
||||
"help",
|
||||
"version",
|
||||
])
|
||||
for (const token of argv.slice(1)) {
|
||||
if (token === "--") break
|
||||
if (token.startsWith("--")) {
|
||||
const flag = token.slice(2).split("=")[0]
|
||||
if (!known.has(flag)) {
|
||||
writeError(
|
||||
`unknown flag --${flag} for '${argv[0]}' - flags are never silently ignored, because a discarded filter changes what the search returns; see --help for the supported flags`,
|
||||
"UNKNOWN_FLAG",
|
||||
)
|
||||
process.exit(1)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
await cli.run()
|
||||
+321
@@ -0,0 +1,321 @@
|
||||
import { defineCommand, option } from "@bunli/core"
|
||||
import { z } from "zod"
|
||||
import { htmlFetch, writeError } from "../helpers.js"
|
||||
|
||||
const BASE_URL = "https://www.jobindex.dk"
|
||||
|
||||
interface DetailResult {
|
||||
id: string
|
||||
title: string
|
||||
company: string | null
|
||||
companyUrl: string | null
|
||||
location: string | null
|
||||
date: string | null
|
||||
deadline: string | null
|
||||
employmentType: string | null
|
||||
hours: string | null
|
||||
applyUrl: string | null
|
||||
url: string
|
||||
description: string | null
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert a Unicode code point to a string. Uses `fromCodePoint` (not
|
||||
* `fromCharCode`) so supplementary-plane code points (e.g. emoji, U+1F600)
|
||||
* decode correctly, and drops out-of-range values instead of throwing.
|
||||
*/
|
||||
function numericEntity(cp: number): string {
|
||||
return cp >= 0 && cp <= 0x10ffff ? String.fromCodePoint(cp) : ""
|
||||
}
|
||||
|
||||
/**
|
||||
* Decode HTML entities in text
|
||||
*/
|
||||
function decodeHtmlEntities(text: string): string {
|
||||
return text
|
||||
.replace(/&/g, "&")
|
||||
.replace(/</g, "<")
|
||||
.replace(/>/g, ">")
|
||||
.replace(/"/g, '"')
|
||||
.replace(/'/g, "'")
|
||||
.replace(/'/g, "'")
|
||||
// Danish letters appear as named entities in employer-hosted ad markup.
|
||||
.replace(/ø/g, "ø")
|
||||
.replace(/Ø/g, "Ø")
|
||||
.replace(/æ/g, "æ")
|
||||
.replace(/Æ/g, "Æ")
|
||||
.replace(/å/g, "å")
|
||||
.replace(/Å/g, "Å")
|
||||
// Numeric character references: decimal (é) and hexadecimal (é).
|
||||
.replace(/&#(\d+);/g, (_, dec) => numericEntity(parseInt(dec, 10)))
|
||||
.replace(/&#[xX]([0-9a-fA-F]+);/g, (_, hex) => numericEntity(parseInt(hex, 16)))
|
||||
.replace(/ /g, " ")
|
||||
}
|
||||
|
||||
/**
|
||||
* Strip HTML tags from text
|
||||
*/
|
||||
function stripTags(html: string): string {
|
||||
return html.replace(/<[^>]+>/g, "").trim()
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract job ID from URL or return as-is if already an ID
|
||||
*/
|
||||
function extractIdFromUrl(url: string): string {
|
||||
// Match IDs like h1647303, r13677312, etc.
|
||||
const match = url.match(/\/jobannonce\/([a-zA-Z]\d+)/)
|
||||
if (match) return match[1]
|
||||
return url
|
||||
}
|
||||
|
||||
function buildUrl(idOrUrl: string): { url: string; id: string } {
|
||||
if (idOrUrl.startsWith("http")) {
|
||||
const id = extractIdFromUrl(idOrUrl)
|
||||
return { url: idOrUrl, id }
|
||||
}
|
||||
// It's a bare ID
|
||||
const url = `${BASE_URL}/jobannonce/${idOrUrl}`
|
||||
return { url, id: idOrUrl }
|
||||
}
|
||||
|
||||
const DANISH_MONTHS: Record<string, string> = {
|
||||
januar: "01", februar: "02", marts: "03", april: "04", maj: "05", juni: "06",
|
||||
juli: "07", august: "08", september: "09", oktober: "10", november: "11", december: "12",
|
||||
}
|
||||
|
||||
/**
|
||||
* Normalize a date found on a detail page to YYYY-MM-DD, or null.
|
||||
* Live pages carry three shapes: ISO, DD-MM-YYYY (the hr-manager widget),
|
||||
* and Danish long form ("13. september 2026", the jobindex-native facts box).
|
||||
*/
|
||||
export function toIsoDate(value: string | null | undefined): string | null {
|
||||
if (!value) return null
|
||||
const text = value.trim()
|
||||
let m = text.match(/^(\d{4})-(\d{2})-(\d{2})/)
|
||||
if (m) return `${m[1]}-${m[2]}-${m[3]}`
|
||||
m = text.match(/^(\d{2})-(\d{2})-(\d{4})/)
|
||||
if (m) return `${m[3]}-${m[2]}-${m[1]}`
|
||||
m = text.match(/^(\d{1,2})\.?\s+([a-zæøå]+)\s+(\d{4})/i)
|
||||
if (m) {
|
||||
const month = DANISH_MONTHS[m[2].toLowerCase()]
|
||||
if (month) return `${m[3]}-${month}-${m[1].padStart(2, "0")}`
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
function metaContent(html: string, matcher: string): string | null {
|
||||
const re = new RegExp(
|
||||
`<meta[^>]+(?:property|name|itemprop)="${matcher}"[^>]+content="([^"]*)"|<meta[^>]+content="([^"]*)"[^>]+(?:property|name|itemprop)="${matcher}"`,
|
||||
"i",
|
||||
)
|
||||
const m = html.match(re)
|
||||
const value = m ? (m[1] ?? m[2]) : null
|
||||
return value ? decodeHtmlEntities(value).trim() || null : null
|
||||
}
|
||||
|
||||
/** The text of the <p> inside a jobindex-native jd-* facts block. */
|
||||
function jdBlockValue(html: string, cls: string): string | null {
|
||||
const m = html.match(new RegExp(`class="${cls}"[^>]*>[\\s\\S]*?<p[^>]*>([\\s\\S]*?)</p>`, "i"))
|
||||
return m ? decodeHtmlEntities(stripTags(m[1])).replace(/\s+/g, " ").trim() || null : null
|
||||
}
|
||||
|
||||
/** Drop head/script/style content so text scans never read CSS or JS. */
|
||||
function visibleHtml(html: string): string {
|
||||
return html
|
||||
.replace(/<head[\s\S]*?<\/head>/gi, "")
|
||||
.replace(/<script[\s\S]*?<\/script>/gi, "")
|
||||
.replace(/<style[\s\S]*?<\/style>/gi, "")
|
||||
.replace(/<!--[\s\S]*?-->/g, "")
|
||||
}
|
||||
|
||||
function bodyText(html: string): string | null {
|
||||
const text = decodeHtmlEntities(stripTags(visibleHtml(html).replace(/<(br|\/p|\/div|\/li|\/h[1-6])[^>]*>/gi, "\n")))
|
||||
.split("\n")
|
||||
.map((line) => line.replace(/\s+/g, " ").trim())
|
||||
.filter(Boolean)
|
||||
.join("\n")
|
||||
return text || null
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse a live detail page. Jobindex serves two shapes (verified live
|
||||
* 2026-08-19; the selectors the previous parser used exist in neither):
|
||||
*
|
||||
* - the jobindex-native shape, recognisable by its `jd-*` facts blocks
|
||||
* (jd-deadline, jd-location, ...), with the company as the `<title>`
|
||||
* prefix ("COMPANY - Job title");
|
||||
* - an external ATS passthrough (hr-manager/Talentech and similar), where
|
||||
* the page IS the employer's hosted ad: `og:url` points at the ATS,
|
||||
* `og:site_name` is the ATS brand, and there is no reliable company
|
||||
* anchor at all - so `company` is honestly null there, never the ATS.
|
||||
*
|
||||
* `id` and `url` are always the caller's jobindex id and its jobannonce
|
||||
* URL: the canonical/og:url on these pages is the external ATS, and
|
||||
* storing that broke /scrape's "store a URL that resolves to the posting".
|
||||
*/
|
||||
export function parseDetailPage(html: string, url: string, id: string): DetailResult {
|
||||
const isNative = html.includes('class="jd-')
|
||||
|
||||
const ogTitle = metaContent(html, "og:title")
|
||||
const itempropName = metaContent(html, "name")
|
||||
const h1 = html.match(/<h1[^>]*>([\s\S]*?)<\/h1>/i)
|
||||
const h1Title = h1 ? decodeHtmlEntities(stripTags(h1[1])).replace(/\s+/g, " ").trim() : null
|
||||
const title = ogTitle ?? itempropName ?? h1Title ?? ""
|
||||
if (!title) {
|
||||
throw new Error("Failed to parse job listing HTML")
|
||||
}
|
||||
|
||||
// Company: only the native shape carries one - as the <title> prefix,
|
||||
// "VELLIV - Udvikler til Camunda/AWS". Require the suffix to be the job
|
||||
// title so an unrelated <title> never becomes a company name.
|
||||
let company: string | null = null
|
||||
if (isNative) {
|
||||
const titleTag = html.match(/<title>([\s\S]*?)<\/title>/i)
|
||||
const pageTitle = titleTag ? decodeHtmlEntities(titleTag[1]).replace(/\s+/g, " ").trim() : ""
|
||||
if (pageTitle.endsWith(` - ${title}`)) {
|
||||
company = pageTitle.slice(0, -(title.length + 3)).trim() || null
|
||||
}
|
||||
}
|
||||
|
||||
let location: string | null = null
|
||||
let deadline: string | null = null
|
||||
let employmentType: string | null = null
|
||||
let hours: string | null = null
|
||||
let description: string | null = null
|
||||
|
||||
if (isNative) {
|
||||
location = jdBlockValue(html, "jd-location")
|
||||
deadline = toIsoDate(jdBlockValue(html, "jd-deadline"))
|
||||
employmentType = jdBlockValue(html, "jd-type")
|
||||
hours = jdBlockValue(html, "jd-workhours")
|
||||
const desc = html.match(/class="jd-description"[^>]*>([\s\S]*?)<\/div>/i)
|
||||
description = desc
|
||||
? decodeHtmlEntities(stripTags(desc[1])).replace(/\s+/g, " ").trim() || null
|
||||
: null
|
||||
} else {
|
||||
// hr-manager-style widget: a rowheader label followed by the value span.
|
||||
const workplace = visibleHtml(html).match(
|
||||
/class="workplace[^"]*"[\s\S]*?<span class="empty">([\s\S]*?)<\/span>/i,
|
||||
)
|
||||
location = workplace
|
||||
? decodeHtmlEntities(stripTags(workplace[1])).replace(/\s+/g, " ").trim() || null
|
||||
: null
|
||||
|
||||
// Deadline: label + a real date within range, scanned only over visible
|
||||
// markup - the label also appears inside a CSS comment on these pages,
|
||||
// which the previous parser captured verbatim as the deadline.
|
||||
const due = visibleHtml(html).match(
|
||||
/(?:Ansøgningsfrist|Application\s*due|Frist)[\s\S]{0,300}?(\d{2}-\d{2}-\d{4}|\d{4}-\d{2}-\d{2}|\d{1,2}\.?\s+[a-zæøå]+\s+\d{4})/i,
|
||||
)
|
||||
deadline = due ? toIsoDate(due[1]) : null
|
||||
|
||||
description = bodyText(html)
|
||||
if (!description || description.length < 100) {
|
||||
description = metaContent(html, "og:description") ?? metaContent(html, "description") ?? description
|
||||
}
|
||||
}
|
||||
|
||||
if (!description) {
|
||||
description = metaContent(html, "og:description")
|
||||
}
|
||||
|
||||
// Apply URL: jobindex's own /c?t= redirect when present.
|
||||
let applyUrl: string | null = null
|
||||
const ctMatch = html.match(/href="(\/c\?t=[^"]+)"/)
|
||||
if (ctMatch) {
|
||||
applyUrl = `${BASE_URL}${decodeHtmlEntities(ctMatch[1])}`
|
||||
}
|
||||
|
||||
const timeMatch = html.match(/<time[^>]+datetime="([^"]+)"/)
|
||||
|
||||
return {
|
||||
id,
|
||||
title,
|
||||
company,
|
||||
companyUrl: null,
|
||||
location,
|
||||
date: timeMatch ? toIsoDate(timeMatch[1]) : null,
|
||||
deadline,
|
||||
employmentType,
|
||||
hours,
|
||||
applyUrl,
|
||||
url,
|
||||
description,
|
||||
}
|
||||
}
|
||||
|
||||
export const detail = defineCommand({
|
||||
name: "detail",
|
||||
description: "Fetch full job listing detail by ID or URL",
|
||||
options: {
|
||||
format: option(z.enum(["json", "plain"]).default("json"), {
|
||||
description: "Output format: json, plain",
|
||||
}),
|
||||
},
|
||||
handler: async ({ positional, flags, signal }) => {
|
||||
if (signal.aborted) return
|
||||
|
||||
const idArg = positional[0]
|
||||
if (!idArg) {
|
||||
writeError("Job ID or URL is required", "MISSING_REQUIRED")
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
const { url, id } = buildUrl(idArg)
|
||||
|
||||
try {
|
||||
const html = await htmlFetch(url)
|
||||
|
||||
if (signal.aborted) return
|
||||
|
||||
let data: DetailResult
|
||||
try {
|
||||
data = parseDetailPage(html, url, id)
|
||||
} catch (parseErr) {
|
||||
const msg = parseErr instanceof Error ? parseErr.message : String(parseErr)
|
||||
writeError(msg, "PARSE_ERROR")
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
// Verify it's a valid job page (has a title)
|
||||
if (!data.title) {
|
||||
writeError("Failed to parse job listing HTML", "PARSE_ERROR")
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
if (flags.format === "json") {
|
||||
console.log(JSON.stringify(data, null, 2))
|
||||
} else {
|
||||
outputPlain(data)
|
||||
}
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err)
|
||||
if (message.includes("Job not found") || message.includes("404") || message.includes("NOT_FOUND")) {
|
||||
writeError("Job not found", "NOT_FOUND")
|
||||
} else if (message.includes("Failed to parse") || message.includes("PARSE_ERROR")) {
|
||||
writeError(message, "PARSE_ERROR")
|
||||
} else {
|
||||
writeError(message, "API_ERROR")
|
||||
}
|
||||
process.exit(1)
|
||||
}
|
||||
},
|
||||
})
|
||||
|
||||
function outputPlain(data: DetailResult): void {
|
||||
console.log(`id: ${data.id}`)
|
||||
console.log(`title: ${data.title}`)
|
||||
console.log(`company: ${data.company ?? "-"}`)
|
||||
console.log(`location: ${data.location ?? "-"}`)
|
||||
console.log(`date: ${data.date ?? "-"}`)
|
||||
console.log(`deadline: ${data.deadline ?? "-"}`)
|
||||
console.log(`employmentType: ${data.employmentType ?? "-"}`)
|
||||
console.log(`hours: ${data.hours ?? "-"}`)
|
||||
console.log(`applyUrl: ${data.applyUrl ?? "-"}`)
|
||||
console.log(`url: ${data.url}`)
|
||||
console.log("")
|
||||
if (data.description) {
|
||||
console.log(data.description)
|
||||
}
|
||||
}
|
||||
+103
@@ -0,0 +1,103 @@
|
||||
import { defineCommand, option } from "@bunli/core"
|
||||
import { z } from "zod"
|
||||
import { BASE_URL, htmlFetch, parseSearchPage, writeError, type JobCard } from "../helpers.js"
|
||||
|
||||
export const search = defineCommand({
|
||||
name: "search",
|
||||
description: "Search for job listings on Jobindex.dk",
|
||||
options: {
|
||||
query: option(z.string().optional(), {
|
||||
short: "q",
|
||||
description: "Keyword search query (e.g. python, grafisk designer)",
|
||||
}),
|
||||
page: option(z.coerce.number().int().min(1).default(1), {
|
||||
description: "Page number (1-indexed)",
|
||||
}),
|
||||
jobage: option(z.coerce.number().int().min(1).default(9999), {
|
||||
description: "Max age of posting in days: 1, 7, 14, 30, or 9999 (all)",
|
||||
}),
|
||||
sort: option(z.string().default("score"), {
|
||||
description: "Sort order: score (relevance) or date (newest first)",
|
||||
}),
|
||||
limit: option(z.coerce.number().int().min(1).optional(), {
|
||||
description: "Cap total results returned by the CLI (client-side)",
|
||||
}),
|
||||
format: option(z.enum(["json", "table", "plain"]).default("json"), {
|
||||
description: "Output format: json, table, plain",
|
||||
}),
|
||||
},
|
||||
handler: async ({ flags, signal }) => {
|
||||
if (!flags.query) {
|
||||
writeError("--query is required", "MISSING_REQUIRED")
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
if (signal.aborted) return
|
||||
|
||||
const params = new URLSearchParams({
|
||||
q: flags.query,
|
||||
page: String(flags.page),
|
||||
jobage: String(flags.jobage),
|
||||
sort: flags.sort,
|
||||
})
|
||||
|
||||
try {
|
||||
const html = await htmlFetch(`${BASE_URL}/jobsoegning?${params.toString()}`)
|
||||
|
||||
if (signal.aborted) return
|
||||
|
||||
const parsed = parseSearchPage(html)
|
||||
const total = parsed.total
|
||||
let results = parsed.results
|
||||
|
||||
if (flags.limit !== undefined) {
|
||||
results = results.slice(0, flags.limit)
|
||||
}
|
||||
|
||||
const output = {
|
||||
meta: {
|
||||
total,
|
||||
page: flags.page,
|
||||
perPage: 20,
|
||||
},
|
||||
results,
|
||||
}
|
||||
|
||||
if (flags.format === "json") {
|
||||
console.log(JSON.stringify(output, null, 2))
|
||||
} else if (flags.format === "table") {
|
||||
outputTable(results)
|
||||
} else {
|
||||
outputPlain(results)
|
||||
}
|
||||
} catch (err) {
|
||||
writeError(err instanceof Error ? err.message : String(err), "API_ERROR")
|
||||
process.exit(1)
|
||||
}
|
||||
},
|
||||
})
|
||||
|
||||
function outputTable(results: JobCard[]): void {
|
||||
console.log("id title company location")
|
||||
for (const r of results) {
|
||||
const id = r.id.padEnd(11)
|
||||
const title = r.title.substring(0, 40).padEnd(40)
|
||||
const company = (r.company ?? "-").substring(0, 20).padEnd(20)
|
||||
const location = r.location ?? "-"
|
||||
console.log(`${id} ${title} ${company} ${location}`)
|
||||
}
|
||||
}
|
||||
|
||||
function outputPlain(results: JobCard[]): void {
|
||||
for (const r of results) {
|
||||
console.log(`id: ${r.id}`)
|
||||
console.log(`title: ${r.title}`)
|
||||
console.log(`company: ${r.company ?? "-"}`)
|
||||
console.log(`location: ${r.location ?? "-"}`)
|
||||
console.log(`date: ${r.date ?? "-"}`)
|
||||
console.log(`deadline: ${r.deadline ?? "-"}`)
|
||||
console.log(`url: ${r.url}`)
|
||||
if (r.description) console.log(`description: ${r.description}`)
|
||||
console.log("")
|
||||
}
|
||||
}
|
||||
+346
@@ -0,0 +1,346 @@
|
||||
export const BASE_URL = "https://www.jobindex.dk"
|
||||
|
||||
export function writeError(error: string, code: string): void {
|
||||
process.stderr.write(JSON.stringify({ error, code }) + "\n")
|
||||
}
|
||||
|
||||
export async function apiFetch<T>(path: string, params?: Record<string, string>): Promise<T> {
|
||||
let url = `${BASE_URL}${path}`
|
||||
if (params && Object.keys(params).length > 0) {
|
||||
const qs = new URLSearchParams(params)
|
||||
url += `?${qs.toString()}`
|
||||
}
|
||||
|
||||
const maxRetries = 6
|
||||
let delay = 500
|
||||
for (let attempt = 0; attempt <= maxRetries; attempt++) {
|
||||
const response = await fetch(url, { signal: AbortSignal.timeout(15000) })
|
||||
if (response.status === 429 || response.status >= 500) {
|
||||
if (attempt === maxRetries) {
|
||||
throw new Error(`API request failed: ${response.status} ${response.statusText}`)
|
||||
}
|
||||
const jitter = Math.floor(Math.random() * 500)
|
||||
await new Promise((resolve) => setTimeout(resolve, delay + jitter))
|
||||
delay = Math.min(delay * 2, 5000)
|
||||
continue
|
||||
}
|
||||
if (!response.ok) {
|
||||
throw new Error(`API request failed: ${response.status} ${response.statusText}`)
|
||||
}
|
||||
return response.json() as Promise<T>
|
||||
}
|
||||
throw new Error("API request failed after max retries")
|
||||
}
|
||||
|
||||
export async function htmlFetch(url: string): Promise<string> {
|
||||
const maxRetries = 6
|
||||
let delay = 500
|
||||
for (let attempt = 0; attempt <= maxRetries; attempt++) {
|
||||
const response = await fetch(url, {
|
||||
headers: {
|
||||
"User-Agent": "Mozilla/5.0 (compatible; jobindex-cli/1.0)",
|
||||
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
|
||||
"Accept-Language": "da,en;q=0.9",
|
||||
},
|
||||
redirect: "follow",
|
||||
signal: AbortSignal.timeout(15000),
|
||||
})
|
||||
if (response.status === 429 || response.status >= 500) {
|
||||
if (attempt === maxRetries) {
|
||||
throw new Error(`API request failed: ${response.status} ${response.statusText}`)
|
||||
}
|
||||
const jitter = Math.floor(Math.random() * 500)
|
||||
await new Promise((resolve) => setTimeout(resolve, delay + jitter))
|
||||
delay = Math.min(delay * 2, 5000)
|
||||
continue
|
||||
}
|
||||
if (response.status === 404) {
|
||||
throw new Error(`Job not found`)
|
||||
}
|
||||
if (!response.ok) {
|
||||
throw new Error(`API request failed: ${response.status} ${response.statusText}`)
|
||||
}
|
||||
return response.text()
|
||||
}
|
||||
throw new Error("Request failed after max retries")
|
||||
}
|
||||
|
||||
export interface JobCard {
|
||||
id: string
|
||||
title: string
|
||||
company: string | null
|
||||
companyUrl: string | null
|
||||
location: string | null
|
||||
date: string | null
|
||||
deadline: string | null
|
||||
url: string
|
||||
description: string | null
|
||||
}
|
||||
|
||||
/**
|
||||
* Jobindex moved its search results client-side. The /jobsoegning.json endpoint
|
||||
* now returns 204 No Content. The HTML page (/jobsoegning) embeds the full result
|
||||
* payload in a `var Stash = {...}` script blob, under
|
||||
* jobsearch/result_app -> storeData -> searchResponse -> { hitcount, results[] }.
|
||||
* This extracts and parses that blob.
|
||||
*/
|
||||
export function extractStash(html: string): any {
|
||||
const marker = "var Stash = "
|
||||
const start = html.indexOf(marker)
|
||||
if (start === -1) throw new Error("Could not locate Stash blob in jobindex HTML")
|
||||
const open = start + marker.length
|
||||
let depth = 0
|
||||
let inStr = false
|
||||
let esc = false
|
||||
let end = -1
|
||||
for (let j = open; j < html.length; j++) {
|
||||
const c = html[j]
|
||||
if (inStr) {
|
||||
if (esc) esc = false
|
||||
else if (c === "\\") esc = true
|
||||
else if (c === '"') inStr = false
|
||||
} else {
|
||||
if (c === '"') inStr = true
|
||||
else if (c === "{") depth++
|
||||
else if (c === "}") {
|
||||
depth--
|
||||
if (depth === 0) {
|
||||
end = j + 1
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if (end === -1) throw new Error("Unterminated Stash blob in jobindex HTML")
|
||||
return JSON.parse(html.slice(open, end))
|
||||
}
|
||||
|
||||
function findSearchResponse(node: any): any {
|
||||
if (node && typeof node === "object") {
|
||||
if (!Array.isArray(node)) {
|
||||
if (
|
||||
node.searchResponse &&
|
||||
typeof node.searchResponse === "object" &&
|
||||
Array.isArray(node.searchResponse.results)
|
||||
) {
|
||||
return node.searchResponse
|
||||
}
|
||||
for (const key of Object.keys(node)) {
|
||||
const found = findSearchResponse(node[key])
|
||||
if (found) return found
|
||||
}
|
||||
} else {
|
||||
for (const item of node) {
|
||||
const found = findSearchResponse(item)
|
||||
if (found) return found
|
||||
}
|
||||
}
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
export interface SearchPageResult {
|
||||
total: number
|
||||
results: JobCard[]
|
||||
}
|
||||
|
||||
export function parseSearchPage(html: string): SearchPageResult {
|
||||
const stash = extractStash(html)
|
||||
const sr = findSearchResponse(stash)
|
||||
if (!sr) throw new Error("Could not locate searchResponse in jobindex Stash")
|
||||
|
||||
const results: JobCard[] = (sr.results ?? []).map((r: any): JobCard => {
|
||||
const tid: string = r.tid ?? ""
|
||||
let location: string | null = r.area ?? null
|
||||
if (!location) {
|
||||
try {
|
||||
location = r.geojson?.features?.[0]?.properties?.title ?? null
|
||||
} catch {
|
||||
location = null
|
||||
}
|
||||
}
|
||||
let deadline: string | null = null
|
||||
// apply_deadline_asap means the posting states no fixed deadline ("apply
|
||||
// now"). The /scrape contract represents that as null, and consumers do
|
||||
// date arithmetic on this field - so the flag maps to null, and wins over
|
||||
// any date field that happens to be present.
|
||||
if (r.apply_deadline_asap) deadline = null
|
||||
else if (typeof r.apply_deadline === "string") deadline = r.apply_deadline.slice(0, 10)
|
||||
else if (typeof r.lastdate === "string") deadline = r.lastdate
|
||||
|
||||
return {
|
||||
id: tid,
|
||||
title: r.headline ?? "",
|
||||
company: r.company?.name ?? r.companytext ?? null,
|
||||
companyUrl: r.company?.homeurl ?? null,
|
||||
location,
|
||||
date: r.firstdate ?? null,
|
||||
deadline,
|
||||
url: tid ? `${BASE_URL}/jobannonce/${tid}` : (r.share_url ?? r.url ?? ""),
|
||||
description: null,
|
||||
}
|
||||
})
|
||||
|
||||
const total = typeof sr.hitcount === "number" ? sr.hitcount : results.length
|
||||
return { total, results }
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert a Unicode code point to a string. Uses `fromCodePoint` (not
|
||||
* `fromCharCode`) so supplementary-plane code points (e.g. emoji, U+1F600)
|
||||
* decode correctly, and drops out-of-range values instead of throwing.
|
||||
*/
|
||||
function numericEntity(cp: number): string {
|
||||
return cp >= 0 && cp <= 0x10ffff ? String.fromCodePoint(cp) : ""
|
||||
}
|
||||
|
||||
/**
|
||||
* Decode HTML entities in text
|
||||
*/
|
||||
function decodeHtmlEntities(text: string): string {
|
||||
return text
|
||||
.replace(/&/g, "&")
|
||||
.replace(/</g, "<")
|
||||
.replace(/>/g, ">")
|
||||
.replace(/"/g, '"')
|
||||
.replace(/'/g, "'")
|
||||
.replace(/'/g, "'")
|
||||
// Numeric character references: decimal (é) and hexadecimal (é).
|
||||
.replace(/&#(\d+);/g, (_, dec) => numericEntity(parseInt(dec, 10)))
|
||||
.replace(/&#[xX]([0-9a-fA-F]+);/g, (_, hex) => numericEntity(parseInt(hex, 16)))
|
||||
.replace(/ /g, " ")
|
||||
}
|
||||
|
||||
/**
|
||||
* Strip HTML tags from text
|
||||
*/
|
||||
function stripTags(html: string): string {
|
||||
return html.replace(/<[^>]+>/g, "").trim()
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse job cards from result_list_box_html using regex.
|
||||
* node-html-parser has nesting bugs with this specific HTML structure
|
||||
* (unclosed tags inside buttons cause incorrect DOM tree).
|
||||
* Regex parsing is more reliable for this specific HTML format.
|
||||
*/
|
||||
export function parseJobCards(html: string): JobCard[] {
|
||||
const results: JobCard[] = []
|
||||
|
||||
// Split HTML by jobad-wrapper to get individual card HTML chunks
|
||||
const wrapperPattern = /<div[^>]+id="jobad-wrapper-(h\d+|r\d+)"[^>]*>([\s\S]*?)(?=<div[^>]+id="jobad-wrapper-|$)/g
|
||||
|
||||
let match: RegExpExecArray | null
|
||||
while ((match = wrapperPattern.exec(html)) !== null) {
|
||||
const id = match[1]
|
||||
const cardHtml = match[2]
|
||||
|
||||
// Extract title: look for <h4>...<a|A href="...">Title</a>...</h4>
|
||||
const titleMatch = cardHtml.match(/<h4[^>]*>[\s\S]*?<[Aa][^>]+href="([^"]+)"[^>]*>([\s\S]*?)<\/[Aa]>/i)
|
||||
if (!titleMatch) continue
|
||||
const rawTitle = stripTags(titleMatch[2])
|
||||
const title = decodeHtmlEntities(rawTitle)
|
||||
if (!title) continue
|
||||
|
||||
// Determine URL: prefer jobindex.dk /jobannonce/ URL, fallback to constructed URL
|
||||
let url: string
|
||||
const jobannonce = cardHtml.match(/href="(https:\/\/www\.jobindex\.dk\/jobannonce\/[^"]+)"/)
|
||||
if (jobannonce) {
|
||||
url = jobannonce[1]
|
||||
} else {
|
||||
// Construct canonical URL from ID
|
||||
url = `${BASE_URL}/jobannonce/${id}`
|
||||
}
|
||||
|
||||
// Extract company: <a ...> inside jix-toolbar-top__company
|
||||
let company: string | null = null
|
||||
let companyUrl: string | null = null
|
||||
const companySection = cardHtml.match(/class="jix-toolbar-top__company"[^>]*>([\s\S]*?)<\/div>/i)
|
||||
if (companySection) {
|
||||
const companyLinkMatch = companySection[1].match(/<[Aa][^>]+href="([^"]+)"[^>]*>([\s\S]*?)<\/[Aa]>/i)
|
||||
if (companyLinkMatch) {
|
||||
company = decodeHtmlEntities(stripTags(companyLinkMatch[2])) || null
|
||||
companyUrl = companyLinkMatch[1] || null
|
||||
}
|
||||
}
|
||||
|
||||
// Extract location: <span class="jix_robotjob--area">Location</span>
|
||||
const locMatch = cardHtml.match(/<span[^>]+class="jix_robotjob--area"[^>]*>([\s\S]*?)<\/span>/i)
|
||||
const location = locMatch ? decodeHtmlEntities(stripTags(locMatch[1])) || null : null
|
||||
|
||||
// Extract date: <time datetime="YYYY-MM-DD">
|
||||
const dateMatch = cardHtml.match(/<time[^>]+datetime="([^"]+)"/)
|
||||
const date = dateMatch ? dateMatch[1] : null
|
||||
|
||||
// Extract description: first <p class="..."> or first standalone <p> (not in toolbar)
|
||||
// Skip the toolbar/menu section and look for the description paragraph
|
||||
let description: string | null = null
|
||||
const innerSection = cardHtml.match(/class="PaidJob-inner"[^>]*>([\s\S]*?)(?:<\/div>\s*<\/div>|$)/i) ||
|
||||
cardHtml.match(/class="jix_robotjob-inner"[^>]*>([\s\S]*?)(?:<\/div>\s*<\/div>|$)/i)
|
||||
if (innerSection) {
|
||||
const pMatch = innerSection[1].match(/<p[^>]*>([\s\S]*?)<\/p>/i)
|
||||
if (pMatch) {
|
||||
const text = decodeHtmlEntities(stripTags(pMatch[1]))
|
||||
description = text.length > 0 ? text.substring(0, 300) : null
|
||||
}
|
||||
} else {
|
||||
// Fallback: look for p after the jobannonce link
|
||||
const pMatches = [...cardHtml.matchAll(/<p[^>]*>([\s\S]*?)<\/p>/gi)]
|
||||
for (const pm of pMatches) {
|
||||
const text = decodeHtmlEntities(stripTags(pm[1]))
|
||||
if (text.length > 20) {
|
||||
description = text.substring(0, 300)
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
results.push({
|
||||
id,
|
||||
title,
|
||||
company: company || null,
|
||||
companyUrl: companyUrl || null,
|
||||
location: location || null,
|
||||
date: date || null,
|
||||
deadline: null,
|
||||
url,
|
||||
description: description || null,
|
||||
})
|
||||
}
|
||||
|
||||
return results
|
||||
}
|
||||
|
||||
export function extractDivContent(html: string, className: string): string | null {
|
||||
const escaped = className.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')
|
||||
const openRe = new RegExp(`<div[^>]*class="[^"]*${escaped}[^"]*"[^>]*>`, 'i')
|
||||
const open = openRe.exec(html)
|
||||
if (!open) return null
|
||||
|
||||
let i = open.index + open[0].length
|
||||
let depth = 1
|
||||
|
||||
while (depth > 0 && i < html.length) {
|
||||
const nextOpen = html.indexOf('<div', i)
|
||||
const nextClose = html.indexOf('</div>', i)
|
||||
|
||||
if (nextClose === -1) return null
|
||||
|
||||
if (nextOpen !== -1 && nextOpen < nextClose) {
|
||||
depth++
|
||||
i = nextOpen + 4
|
||||
} else {
|
||||
depth--
|
||||
i = nextClose + 6
|
||||
}
|
||||
}
|
||||
|
||||
return html.slice(open.index + open[0].length, i - 6)
|
||||
}
|
||||
|
||||
export function parseHitCount(html: string): number {
|
||||
const match = html.match(/af <strong>([\d.]+)<\/strong>/)
|
||||
if (!match) return 0
|
||||
const numStr = match[1].replace(/\./g, "")
|
||||
return parseInt(numStr, 10) || 0
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import { runCLI } from "./helpers";
|
||||
|
||||
describe("Jobindex CLI error contract", () => {
|
||||
test("search without a query fails with JSON on stderr", async () => {
|
||||
const result = await runCLI(["search"]);
|
||||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.stdout).toBe("");
|
||||
expect(JSON.parse(result.stderr)).toEqual({
|
||||
error: "--query is required",
|
||||
code: "MISSING_REQUIRED",
|
||||
});
|
||||
});
|
||||
|
||||
test("detail without an ID fails before making a request", async () => {
|
||||
const result = await runCLI(["detail"]);
|
||||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.stdout).toBe("");
|
||||
expect(JSON.parse(result.stderr)).toEqual({
|
||||
error: "Job ID or URL is required",
|
||||
code: "MISSING_REQUIRED",
|
||||
});
|
||||
});
|
||||
|
||||
test("an invalid numeric option fails before making a request", async () => {
|
||||
const result = await runCLI(["search", "--query", "test", "--page", "not-a-number"]);
|
||||
const error = JSON.parse(result.stderr);
|
||||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.stdout).toBe("");
|
||||
expect(error.ok).toBe(false);
|
||||
expect(error.error.kind).toBe("validation");
|
||||
expect(error.error.option).toBe("page");
|
||||
expect(error.error.message).toContain("Expected number");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,80 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import { runCLI } from "./helpers";
|
||||
|
||||
// All cases fail schema validation (or the required-flag guard) before any
|
||||
// network request, so the suite is network-free. Regression context: a bare
|
||||
// z.coerce.number() accepted --limit=-1, and slice(0, -1) then silently
|
||||
// dropped the last result instead of erroring. Filter flags (--jobage) also
|
||||
// accepted negative and fractional values that were sent raw to the portal.
|
||||
|
||||
function expectValidationError(result: { exitCode: number; stdout: string; stderr: string }, option: string) {
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.stdout).toBe("");
|
||||
const error = JSON.parse(result.stderr);
|
||||
expect(error.ok).toBe(false);
|
||||
expect(error.error.kind).toBe("validation");
|
||||
expect(error.error.option).toBe(option);
|
||||
}
|
||||
|
||||
describe("Jobindex CLI flag validation", () => {
|
||||
test("--limit=-1 is rejected instead of silently dropping the last result", async () => {
|
||||
const result = await runCLI(["search", "--query", "test", "--limit=-1"]);
|
||||
expectValidationError(result, "limit");
|
||||
expect(JSON.parse(result.stderr).error.message).toContain("greater than or equal to 1");
|
||||
});
|
||||
|
||||
test("--limit=0 is rejected", async () => {
|
||||
const result = await runCLI(["search", "--query", "test", "--limit=0"]);
|
||||
expectValidationError(result, "limit");
|
||||
});
|
||||
|
||||
test("--limit=1.5 is rejected as non-integer", async () => {
|
||||
const result = await runCLI(["search", "--query", "test", "--limit=1.5"]);
|
||||
expectValidationError(result, "limit");
|
||||
expect(JSON.parse(result.stderr).error.message).toContain("Expected integer");
|
||||
});
|
||||
|
||||
test("--page=0 is rejected on the 1-indexed portal", async () => {
|
||||
const result = await runCLI(["search", "--query", "test", "--page=0"]);
|
||||
expectValidationError(result, "page");
|
||||
});
|
||||
|
||||
test("--jobage=-5 is rejected", async () => {
|
||||
const result = await runCLI(["search", "--query", "test", "--jobage=-5"]);
|
||||
expectValidationError(result, "jobage");
|
||||
expect(JSON.parse(result.stderr).error.message).toContain("greater than or equal to 1");
|
||||
});
|
||||
|
||||
test("--jobage=1.5 is rejected as non-integer", async () => {
|
||||
const result = await runCLI(["search", "--query", "test", "--jobage=1.5"]);
|
||||
expectValidationError(result, "jobage");
|
||||
expect(JSON.parse(result.stderr).error.message).toContain("Expected integer");
|
||||
});
|
||||
|
||||
test("valid numeric flags pass schema validation (proven offline via the required-flag guard)", async () => {
|
||||
const result = await runCLI(["search", "--page=2", "--limit=5"]);
|
||||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(JSON.parse(result.stderr)).toEqual({
|
||||
error: "--query is required",
|
||||
code: "MISSING_REQUIRED",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
describe("unknown flag rejection", () => {
|
||||
// add-portal.md's contract: "a bogus flag or missing required arg exits 1
|
||||
// with a JSON error on stderr". A silently discarded flag is worse than an
|
||||
// error: on jobdanmark a wrong flag name returned the entire database
|
||||
// (13,862 results) as if it matched the query (review finding F13,
|
||||
// 2026-08-19). Rejection happens before dispatch, so these are network-free.
|
||||
test("a bogus --flag exits 1 with a JSON error instead of being silently discarded", async () => {
|
||||
const result = await runCLI(["search", "--query", "test", "--bogus-flag", "xyz"]);
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.stdout).toBe("");
|
||||
const error = JSON.parse(result.stderr);
|
||||
expect(error.code).toBe("UNKNOWN_FLAG");
|
||||
expect(error.error).toContain("--bogus-flag");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,152 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import { parseDetailPage } from "../src/commands/detail";
|
||||
|
||||
// Jobindex redesigned its detail pages; every selector the old parser used
|
||||
// (job-text, jix-info, jix_robotjob--area, jix-toolbar-top__company) is gone
|
||||
// from live pages, so detail returned null company/location/date, CSS-comment
|
||||
// text as the deadline, and an external ATS URL as its own id/url - exit 0,
|
||||
// nothing signalling breakage (review finding F14, 2026-08-19, measured on
|
||||
// 5/5 live postings). These fixtures are trimmed from live pages captured
|
||||
// 2026-08-19: the jobindex-native "jd-*" shape and the external-ATS
|
||||
// (hr-manager/Talentech) passthrough shape.
|
||||
|
||||
const NATIVE_PAGE = `<!DOCTYPE html>
|
||||
<html lang="da">
|
||||
<head>
|
||||
<title>VELLIV - Udvikler til Camunda/AWS</title>
|
||||
<meta property="og:title" content="Udvikler til Camunda/AWS" />
|
||||
<meta property="og:url" content="https://www.jobindex.dk/jobannonce/h1690934" />
|
||||
<meta property="og:description" content="VELLIV" />
|
||||
</head>
|
||||
<body>
|
||||
<div class="jd-appetizer">
|
||||
<h1>Udvikler til Camunda/AWS</h1>
|
||||
</div>
|
||||
<div id="container" class="container">
|
||||
<div class="row">
|
||||
<div class="twelve columns jd-details">
|
||||
<div class="jd-description">
|
||||
<p class="appetizer">En virksomhed med mere end 100 års historie, der samtidig er cloud-only, er ikke hverdagskost.</p>
|
||||
<p>Hos Velliv får du mulighed for at arbejde med Camunda, AWS og automatisering af processer.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="four columns jd-facts">
|
||||
<div class="jd-type">
|
||||
<h3>Jobtype:</h3>
|
||||
<p>Fast</p>
|
||||
</div>
|
||||
<div class="jd-workhours">
|
||||
<h3>Arbejdstid:</h3>
|
||||
<p>Fuldtid</p>
|
||||
</div>
|
||||
<div class="jd-worktime">
|
||||
<h3>Arbejdsdage:</h3>
|
||||
<p>Dag</p>
|
||||
</div>
|
||||
<div class="jd-deadline">
|
||||
<h3>Ansøgningsfrist:</h3>
|
||||
<p>13. september 2026</p>
|
||||
</div>
|
||||
<div class="jd-location">
|
||||
<h3>Arbejdssted:</h3>
|
||||
<p>Ballerup</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</body>
|
||||
</html>`;
|
||||
|
||||
// The external shape: an employer's ATS-hosted ad served through jobindex.
|
||||
// og:url points at the ATS (NOT the posting), og:site_name is the ATS brand,
|
||||
// and the only occurrence of the deadline label outside the widget is inside
|
||||
// a CSS comment - the exact text the old regex captured as the deadline.
|
||||
const EXTERNAL_PAGE = `<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>
|
||||
Talentech - C#-udvikler til kritiske analyseløsninger i elnettet
|
||||
</title>
|
||||
<meta name="description" content="Vi søger en udvikler til elnettet" />
|
||||
<meta itemprop="name" content="C#-udvikler til kritiske analyseløsninger i elnettet" />
|
||||
<meta property="og:title" content="C#-udvikler til kritiske analyseløsninger i elnettet" />
|
||||
<meta property="og:site_name" content="Talentech" />
|
||||
<meta property="og:url" content="https://candidate.hr-manager.net/ApplicationInit.aspx?cid=316&ProjectId=188792" />
|
||||
<style>
|
||||
/* Defines the style of the Application Due text */
|
||||
/* DK: Ansøgningsfrist */
|
||||
/* BOKSTAV: K */
|
||||
.frist { padding-bottom: 15px; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<h1>C#-udvikler til kritiske analyseløsninger i elnettet</h1>
|
||||
<p>Vil du være med til at udvikle og drifte de systemer, der understøtter udbygningen af Danmarks kommende elnet? Vi arbejder i krydsfeltet mellem IT og energi.</p>
|
||||
<div class="workplacelist emptyparent"><div id="workplacelist_lang" class="rowheader">Workplace</div><div class="widget-line"></div><span class="empty">Fredericia</span><br></div>
|
||||
<div class="frist emptyparent"><div id="frist_lang" class="rowheader">Application due</div><div class="widget-line"></div><span class="empty">21-09-2026</span><br></div>
|
||||
</body>
|
||||
</html>`;
|
||||
|
||||
const JOBANNONCE_URL = "https://www.jobindex.dk/jobannonce/";
|
||||
|
||||
describe("parseDetailPage - jobindex-native (jd-*) shape", () => {
|
||||
const job = parseDetailPage(NATIVE_PAGE, `${JOBANNONCE_URL}h1690934`, "h1690934");
|
||||
|
||||
test("extracts the contract fields", () => {
|
||||
expect(job).toMatchObject({
|
||||
id: "h1690934",
|
||||
title: "Udvikler til Camunda/AWS",
|
||||
company: "VELLIV",
|
||||
location: "Ballerup",
|
||||
deadline: "2026-09-13",
|
||||
url: `${JOBANNONCE_URL}h1690934`,
|
||||
});
|
||||
});
|
||||
|
||||
test("converts the Danish long date to ISO", () => {
|
||||
expect(job.deadline).toBe("2026-09-13");
|
||||
});
|
||||
|
||||
test("extracts employment metadata and the description body", () => {
|
||||
expect(job.employmentType).toBe("Fast");
|
||||
expect(job.hours).toBe("Fuldtid");
|
||||
expect(job.description).toContain("Camunda, AWS og automatisering");
|
||||
});
|
||||
});
|
||||
|
||||
describe("parseDetailPage - external ATS passthrough shape", () => {
|
||||
const job = parseDetailPage(EXTERNAL_PAGE, `${JOBANNONCE_URL}h1690445`, "h1690445");
|
||||
|
||||
test("keeps the jobindex id and jobannonce URL, never the ATS og:url", () => {
|
||||
expect(job.id).toBe("h1690445");
|
||||
expect(job.url).toBe(`${JOBANNONCE_URL}h1690445`);
|
||||
expect(job.url).not.toContain("hr-manager.net");
|
||||
});
|
||||
|
||||
test("extracts the title and never reports the ATS brand as the company", () => {
|
||||
expect(job.title).toBe("C#-udvikler til kritiske analyseløsninger i elnettet");
|
||||
expect(job.company).toBeNull();
|
||||
});
|
||||
|
||||
test("extracts the workplace widget location", () => {
|
||||
expect(job.location).toBe("Fredericia");
|
||||
});
|
||||
|
||||
test("finds the real deadline, not the CSS comment", () => {
|
||||
expect(job.deadline).toBe("2026-09-21");
|
||||
});
|
||||
|
||||
test("description is readable body text, not a stylesheet", () => {
|
||||
expect(job.description).toContain("udbygningen af Danmarks kommende elnet");
|
||||
expect(job.description).not.toContain("padding-bottom");
|
||||
});
|
||||
|
||||
test("deadline is null when only the CSS comment mentions the label", () => {
|
||||
const noDueWidget = EXTERNAL_PAGE.replace(
|
||||
/<div class="frist emptyparent">[\s\S]*?<br><\/div>/,
|
||||
"",
|
||||
);
|
||||
const parsed = parseDetailPage(noDueWidget, `${JOBANNONCE_URL}h9`, "h9");
|
||||
expect(parsed.deadline).toBeNull();
|
||||
});
|
||||
});
|
||||
+39
@@ -0,0 +1,39 @@
|
||||
import { join } from "path";
|
||||
|
||||
const CLI_PATH = join(import.meta.dir, "../src/cli.ts");
|
||||
|
||||
export interface CLIResult {
|
||||
stdout: string;
|
||||
stderr: string;
|
||||
exitCode: number;
|
||||
}
|
||||
|
||||
export async function runCLI(args: string[]): Promise<CLIResult> {
|
||||
const proc = Bun.spawn(["bun", "run", CLI_PATH, ...args], {
|
||||
stdout: "pipe",
|
||||
stderr: "pipe",
|
||||
});
|
||||
|
||||
const [stdout, stderr, exitCode] = await Promise.all([
|
||||
new Response(proc.stdout).text(),
|
||||
new Response(proc.stderr).text(),
|
||||
proc.exited,
|
||||
]);
|
||||
|
||||
return { stdout: stdout.trim(), stderr: stderr.trim(), exitCode };
|
||||
}
|
||||
|
||||
export function parseJSON<T = unknown>(result: CLIResult): T {
|
||||
if (result.exitCode !== 0) {
|
||||
throw new Error(
|
||||
`CLI exited with code ${result.exitCode}. stderr: ${result.stderr}`
|
||||
);
|
||||
}
|
||||
try {
|
||||
return JSON.parse(result.stdout) as T;
|
||||
} catch {
|
||||
throw new Error(
|
||||
`Failed to parse JSON. stdout: ${result.stdout}\nstderr: ${result.stderr}`
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,98 @@
|
||||
import { describe, test, expect } from "bun:test";
|
||||
import { parseJobCards, extractDivContent } from "../src/helpers";
|
||||
|
||||
// Minimal jobad-wrapper markup: parseJobCards splits on `jobad-wrapper-<id>`
|
||||
// and reads the title from the <h4><a href>…</a></h4> and the company from the
|
||||
// jix-toolbar-top__company link. We inject HTML entities to exercise decoding.
|
||||
function card(id: string, title: string, company = "Acme A/S"): string {
|
||||
return `<div id="jobad-wrapper-${id}" class="PaidJob">
|
||||
<h4><a href="https://www.jobindex.dk/jobannonce/${id}">${title}</a></h4>
|
||||
<div class="jix-toolbar-top__company">
|
||||
<a href="https://www.jobindex.dk/virksomhed/acme">${company}</a>
|
||||
</div>
|
||||
</div>`;
|
||||
}
|
||||
|
||||
describe("decodeHtmlEntities (via parseJobCards)", () => {
|
||||
test("decodes hexadecimal numeric entities (ø -> ø)", () => {
|
||||
const [c] = parseJobCards(card("h1", "Smørrebro Chef"));
|
||||
expect(c.title).toBe("Smørrebro Chef");
|
||||
});
|
||||
|
||||
test("decodes uppercase-X hexadecimal entities (æ -> æ)", () => {
|
||||
const [c] = parseJobCards(card("h2", "Kære Kollega"));
|
||||
expect(c.title).toBe("Kære Kollega");
|
||||
});
|
||||
|
||||
test("still decodes decimal numeric entities (å -> å) — regression", () => {
|
||||
const [c] = parseJobCards(card("h3", "århus Lead"));
|
||||
expect(c.title).toBe("århus Lead");
|
||||
});
|
||||
|
||||
test("decodes supplementary-plane code points with fromCodePoint (😀)", () => {
|
||||
const [c] = parseJobCards(card("h4", "Growth 😀"));
|
||||
expect(c.title).toBe("Growth 😀");
|
||||
});
|
||||
|
||||
test("decodes hex supplementary-plane code points (😀)", () => {
|
||||
const [c] = parseJobCards(card("h5", "Growth 😀"));
|
||||
expect(c.title).toBe("Growth 😀");
|
||||
});
|
||||
|
||||
test("decodes hex entities in the company name too", () => {
|
||||
const [c] = parseJobCards(card("h6", "Engineer", "Nørrebro ApS"));
|
||||
expect(c.company).toBe("Nørrebro ApS");
|
||||
});
|
||||
});
|
||||
|
||||
describe("extractDivContent", () => {
|
||||
test("extracts content from simple div", () => {
|
||||
const html = '<div class="job-text">Simple text</div>';
|
||||
expect(extractDivContent(html, "job-text")).toBe("Simple text");
|
||||
});
|
||||
|
||||
test("extracts content with nested divs — the regression case", () => {
|
||||
const html = `<div class="job-text">
|
||||
<div class="highlight">First section</div>
|
||||
<div class="details">Second section with important info</div>
|
||||
</div>`;
|
||||
expect(extractDivContent(html, "job-text")).toBe(
|
||||
'\n <div class="highlight">First section</div>\n <div class="details">Second section with important info</div>\n ',
|
||||
);
|
||||
});
|
||||
|
||||
test("returns null when class not found", () => {
|
||||
expect(extractDivContent("<div>no class</div>", "nonexistent")).toBeNull();
|
||||
});
|
||||
|
||||
test("works with extra attributes on the div", () => {
|
||||
const html = '<div id="desc" class="job-text" data-x="1">Content</div>';
|
||||
expect(extractDivContent(html, "job-text")).toBe("Content");
|
||||
});
|
||||
|
||||
test("handles deeply nested divs (3 levels)", () => {
|
||||
const html = `<div class="job-text">
|
||||
<div>
|
||||
<div>Deep content</div>
|
||||
</div>
|
||||
</div>`;
|
||||
expect(extractDivContent(html, "job-text")).toBe(
|
||||
'\n <div>\n <div>Deep content</div>\n </div>\n ',
|
||||
);
|
||||
});
|
||||
|
||||
test("handles empty content", () => {
|
||||
const html = '<div class="job-text"></div>';
|
||||
expect(extractDivContent(html, "job-text")).toBe("");
|
||||
});
|
||||
|
||||
test("handles br and other non-div tags", () => {
|
||||
const html = '<div class="job-text">Line1<br>Line2<br>Line3</div>';
|
||||
expect(extractDivContent(html, "job-text")).toBe("Line1<br>Line2<br>Line3");
|
||||
});
|
||||
|
||||
test("escapes special regex characters in class name", () => {
|
||||
const html = '<div class="job-text (special)">Content</div>';
|
||||
expect(extractDivContent(html, "job-text (special)")).toBe("Content");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,34 @@
|
||||
import { afterEach, describe, expect, test } from "bun:test";
|
||||
import { apiFetch, htmlFetch } from "../src/helpers";
|
||||
|
||||
// A stalled upstream connection (accepted socket, no response) would otherwise
|
||||
// hang the CLI forever - fetch has no default timeout. Assert both request
|
||||
// wrappers carry an AbortSignal timeout. Fails on the pre-fix code (no signal).
|
||||
const originalFetch = globalThis.fetch;
|
||||
afterEach(() => {
|
||||
globalThis.fetch = originalFetch;
|
||||
});
|
||||
|
||||
describe("request timeout", () => {
|
||||
test("apiFetch passes an AbortSignal timeout to fetch", async () => {
|
||||
let init: RequestInit | undefined;
|
||||
globalThis.fetch = (async (_url: string | URL | Request, i?: RequestInit) => {
|
||||
init = i;
|
||||
return new Response("{}", { status: 200 });
|
||||
}) as unknown as typeof fetch;
|
||||
|
||||
await apiFetch("/search");
|
||||
expect(init?.signal).toBeInstanceOf(AbortSignal);
|
||||
});
|
||||
|
||||
test("htmlFetch passes an AbortSignal timeout to fetch", async () => {
|
||||
let init: RequestInit | undefined;
|
||||
globalThis.fetch = (async (_url: string | URL | Request, i?: RequestInit) => {
|
||||
init = i;
|
||||
return new Response("<html></html>", { status: 200 });
|
||||
}) as unknown as typeof fetch;
|
||||
|
||||
await htmlFetch("https://www.jobindex.dk/x");
|
||||
expect(init?.signal).toBeInstanceOf(AbortSignal);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,89 @@
|
||||
import { afterEach, describe, expect, test } from "bun:test";
|
||||
import { apiFetch, htmlFetch } from "../src/helpers";
|
||||
|
||||
// The portal contract requires backoff on 429/5xx. These tests pin the retry
|
||||
// loop offline: a stubbed fetch counts attempts, and a stubbed setTimeout
|
||||
// fires immediately so the exhaustion case does not sleep through the real
|
||||
// 500ms -> 5s backoff schedule. apiFetch and htmlFetch carry separate copies
|
||||
// of the loop, so both are exercised to keep them from drifting apart.
|
||||
|
||||
const originalFetch = globalThis.fetch;
|
||||
const originalSetTimeout = globalThis.setTimeout;
|
||||
|
||||
afterEach(() => {
|
||||
globalThis.fetch = originalFetch;
|
||||
globalThis.setTimeout = originalSetTimeout;
|
||||
});
|
||||
|
||||
function instantTimers() {
|
||||
globalThis.setTimeout = ((fn: () => void) =>
|
||||
originalSetTimeout(fn, 0)) as unknown as typeof setTimeout;
|
||||
}
|
||||
|
||||
function stubFetch(responses: Array<() => Response>): { calls: number } {
|
||||
const state = { calls: 0 };
|
||||
globalThis.fetch = (async () => {
|
||||
const i = Math.min(state.calls, responses.length - 1);
|
||||
state.calls++;
|
||||
return responses[i]();
|
||||
}) as unknown as typeof fetch;
|
||||
return state;
|
||||
}
|
||||
|
||||
describe("htmlFetch retry/backoff", () => {
|
||||
test("retries a 429 and succeeds on the next attempt", async () => {
|
||||
instantTimers();
|
||||
const state = stubFetch([
|
||||
() => new Response("", { status: 429 }),
|
||||
() => new Response("<html>ok</html>", { status: 200 }),
|
||||
]);
|
||||
|
||||
const html = await htmlFetch("https://www.jobindex.dk/x");
|
||||
expect(html).toContain("ok");
|
||||
expect(state.calls).toBe(2);
|
||||
});
|
||||
|
||||
test("does not retry a plain 4xx", async () => {
|
||||
const state = stubFetch([() => new Response("", { status: 400 })]);
|
||||
|
||||
await expect(htmlFetch("https://www.jobindex.dk/x")).rejects.toThrow(/400/);
|
||||
expect(state.calls).toBe(1);
|
||||
});
|
||||
|
||||
test("gives up after the initial attempt plus six retries on persistent 5xx", async () => {
|
||||
instantTimers();
|
||||
const state = stubFetch([() => new Response("", { status: 500 })]);
|
||||
|
||||
await expect(htmlFetch("https://www.jobindex.dk/x")).rejects.toThrow(/500/);
|
||||
expect(state.calls).toBe(7);
|
||||
});
|
||||
});
|
||||
|
||||
describe("apiFetch retry/backoff", () => {
|
||||
test("retries a 429 and succeeds on the next attempt", async () => {
|
||||
instantTimers();
|
||||
const state = stubFetch([
|
||||
() => new Response("", { status: 429 }),
|
||||
() => new Response('{"ok":true}', { status: 200 }),
|
||||
]);
|
||||
|
||||
const data = await apiFetch<{ ok: boolean }>("/x");
|
||||
expect(data.ok).toBe(true);
|
||||
expect(state.calls).toBe(2);
|
||||
});
|
||||
|
||||
test("does not retry a plain 4xx", async () => {
|
||||
const state = stubFetch([() => new Response("", { status: 400 })]);
|
||||
|
||||
await expect(apiFetch("/x")).rejects.toThrow(/400/);
|
||||
expect(state.calls).toBe(1);
|
||||
});
|
||||
|
||||
test("gives up after the initial attempt plus six retries on persistent 5xx", async () => {
|
||||
instantTimers();
|
||||
const state = stubFetch([() => new Response("", { status: 500 })]);
|
||||
|
||||
await expect(apiFetch("/x")).rejects.toThrow(/500/);
|
||||
expect(state.calls).toBe(7);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,69 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import { parseSearchPage } from "../src/helpers";
|
||||
|
||||
// parseSearchPage had no tests at all: mutating total to stop using
|
||||
// sr.hitcount survived the whole suite (review finding F35, 2026-08-19).
|
||||
// The fixture mirrors the real Stash nesting documented in helpers.ts:
|
||||
// jobsearch/result_app -> storeData -> searchResponse -> { hitcount, results[] }.
|
||||
function stashPage(searchResponse: object): string {
|
||||
const stash = { jobsearch: { result_app: { storeData: { searchResponse } } } };
|
||||
return `<html><head><script>var Stash = ${JSON.stringify(stash)};</script></head></html>`;
|
||||
}
|
||||
|
||||
const RESULT = {
|
||||
tid: "h1689961",
|
||||
headline: "Softwareudvikler",
|
||||
company: { name: "Acme A/S", homeurl: "https://acme.example" },
|
||||
area: "Aarhus",
|
||||
firstdate: "2026-08-10",
|
||||
apply_deadline: "2026-09-11T00:00:00",
|
||||
};
|
||||
|
||||
describe("parseSearchPage", () => {
|
||||
test("total comes from hitcount, not the page's result count", () => {
|
||||
const page = stashPage({ hitcount: 435, results: [RESULT] });
|
||||
const parsed = parseSearchPage(page);
|
||||
expect(parsed.total).toBe(435);
|
||||
expect(parsed.results).toHaveLength(1);
|
||||
});
|
||||
|
||||
test("total falls back to the result count when hitcount is absent", () => {
|
||||
const page = stashPage({ results: [RESULT, { ...RESULT, tid: "h2" }] });
|
||||
expect(parseSearchPage(page).total).toBe(2);
|
||||
});
|
||||
|
||||
test("maps the contract fields from a Stash result", () => {
|
||||
const [job] = parseSearchPage(stashPage({ hitcount: 1, results: [RESULT] })).results;
|
||||
expect(job).toMatchObject({
|
||||
id: "h1689961",
|
||||
title: "Softwareudvikler",
|
||||
company: "Acme A/S",
|
||||
location: "Aarhus",
|
||||
date: "2026-08-10",
|
||||
deadline: "2026-09-11",
|
||||
url: "https://www.jobindex.dk/jobannonce/h1689961",
|
||||
});
|
||||
});
|
||||
|
||||
test("maps an ASAP posting's deadline to null (no stated deadline)", () => {
|
||||
// apply_deadline_asap means "no fixed deadline, apply now". The /scrape
|
||||
// schema defines null as exactly that, and every consumer (rank's expiry
|
||||
// sweep, notion-sync's typed date column) does date arithmetic on this
|
||||
// field - a bare "ASAP" string broke all of them on half of live results
|
||||
// (review finding F12, 2026-08-19). lastdate present too: the flag wins.
|
||||
const [job] = parseSearchPage(
|
||||
stashPage({
|
||||
hitcount: 1,
|
||||
results: [{ ...RESULT, apply_deadline: undefined, apply_deadline_asap: true, lastdate: "2026-09-30" }],
|
||||
}),
|
||||
).results;
|
||||
expect(job.deadline).toBeNull();
|
||||
});
|
||||
|
||||
test("falls back to lastdate when apply_deadline is absent", () => {
|
||||
const [job] = parseSearchPage(
|
||||
stashPage({ hitcount: 1, results: [{ ...RESULT, apply_deadline: undefined, lastdate: "2026-09-30" }] }),
|
||||
).results;
|
||||
expect(job.deadline).toBe("2026-09-30");
|
||||
});
|
||||
});
|
||||
+20
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ESNext",
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "bundler",
|
||||
"lib": ["ESNext"],
|
||||
"types": ["bun-types"],
|
||||
"strict": true,
|
||||
"noImplicitAny": true,
|
||||
"strictNullChecks": true,
|
||||
"noUnusedLocals": true,
|
||||
"noUnusedParameters": true,
|
||||
"noFallthroughCasesInSwitch": true,
|
||||
"skipLibCheck": true,
|
||||
"outDir": "dist",
|
||||
"rootDir": "."
|
||||
},
|
||||
"include": ["src/**/*", "tests/**/*"],
|
||||
"exclude": ["node_modules", "dist"]
|
||||
}
|
||||
Reference in new issue
Block a user