feat: add skills

This commit is contained in:
Prad Nukala committed 2026-08-26 10:35:34 -04:00
1 parent 6984212b17
commit c50aa92862
150 files changed
+12702

No files matched your search

+247
View File
@@ -0,0 +1,247 @@
# jobindex-cli
CLI for searching jobs on [Jobindex.dk](https://www.jobindex.dk).
**Base URL**: `https://www.jobindex.dk/`
**Authentication**: None required.
**Format**: The API returns JSON with embedded HTML blobs. The CLI parses the HTML internally and emits clean JSON.
---
## Installation
```bash
cd skills/jobindex-search/cli
bun install
```
---
## Commands
| Command | Description |
|---------|-------------|
| `search` | Search for job listings |
| `detail` | Fetch full detail for a single job listing |
All commands accept `--format json|table|plain` (default: `json`).
All errors are written to **stderr** as `{ "error": "...", "code": "..." }` and the process exits with code `1`.
---
## `search` — Search for job listings
**Endpoint**: `GET https://www.jobindex.dk/jobsoegning.json`
```bash
bun run src/cli.ts search [flags]
```
The API always returns 20 results per page (fixed — no `--per-page` flag). The CLI parses the `result_list_box_html` HTML blob from the response to extract structured job records.
### Flags
| Flag | Type | Default | Description |
|------|------|---------|-------------|
| `--query` / `-q` | string | — | Keyword search query (e.g. `python`, `grafisk designer`) |
| `--page` | number | `1` | Page number (1-indexed) |
| `--jobage` | number | `9999` | Max age of posting in days: `1`, `7`, `14`, `30`, or `9999` (all) |
| `--sort` | string | `score` | Sort order: `score` (relevance) or `date` (newest first) |
| `--limit` | number | — | Cap total results returned by the CLI (client-side) |
| `--format` | string | `json` | Output format: `json`, `table`, `plain` |
### Sort options
| Value | Description |
|-------|-------------|
| `score` | Relevance / best match (default) |
| `date` | Newest postings first |
### jobage options
| Value | Description |
|-------|-------------|
| `1` | Posted today |
| `7` | Last 7 days |
| `14` | Last 14 days |
| `30` | Last 30 days |
| `9999` | All time (default) |
### Example
```bash
# Search for Python jobs posted in the last 7 days, sorted by date
bun run src/cli.ts search --query python --jobage 7 --sort date
# Search for "grafisk designer" jobs — show first 5 results
bun run src/cli.ts search --query "grafisk designer" --limit 5
# Page 2 of results for data engineer
bun run src/cli.ts search --query "data engineer" --page 2 --format table
```
### Response shape
```json
{
"meta": {
"total": 237,
"page": 1,
"perPage": 20
},
"results": [
{
"id": "h1647303",
"title": "Data Engineer til opbygning af Gavefabrikkens dataplatform",
"company": "Gavefabrikken",
"companyUrl": "https://www.gavefabrikken.dk/",
"location": "Valby",
"date": "2026-03-12",
"url": "https://www.jobindex.dk/jobannonce/h1647303/data-engineer-til-opbygning-af-gavefabrikkens-dataplatform",
"description": "Vi søger en dygtig Data Engineer til at opbygge og vedligeholde vores dataplatform..."
}
]
}
```
**Field notes:**
- `id` — string ID prefixed with `h` (e.g. `h1647303`). Use this with the `detail` command.
- `company` — company name; may be `null` for some aggregated listings.
- `companyUrl` — company homepage URL; may be `null` if not present.
- `location` — city or area; may be `null` if not listed.
- `date` — ISO date string (`YYYY-MM-DD`) from the `datetime` attribute on the `<time>` element; may be `null`.
- `description` — short excerpt from the listing; may be `null` or empty.
- `url` — full Jobindex.dk URL for the listing.
- `total` in `meta` — parsed from `hitcount_html` (Danish thousands separator `.` is stripped before parsing, e.g. `18.903` → `18903`).
> **Note on area filtering**: The Jobindex API does not reliably support area/region filtering via query parameters. `area` and `geoareaid` params are silently ignored. To filter by location, use `--query` with a city name (e.g. `--query "python aarhus"`) or apply `--limit` and filter the JSON output externally.
---
## `detail` — Fetch full job listing detail
**URL**: `https://www.jobindex.dk/jobannonce/{id}/{slug}`
```bash
bun run src/cli.ts detail <id> [--format json|plain]
```
The `id` is the job ID from `search` results (e.g. `h1647303`). The slug is optional — the CLI fetches the canonical URL by first constructing `https://www.jobindex.dk/jobannonce/{id}` and following any redirect, or by using the full URL from the `url` field in `search` results.
You may also pass the full URL directly as the `id` argument.
### Flags
| Flag | Type | Default | Description |
|------|------|---------|-------------|
| `--format` | string | `json` | Output format: `json`, `plain` |
### Example
```bash
# Using ID from search results
bun run src/cli.ts detail h1647303
# Using full URL
bun run src/cli.ts detail "https://www.jobindex.dk/jobannonce/h1647303/data-engineer-til-opbygning-af-gavefabrikkens-dataplatform"
# Plain text output
bun run src/cli.ts detail h1647303 --format plain
```
### Response shape
```json
{
"id": "h1647303",
"title": "Data Engineer til opbygning af Gavefabrikkens dataplatform",
"company": "Gavefabrikken",
"companyUrl": "https://www.gavefabrikken.dk/",
"location": "Valby, København",
"date": "2026-03-12",
"deadline": "2026-04-01",
"employmentType": "Fastansættelse",
"hours": "Fuldtid",
"applyUrl": "https://www.gavefabrikken.dk/jobs/apply/123",
"url": "https://www.jobindex.dk/jobannonce/h1647303/data-engineer-til-opbygning-af-gavefabrikkens-dataplatform",
"description": "Full job description text here..."
}
```
**Field notes:**
Jobindex serves detail pages in two shapes, and field availability differs:
a **jobindex-native** page (recognisable by its `jd-*` facts blocks) carries
company, location, an ISO deadline, employment type and hours; an **external
ATS passthrough** (the employer's hosted ad, e.g. hr-manager/Talentech, served
through jobindex) has no reliable company anchor, so `company` is `null` there
rather than the ATS brand, and location/deadline come from the ad's own
widgets when present.
- `id` / `url` — always the jobindex id and its `jobannonce` URL, never the
page's `og:url`/canonical (on passthrough pages those point at the external
ATS, not the posting).
- `deadline` — `YYYY-MM-DD` or `null`; Danish long dates ("13. september
2026") and `DD-MM-YYYY` widget dates are converted.
- `employmentType` / `hours` — from the native facts blocks; `null` on
passthrough pages.
- `companyUrl` — currently always `null`; no page shape carries a usable
company link.
- `applyUrl` — the Jobindex redirect link (`/c?t=...`) when present; `null`
otherwise.
- `description` — plain text of the ad body (HTML stripped), falling back to
the page's meta description when the body is empty.
- All fields except `id`, `title`, and `url` may be `null`.
---
## Error handling
All errors are written to **stderr** in JSON format and exit with code `1`:
```json
{ "error": "Job not found", "code": "NOT_FOUND" }
{ "error": "API request failed: 500 Internal Server Error", "code": "API_ERROR" }
{ "error": "Failed to parse job listing HTML", "code": "PARSE_ERROR" }
{ "error": "--query is required", "code": "MISSING_REQUIRED" }
```
---
## URL construction
Job detail pages on jobindex.dk:
- `https://www.jobindex.dk/jobannonce/{id}/{slug}`
The slug is part of the `url` returned by `search`. When calling `detail` with just an ID, the CLI fetches `https://www.jobindex.dk/jobannonce/{id}` which redirects to the full URL.
---
## Parsing notes
### Total count from `hitcount_html`
The API returns pagination info as an HTML string like:
```html
<div class="jix_pagination_total"><strong>1</strong> til <strong>20</strong> af <strong>18.903</strong> resultater.</div>
```
Parse total with: `/af <strong>([\d.]+)<\/strong>/` and strip `.` before converting to integer.
### Job card selectors
Each job card is wrapped in `[data-beacon-tid]`. Inside, select:
| Field | Selector |
|-------|----------|
| `id` | `[data-beacon-tid]` attribute value |
| `title` | `h4 > a` text content |
| `url` | `h4 > a[href]` |
| `company` | `.jix-toolbar-top__company a` text |
| `companyUrl` | `.jix-toolbar-top__company a[href]` |
| `location` | `span.jix_robotjob--area` text |
| `date` | `time[datetime]` attribute value |
| `description` | `p` text content (first `<p>` in card) |
Two card types exist: `div.PaidJob` (sponsored) and `div.jix_robotjob` (aggregated). Both use the same selector pattern.
+25
View File
@@ -0,0 +1,25 @@
{
"name": "jobindex-cli",
"version": "1.0.0",
"description": "CLI for searching jobs on Jobindex.dk",
"type": "module",
"main": "src/cli.ts",
"bin": {
"jobindex": "src/cli.ts"
},
"scripts": {
"start": "bun run src/cli.ts",
"test": "bun test --timeout 30000",
"typecheck": "tsc --noEmit"
},
"dependencies": {
"@bunli/core": "0.9.1",
"@bunli/utils": "0.6.0",
"node-html-parser": "^6.1.13",
"zod": "^3.23.0"
},
"devDependencies": {
"typescript": "^5.4.0",
"@types/bun": "1.3.14"
}
}
+45
View File
@@ -0,0 +1,45 @@
import { createCLI } from "@bunli/core"
import { writeError } from "./helpers.js"
import { search } from "./commands/search.js"
import { detail } from "./commands/detail.js"
const cli = await createCLI({
name: "jobindex-cli",
version: "0.1.0",
description: "CLI for searching jobs on Jobindex.dk",
})
const commands = [search, detail]
for (const command of commands) {
cli.command(command)
}
// Reject unknown --flags before dispatch. bunli silently discards them, and a
// silently discarded filter changes what the search returns without any error
// (a wrong flag name once returned an entire portal's database as if it
// matched the query). add-portal.md's contract requires a bogus flag to exit 1
// with a JSON error on stderr; this enforces it for the reference CLIs too.
const argv = process.argv.slice(2)
const invoked = commands.find((c) => (c as { name?: string }).name === argv[0])
if (invoked) {
const known = new Set([
...Object.keys((invoked as { options?: Record<string, unknown> }).options ?? {}),
"help",
"version",
])
for (const token of argv.slice(1)) {
if (token === "--") break
if (token.startsWith("--")) {
const flag = token.slice(2).split("=")[0]
if (!known.has(flag)) {
writeError(
`unknown flag --${flag} for '${argv[0]}' - flags are never silently ignored, because a discarded filter changes what the search returns; see --help for the supported flags`,
"UNKNOWN_FLAG",
)
process.exit(1)
}
}
}
}
await cli.run()
+321
View File
@@ -0,0 +1,321 @@
import { defineCommand, option } from "@bunli/core"
import { z } from "zod"
import { htmlFetch, writeError } from "../helpers.js"
const BASE_URL = "https://www.jobindex.dk"
interface DetailResult {
id: string
title: string
company: string | null
companyUrl: string | null
location: string | null
date: string | null
deadline: string | null
employmentType: string | null
hours: string | null
applyUrl: string | null
url: string
description: string | null
}
/**
* Convert a Unicode code point to a string. Uses `fromCodePoint` (not
* `fromCharCode`) so supplementary-plane code points (e.g. emoji, U+1F600)
* decode correctly, and drops out-of-range values instead of throwing.
*/
function numericEntity(cp: number): string {
return cp >= 0 && cp <= 0x10ffff ? String.fromCodePoint(cp) : ""
}
/**
* Decode HTML entities in text
*/
function decodeHtmlEntities(text: string): string {
return text
.replace(/&amp;/g, "&")
.replace(/&lt;/g, "<")
.replace(/&gt;/g, ">")
.replace(/&quot;/g, '"')
.replace(/&#39;/g, "'")
.replace(/&apos;/g, "'")
// Danish letters appear as named entities in employer-hosted ad markup.
.replace(/&oslash;/g, "ø")
.replace(/&Oslash;/g, "Ø")
.replace(/&aelig;/g, "æ")
.replace(/&AElig;/g, "Æ")
.replace(/&aring;/g, "å")
.replace(/&Aring;/g, "Å")
// Numeric character references: decimal (&#233;) and hexadecimal (&#xE9;).
.replace(/&#(\d+);/g, (_, dec) => numericEntity(parseInt(dec, 10)))
.replace(/&#[xX]([0-9a-fA-F]+);/g, (_, hex) => numericEntity(parseInt(hex, 16)))
.replace(/&nbsp;/g, " ")
}
/**
* Strip HTML tags from text
*/
function stripTags(html: string): string {
return html.replace(/<[^>]+>/g, "").trim()
}
/**
* Extract job ID from URL or return as-is if already an ID
*/
function extractIdFromUrl(url: string): string {
// Match IDs like h1647303, r13677312, etc.
const match = url.match(/\/jobannonce\/([a-zA-Z]\d+)/)
if (match) return match[1]
return url
}
function buildUrl(idOrUrl: string): { url: string; id: string } {
if (idOrUrl.startsWith("http")) {
const id = extractIdFromUrl(idOrUrl)
return { url: idOrUrl, id }
}
// It's a bare ID
const url = `${BASE_URL}/jobannonce/${idOrUrl}`
return { url, id: idOrUrl }
}
const DANISH_MONTHS: Record<string, string> = {
januar: "01", februar: "02", marts: "03", april: "04", maj: "05", juni: "06",
juli: "07", august: "08", september: "09", oktober: "10", november: "11", december: "12",
}
/**
* Normalize a date found on a detail page to YYYY-MM-DD, or null.
* Live pages carry three shapes: ISO, DD-MM-YYYY (the hr-manager widget),
* and Danish long form ("13. september 2026", the jobindex-native facts box).
*/
export function toIsoDate(value: string | null | undefined): string | null {
if (!value) return null
const text = value.trim()
let m = text.match(/^(\d{4})-(\d{2})-(\d{2})/)
if (m) return `${m[1]}-${m[2]}-${m[3]}`
m = text.match(/^(\d{2})-(\d{2})-(\d{4})/)
if (m) return `${m[3]}-${m[2]}-${m[1]}`
m = text.match(/^(\d{1,2})\.?\s+([a-zæøå]+)\s+(\d{4})/i)
if (m) {
const month = DANISH_MONTHS[m[2].toLowerCase()]
if (month) return `${m[3]}-${month}-${m[1].padStart(2, "0")}`
}
return null
}
function metaContent(html: string, matcher: string): string | null {
const re = new RegExp(
`<meta[^>]+(?:property|name|itemprop)="${matcher}"[^>]+content="([^"]*)"|<meta[^>]+content="([^"]*)"[^>]+(?:property|name|itemprop)="${matcher}"`,
"i",
)
const m = html.match(re)
const value = m ? (m[1] ?? m[2]) : null
return value ? decodeHtmlEntities(value).trim() || null : null
}
/** The text of the <p> inside a jobindex-native jd-* facts block. */
function jdBlockValue(html: string, cls: string): string | null {
const m = html.match(new RegExp(`class="${cls}"[^>]*>[\\s\\S]*?<p[^>]*>([\\s\\S]*?)</p>`, "i"))
return m ? decodeHtmlEntities(stripTags(m[1])).replace(/\s+/g, " ").trim() || null : null
}
/** Drop head/script/style content so text scans never read CSS or JS. */
function visibleHtml(html: string): string {
return html
.replace(/<head[\s\S]*?<\/head>/gi, "")
.replace(/<script[\s\S]*?<\/script>/gi, "")
.replace(/<style[\s\S]*?<\/style>/gi, "")
.replace(/<!--[\s\S]*?-->/g, "")
}
function bodyText(html: string): string | null {
const text = decodeHtmlEntities(stripTags(visibleHtml(html).replace(/<(br|\/p|\/div|\/li|\/h[1-6])[^>]*>/gi, "\n")))
.split("\n")
.map((line) => line.replace(/\s+/g, " ").trim())
.filter(Boolean)
.join("\n")
return text || null
}
/**
* Parse a live detail page. Jobindex serves two shapes (verified live
* 2026-08-19; the selectors the previous parser used exist in neither):
*
* - the jobindex-native shape, recognisable by its `jd-*` facts blocks
* (jd-deadline, jd-location, ...), with the company as the `<title>`
* prefix ("COMPANY - Job title");
* - an external ATS passthrough (hr-manager/Talentech and similar), where
* the page IS the employer's hosted ad: `og:url` points at the ATS,
* `og:site_name` is the ATS brand, and there is no reliable company
* anchor at all - so `company` is honestly null there, never the ATS.
*
* `id` and `url` are always the caller's jobindex id and its jobannonce
* URL: the canonical/og:url on these pages is the external ATS, and
* storing that broke /scrape's "store a URL that resolves to the posting".
*/
export function parseDetailPage(html: string, url: string, id: string): DetailResult {
const isNative = html.includes('class="jd-')
const ogTitle = metaContent(html, "og:title")
const itempropName = metaContent(html, "name")
const h1 = html.match(/<h1[^>]*>([\s\S]*?)<\/h1>/i)
const h1Title = h1 ? decodeHtmlEntities(stripTags(h1[1])).replace(/\s+/g, " ").trim() : null
const title = ogTitle ?? itempropName ?? h1Title ?? ""
if (!title) {
throw new Error("Failed to parse job listing HTML")
}
// Company: only the native shape carries one - as the <title> prefix,
// "VELLIV - Udvikler til Camunda/AWS". Require the suffix to be the job
// title so an unrelated <title> never becomes a company name.
let company: string | null = null
if (isNative) {
const titleTag = html.match(/<title>([\s\S]*?)<\/title>/i)
const pageTitle = titleTag ? decodeHtmlEntities(titleTag[1]).replace(/\s+/g, " ").trim() : ""
if (pageTitle.endsWith(` - ${title}`)) {
company = pageTitle.slice(0, -(title.length + 3)).trim() || null
}
}
let location: string | null = null
let deadline: string | null = null
let employmentType: string | null = null
let hours: string | null = null
let description: string | null = null
if (isNative) {
location = jdBlockValue(html, "jd-location")
deadline = toIsoDate(jdBlockValue(html, "jd-deadline"))
employmentType = jdBlockValue(html, "jd-type")
hours = jdBlockValue(html, "jd-workhours")
const desc = html.match(/class="jd-description"[^>]*>([\s\S]*?)<\/div>/i)
description = desc
? decodeHtmlEntities(stripTags(desc[1])).replace(/\s+/g, " ").trim() || null
: null
} else {
// hr-manager-style widget: a rowheader label followed by the value span.
const workplace = visibleHtml(html).match(
/class="workplace[^"]*"[\s\S]*?<span class="empty">([\s\S]*?)<\/span>/i,
)
location = workplace
? decodeHtmlEntities(stripTags(workplace[1])).replace(/\s+/g, " ").trim() || null
: null
// Deadline: label + a real date within range, scanned only over visible
// markup - the label also appears inside a CSS comment on these pages,
// which the previous parser captured verbatim as the deadline.
const due = visibleHtml(html).match(
/(?:Ansøgningsfrist|Application\s*due|Frist)[\s\S]{0,300}?(\d{2}-\d{2}-\d{4}|\d{4}-\d{2}-\d{2}|\d{1,2}\.?\s+[a-zæøå]+\s+\d{4})/i,
)
deadline = due ? toIsoDate(due[1]) : null
description = bodyText(html)
if (!description || description.length < 100) {
description = metaContent(html, "og:description") ?? metaContent(html, "description") ?? description
}
}
if (!description) {
description = metaContent(html, "og:description")
}
// Apply URL: jobindex's own /c?t= redirect when present.
let applyUrl: string | null = null
const ctMatch = html.match(/href="(\/c\?t=[^"]+)"/)
if (ctMatch) {
applyUrl = `${BASE_URL}${decodeHtmlEntities(ctMatch[1])}`
}
const timeMatch = html.match(/<time[^>]+datetime="([^"]+)"/)
return {
id,
title,
company,
companyUrl: null,
location,
date: timeMatch ? toIsoDate(timeMatch[1]) : null,
deadline,
employmentType,
hours,
applyUrl,
url,
description,
}
}
export const detail = defineCommand({
name: "detail",
description: "Fetch full job listing detail by ID or URL",
options: {
format: option(z.enum(["json", "plain"]).default("json"), {
description: "Output format: json, plain",
}),
},
handler: async ({ positional, flags, signal }) => {
if (signal.aborted) return
const idArg = positional[0]
if (!idArg) {
writeError("Job ID or URL is required", "MISSING_REQUIRED")
process.exit(1)
}
const { url, id } = buildUrl(idArg)
try {
const html = await htmlFetch(url)
if (signal.aborted) return
let data: DetailResult
try {
data = parseDetailPage(html, url, id)
} catch (parseErr) {
const msg = parseErr instanceof Error ? parseErr.message : String(parseErr)
writeError(msg, "PARSE_ERROR")
process.exit(1)
}
// Verify it's a valid job page (has a title)
if (!data.title) {
writeError("Failed to parse job listing HTML", "PARSE_ERROR")
process.exit(1)
}
if (flags.format === "json") {
console.log(JSON.stringify(data, null, 2))
} else {
outputPlain(data)
}
} catch (err) {
const message = err instanceof Error ? err.message : String(err)
if (message.includes("Job not found") || message.includes("404") || message.includes("NOT_FOUND")) {
writeError("Job not found", "NOT_FOUND")
} else if (message.includes("Failed to parse") || message.includes("PARSE_ERROR")) {
writeError(message, "PARSE_ERROR")
} else {
writeError(message, "API_ERROR")
}
process.exit(1)
}
},
})
function outputPlain(data: DetailResult): void {
console.log(`id: ${data.id}`)
console.log(`title: ${data.title}`)
console.log(`company: ${data.company ?? "-"}`)
console.log(`location: ${data.location ?? "-"}`)
console.log(`date: ${data.date ?? "-"}`)
console.log(`deadline: ${data.deadline ?? "-"}`)
console.log(`employmentType: ${data.employmentType ?? "-"}`)
console.log(`hours: ${data.hours ?? "-"}`)
console.log(`applyUrl: ${data.applyUrl ?? "-"}`)
console.log(`url: ${data.url}`)
console.log("")
if (data.description) {
console.log(data.description)
}
}
+103
View File
@@ -0,0 +1,103 @@
import { defineCommand, option } from "@bunli/core"
import { z } from "zod"
import { BASE_URL, htmlFetch, parseSearchPage, writeError, type JobCard } from "../helpers.js"
export const search = defineCommand({
name: "search",
description: "Search for job listings on Jobindex.dk",
options: {
query: option(z.string().optional(), {
short: "q",
description: "Keyword search query (e.g. python, grafisk designer)",
}),
page: option(z.coerce.number().int().min(1).default(1), {
description: "Page number (1-indexed)",
}),
jobage: option(z.coerce.number().int().min(1).default(9999), {
description: "Max age of posting in days: 1, 7, 14, 30, or 9999 (all)",
}),
sort: option(z.string().default("score"), {
description: "Sort order: score (relevance) or date (newest first)",
}),
limit: option(z.coerce.number().int().min(1).optional(), {
description: "Cap total results returned by the CLI (client-side)",
}),
format: option(z.enum(["json", "table", "plain"]).default("json"), {
description: "Output format: json, table, plain",
}),
},
handler: async ({ flags, signal }) => {
if (!flags.query) {
writeError("--query is required", "MISSING_REQUIRED")
process.exit(1)
}
if (signal.aborted) return
const params = new URLSearchParams({
q: flags.query,
page: String(flags.page),
jobage: String(flags.jobage),
sort: flags.sort,
})
try {
const html = await htmlFetch(`${BASE_URL}/jobsoegning?${params.toString()}`)
if (signal.aborted) return
const parsed = parseSearchPage(html)
const total = parsed.total
let results = parsed.results
if (flags.limit !== undefined) {
results = results.slice(0, flags.limit)
}
const output = {
meta: {
total,
page: flags.page,
perPage: 20,
},
results,
}
if (flags.format === "json") {
console.log(JSON.stringify(output, null, 2))
} else if (flags.format === "table") {
outputTable(results)
} else {
outputPlain(results)
}
} catch (err) {
writeError(err instanceof Error ? err.message : String(err), "API_ERROR")
process.exit(1)
}
},
})
function outputTable(results: JobCard[]): void {
console.log("id title company location")
for (const r of results) {
const id = r.id.padEnd(11)
const title = r.title.substring(0, 40).padEnd(40)
const company = (r.company ?? "-").substring(0, 20).padEnd(20)
const location = r.location ?? "-"
console.log(`${id} ${title} ${company} ${location}`)
}
}
function outputPlain(results: JobCard[]): void {
for (const r of results) {
console.log(`id: ${r.id}`)
console.log(`title: ${r.title}`)
console.log(`company: ${r.company ?? "-"}`)
console.log(`location: ${r.location ?? "-"}`)
console.log(`date: ${r.date ?? "-"}`)
console.log(`deadline: ${r.deadline ?? "-"}`)
console.log(`url: ${r.url}`)
if (r.description) console.log(`description: ${r.description}`)
console.log("")
}
}
+346
View File
@@ -0,0 +1,346 @@
export const BASE_URL = "https://www.jobindex.dk"
export function writeError(error: string, code: string): void {
process.stderr.write(JSON.stringify({ error, code }) + "\n")
}
export async function apiFetch<T>(path: string, params?: Record<string, string>): Promise<T> {
let url = `${BASE_URL}${path}`
if (params && Object.keys(params).length > 0) {
const qs = new URLSearchParams(params)
url += `?${qs.toString()}`
}
const maxRetries = 6
let delay = 500
for (let attempt = 0; attempt <= maxRetries; attempt++) {
const response = await fetch(url, { signal: AbortSignal.timeout(15000) })
if (response.status === 429 || response.status >= 500) {
if (attempt === maxRetries) {
throw new Error(`API request failed: ${response.status} ${response.statusText}`)
}
const jitter = Math.floor(Math.random() * 500)
await new Promise((resolve) => setTimeout(resolve, delay + jitter))
delay = Math.min(delay * 2, 5000)
continue
}
if (!response.ok) {
throw new Error(`API request failed: ${response.status} ${response.statusText}`)
}
return response.json() as Promise<T>
}
throw new Error("API request failed after max retries")
}
export async function htmlFetch(url: string): Promise<string> {
const maxRetries = 6
let delay = 500
for (let attempt = 0; attempt <= maxRetries; attempt++) {
const response = await fetch(url, {
headers: {
"User-Agent": "Mozilla/5.0 (compatible; jobindex-cli/1.0)",
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"Accept-Language": "da,en;q=0.9",
},
redirect: "follow",
signal: AbortSignal.timeout(15000),
})
if (response.status === 429 || response.status >= 500) {
if (attempt === maxRetries) {
throw new Error(`API request failed: ${response.status} ${response.statusText}`)
}
const jitter = Math.floor(Math.random() * 500)
await new Promise((resolve) => setTimeout(resolve, delay + jitter))
delay = Math.min(delay * 2, 5000)
continue
}
if (response.status === 404) {
throw new Error(`Job not found`)
}
if (!response.ok) {
throw new Error(`API request failed: ${response.status} ${response.statusText}`)
}
return response.text()
}
throw new Error("Request failed after max retries")
}
export interface JobCard {
id: string
title: string
company: string | null
companyUrl: string | null
location: string | null
date: string | null
deadline: string | null
url: string
description: string | null
}
/**
* Jobindex moved its search results client-side. The /jobsoegning.json endpoint
* now returns 204 No Content. The HTML page (/jobsoegning) embeds the full result
* payload in a `var Stash = {...}` script blob, under
* jobsearch/result_app -> storeData -> searchResponse -> { hitcount, results[] }.
* This extracts and parses that blob.
*/
export function extractStash(html: string): any {
const marker = "var Stash = "
const start = html.indexOf(marker)
if (start === -1) throw new Error("Could not locate Stash blob in jobindex HTML")
const open = start + marker.length
let depth = 0
let inStr = false
let esc = false
let end = -1
for (let j = open; j < html.length; j++) {
const c = html[j]
if (inStr) {
if (esc) esc = false
else if (c === "\\") esc = true
else if (c === '"') inStr = false
} else {
if (c === '"') inStr = true
else if (c === "{") depth++
else if (c === "}") {
depth--
if (depth === 0) {
end = j + 1
break
}
}
}
}
if (end === -1) throw new Error("Unterminated Stash blob in jobindex HTML")
return JSON.parse(html.slice(open, end))
}
function findSearchResponse(node: any): any {
if (node && typeof node === "object") {
if (!Array.isArray(node)) {
if (
node.searchResponse &&
typeof node.searchResponse === "object" &&
Array.isArray(node.searchResponse.results)
) {
return node.searchResponse
}
for (const key of Object.keys(node)) {
const found = findSearchResponse(node[key])
if (found) return found
}
} else {
for (const item of node) {
const found = findSearchResponse(item)
if (found) return found
}
}
}
return null
}
export interface SearchPageResult {
total: number
results: JobCard[]
}
export function parseSearchPage(html: string): SearchPageResult {
const stash = extractStash(html)
const sr = findSearchResponse(stash)
if (!sr) throw new Error("Could not locate searchResponse in jobindex Stash")
const results: JobCard[] = (sr.results ?? []).map((r: any): JobCard => {
const tid: string = r.tid ?? ""
let location: string | null = r.area ?? null
if (!location) {
try {
location = r.geojson?.features?.[0]?.properties?.title ?? null
} catch {
location = null
}
}
let deadline: string | null = null
// apply_deadline_asap means the posting states no fixed deadline ("apply
// now"). The /scrape contract represents that as null, and consumers do
// date arithmetic on this field - so the flag maps to null, and wins over
// any date field that happens to be present.
if (r.apply_deadline_asap) deadline = null
else if (typeof r.apply_deadline === "string") deadline = r.apply_deadline.slice(0, 10)
else if (typeof r.lastdate === "string") deadline = r.lastdate
return {
id: tid,
title: r.headline ?? "",
company: r.company?.name ?? r.companytext ?? null,
companyUrl: r.company?.homeurl ?? null,
location,
date: r.firstdate ?? null,
deadline,
url: tid ? `${BASE_URL}/jobannonce/${tid}` : (r.share_url ?? r.url ?? ""),
description: null,
}
})
const total = typeof sr.hitcount === "number" ? sr.hitcount : results.length
return { total, results }
}
/**
* Convert a Unicode code point to a string. Uses `fromCodePoint` (not
* `fromCharCode`) so supplementary-plane code points (e.g. emoji, U+1F600)
* decode correctly, and drops out-of-range values instead of throwing.
*/
function numericEntity(cp: number): string {
return cp >= 0 && cp <= 0x10ffff ? String.fromCodePoint(cp) : ""
}
/**
* Decode HTML entities in text
*/
function decodeHtmlEntities(text: string): string {
return text
.replace(/&amp;/g, "&")
.replace(/&lt;/g, "<")
.replace(/&gt;/g, ">")
.replace(/&quot;/g, '"')
.replace(/&#39;/g, "'")
.replace(/&apos;/g, "'")
// Numeric character references: decimal (&#233;) and hexadecimal (&#xE9;).
.replace(/&#(\d+);/g, (_, dec) => numericEntity(parseInt(dec, 10)))
.replace(/&#[xX]([0-9a-fA-F]+);/g, (_, hex) => numericEntity(parseInt(hex, 16)))
.replace(/&nbsp;/g, " ")
}
/**
* Strip HTML tags from text
*/
function stripTags(html: string): string {
return html.replace(/<[^>]+>/g, "").trim()
}
/**
* Parse job cards from result_list_box_html using regex.
* node-html-parser has nesting bugs with this specific HTML structure
* (unclosed tags inside buttons cause incorrect DOM tree).
* Regex parsing is more reliable for this specific HTML format.
*/
export function parseJobCards(html: string): JobCard[] {
const results: JobCard[] = []
// Split HTML by jobad-wrapper to get individual card HTML chunks
const wrapperPattern = /<div[^>]+id="jobad-wrapper-(h\d+|r\d+)"[^>]*>([\s\S]*?)(?=<div[^>]+id="jobad-wrapper-|$)/g
let match: RegExpExecArray | null
while ((match = wrapperPattern.exec(html)) !== null) {
const id = match[1]
const cardHtml = match[2]
// Extract title: look for <h4>...<a|A href="...">Title</a>...</h4>
const titleMatch = cardHtml.match(/<h4[^>]*>[\s\S]*?<[Aa][^>]+href="([^"]+)"[^>]*>([\s\S]*?)<\/[Aa]>/i)
if (!titleMatch) continue
const rawTitle = stripTags(titleMatch[2])
const title = decodeHtmlEntities(rawTitle)
if (!title) continue
// Determine URL: prefer jobindex.dk /jobannonce/ URL, fallback to constructed URL
let url: string
const jobannonce = cardHtml.match(/href="(https:\/\/www\.jobindex\.dk\/jobannonce\/[^"]+)"/)
if (jobannonce) {
url = jobannonce[1]
} else {
// Construct canonical URL from ID
url = `${BASE_URL}/jobannonce/${id}`
}
// Extract company: <a ...> inside jix-toolbar-top__company
let company: string | null = null
let companyUrl: string | null = null
const companySection = cardHtml.match(/class="jix-toolbar-top__company"[^>]*>([\s\S]*?)<\/div>/i)
if (companySection) {
const companyLinkMatch = companySection[1].match(/<[Aa][^>]+href="([^"]+)"[^>]*>([\s\S]*?)<\/[Aa]>/i)
if (companyLinkMatch) {
company = decodeHtmlEntities(stripTags(companyLinkMatch[2])) || null
companyUrl = companyLinkMatch[1] || null
}
}
// Extract location: <span class="jix_robotjob--area">Location</span>
const locMatch = cardHtml.match(/<span[^>]+class="jix_robotjob--area"[^>]*>([\s\S]*?)<\/span>/i)
const location = locMatch ? decodeHtmlEntities(stripTags(locMatch[1])) || null : null
// Extract date: <time datetime="YYYY-MM-DD">
const dateMatch = cardHtml.match(/<time[^>]+datetime="([^"]+)"/)
const date = dateMatch ? dateMatch[1] : null
// Extract description: first <p class="..."> or first standalone <p> (not in toolbar)
// Skip the toolbar/menu section and look for the description paragraph
let description: string | null = null
const innerSection = cardHtml.match(/class="PaidJob-inner"[^>]*>([\s\S]*?)(?:<\/div>\s*<\/div>|$)/i) ||
cardHtml.match(/class="jix_robotjob-inner"[^>]*>([\s\S]*?)(?:<\/div>\s*<\/div>|$)/i)
if (innerSection) {
const pMatch = innerSection[1].match(/<p[^>]*>([\s\S]*?)<\/p>/i)
if (pMatch) {
const text = decodeHtmlEntities(stripTags(pMatch[1]))
description = text.length > 0 ? text.substring(0, 300) : null
}
} else {
// Fallback: look for p after the jobannonce link
const pMatches = [...cardHtml.matchAll(/<p[^>]*>([\s\S]*?)<\/p>/gi)]
for (const pm of pMatches) {
const text = decodeHtmlEntities(stripTags(pm[1]))
if (text.length > 20) {
description = text.substring(0, 300)
break
}
}
}
results.push({
id,
title,
company: company || null,
companyUrl: companyUrl || null,
location: location || null,
date: date || null,
deadline: null,
url,
description: description || null,
})
}
return results
}
export function extractDivContent(html: string, className: string): string | null {
const escaped = className.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')
const openRe = new RegExp(`<div[^>]*class="[^"]*${escaped}[^"]*"[^>]*>`, 'i')
const open = openRe.exec(html)
if (!open) return null
let i = open.index + open[0].length
let depth = 1
while (depth > 0 && i < html.length) {
const nextOpen = html.indexOf('<div', i)
const nextClose = html.indexOf('</div>', i)
if (nextClose === -1) return null
if (nextOpen !== -1 && nextOpen < nextClose) {
depth++
i = nextOpen + 4
} else {
depth--
i = nextClose + 6
}
}
return html.slice(open.index + open[0].length, i - 6)
}
export function parseHitCount(html: string): number {
const match = html.match(/af <strong>([\d.]+)<\/strong>/)
if (!match) return 0
const numStr = match[1].replace(/\./g, "")
return parseInt(numStr, 10) || 0
}
@@ -0,0 +1,38 @@
import { describe, expect, test } from "bun:test";
import { runCLI } from "./helpers";
describe("Jobindex CLI error contract", () => {
test("search without a query fails with JSON on stderr", async () => {
const result = await runCLI(["search"]);
expect(result.exitCode).toBe(1);
expect(result.stdout).toBe("");
expect(JSON.parse(result.stderr)).toEqual({
error: "--query is required",
code: "MISSING_REQUIRED",
});
});
test("detail without an ID fails before making a request", async () => {
const result = await runCLI(["detail"]);
expect(result.exitCode).toBe(1);
expect(result.stdout).toBe("");
expect(JSON.parse(result.stderr)).toEqual({
error: "Job ID or URL is required",
code: "MISSING_REQUIRED",
});
});
test("an invalid numeric option fails before making a request", async () => {
const result = await runCLI(["search", "--query", "test", "--page", "not-a-number"]);
const error = JSON.parse(result.stderr);
expect(result.exitCode).toBe(1);
expect(result.stdout).toBe("");
expect(error.ok).toBe(false);
expect(error.error.kind).toBe("validation");
expect(error.error.option).toBe("page");
expect(error.error.message).toContain("Expected number");
});
});
@@ -0,0 +1,80 @@
import { describe, expect, test } from "bun:test";
import { runCLI } from "./helpers";
// All cases fail schema validation (or the required-flag guard) before any
// network request, so the suite is network-free. Regression context: a bare
// z.coerce.number() accepted --limit=-1, and slice(0, -1) then silently
// dropped the last result instead of erroring. Filter flags (--jobage) also
// accepted negative and fractional values that were sent raw to the portal.
function expectValidationError(result: { exitCode: number; stdout: string; stderr: string }, option: string) {
expect(result.exitCode).toBe(1);
expect(result.stdout).toBe("");
const error = JSON.parse(result.stderr);
expect(error.ok).toBe(false);
expect(error.error.kind).toBe("validation");
expect(error.error.option).toBe(option);
}
describe("Jobindex CLI flag validation", () => {
test("--limit=-1 is rejected instead of silently dropping the last result", async () => {
const result = await runCLI(["search", "--query", "test", "--limit=-1"]);
expectValidationError(result, "limit");
expect(JSON.parse(result.stderr).error.message).toContain("greater than or equal to 1");
});
test("--limit=0 is rejected", async () => {
const result = await runCLI(["search", "--query", "test", "--limit=0"]);
expectValidationError(result, "limit");
});
test("--limit=1.5 is rejected as non-integer", async () => {
const result = await runCLI(["search", "--query", "test", "--limit=1.5"]);
expectValidationError(result, "limit");
expect(JSON.parse(result.stderr).error.message).toContain("Expected integer");
});
test("--page=0 is rejected on the 1-indexed portal", async () => {
const result = await runCLI(["search", "--query", "test", "--page=0"]);
expectValidationError(result, "page");
});
test("--jobage=-5 is rejected", async () => {
const result = await runCLI(["search", "--query", "test", "--jobage=-5"]);
expectValidationError(result, "jobage");
expect(JSON.parse(result.stderr).error.message).toContain("greater than or equal to 1");
});
test("--jobage=1.5 is rejected as non-integer", async () => {
const result = await runCLI(["search", "--query", "test", "--jobage=1.5"]);
expectValidationError(result, "jobage");
expect(JSON.parse(result.stderr).error.message).toContain("Expected integer");
});
test("valid numeric flags pass schema validation (proven offline via the required-flag guard)", async () => {
const result = await runCLI(["search", "--page=2", "--limit=5"]);
expect(result.exitCode).toBe(1);
expect(JSON.parse(result.stderr)).toEqual({
error: "--query is required",
code: "MISSING_REQUIRED",
});
});
});
describe("unknown flag rejection", () => {
// add-portal.md's contract: "a bogus flag or missing required arg exits 1
// with a JSON error on stderr". A silently discarded flag is worse than an
// error: on jobdanmark a wrong flag name returned the entire database
// (13,862 results) as if it matched the query (review finding F13,
// 2026-08-19). Rejection happens before dispatch, so these are network-free.
test("a bogus --flag exits 1 with a JSON error instead of being silently discarded", async () => {
const result = await runCLI(["search", "--query", "test", "--bogus-flag", "xyz"]);
expect(result.exitCode).toBe(1);
expect(result.stdout).toBe("");
const error = JSON.parse(result.stderr);
expect(error.code).toBe("UNKNOWN_FLAG");
expect(error.error).toContain("--bogus-flag");
});
});
@@ -0,0 +1,152 @@
import { describe, expect, test } from "bun:test";
import { parseDetailPage } from "../src/commands/detail";
// Jobindex redesigned its detail pages; every selector the old parser used
// (job-text, jix-info, jix_robotjob--area, jix-toolbar-top__company) is gone
// from live pages, so detail returned null company/location/date, CSS-comment
// text as the deadline, and an external ATS URL as its own id/url - exit 0,
// nothing signalling breakage (review finding F14, 2026-08-19, measured on
// 5/5 live postings). These fixtures are trimmed from live pages captured
// 2026-08-19: the jobindex-native "jd-*" shape and the external-ATS
// (hr-manager/Talentech) passthrough shape.
const NATIVE_PAGE = `<!DOCTYPE html>
<html lang="da">
<head>
<title>VELLIV - Udvikler til Camunda/AWS</title>
<meta property="og:title" content="Udvikler til Camunda/AWS" />
<meta property="og:url" content="https://www.jobindex.dk/jobannonce/h1690934" />
<meta property="og:description" content="VELLIV" />
</head>
<body>
<div class="jd-appetizer">
<h1>Udvikler til Camunda/AWS</h1>
</div>
<div id="container" class="container">
<div class="row">
<div class="twelve columns jd-details">
<div class="jd-description">
<p class="appetizer">En virksomhed med mere end 100 &aring;rs historie, der samtidig er cloud-only, er ikke hverdagskost.</p>
<p>Hos Velliv f&aring;r du mulighed for at arbejde med Camunda, AWS og automatisering af processer.</p>
</div>
</div>
<div class="four columns jd-facts">
<div class="jd-type">
<h3>Jobtype:</h3>
<p>Fast</p>
</div>
<div class="jd-workhours">
<h3>Arbejdstid:</h3>
<p>Fuldtid</p>
</div>
<div class="jd-worktime">
<h3>Arbejdsdage:</h3>
<p>Dag</p>
</div>
<div class="jd-deadline">
<h3>Ans&oslash;gningsfrist:</h3>
<p>13. september 2026</p>
</div>
<div class="jd-location">
<h3>Arbejdssted:</h3>
<p>Ballerup</p>
</div>
</div>
</div>
</div>
</body>
</html>`;
// The external shape: an employer's ATS-hosted ad served through jobindex.
// og:url points at the ATS (NOT the posting), og:site_name is the ATS brand,
// and the only occurrence of the deadline label outside the widget is inside
// a CSS comment - the exact text the old regex captured as the deadline.
const EXTERNAL_PAGE = `<!DOCTYPE html>
<html>
<head>
<title>
Talentech - C#-udvikler til kritiske analysel&oslash;sninger i elnettet
</title>
<meta name="description" content="Vi s&oslash;ger en udvikler til elnettet" />
<meta itemprop="name" content="C#-udvikler til kritiske analysel&oslash;sninger i elnettet" />
<meta property="og:title" content="C#-udvikler til kritiske analysel&oslash;sninger i elnettet" />
<meta property="og:site_name" content="Talentech" />
<meta property="og:url" content="https://candidate.hr-manager.net/ApplicationInit.aspx?cid=316&amp;ProjectId=188792" />
<style>
/* Defines the style of the Application Due text */
/* DK: Ansøgningsfrist */
/* BOKSTAV: K */
.frist { padding-bottom: 15px; }
</style>
</head>
<body>
<h1>C#-udvikler til kritiske analysel&oslash;sninger i elnettet</h1>
<p>Vil du v&aelig;re med til at udvikle og drifte de systemer, der underst&oslash;tter udbygningen af Danmarks kommende elnet? Vi arbejder i krydsfeltet mellem IT og energi.</p>
<div class="workplacelist emptyparent"><div id="workplacelist_lang" class="rowheader">Workplace</div><div class="widget-line"></div><span class="empty">Fredericia</span><br></div>
<div class="frist emptyparent"><div id="frist_lang" class="rowheader">Application due</div><div class="widget-line"></div><span class="empty">21-09-2026</span><br></div>
</body>
</html>`;
const JOBANNONCE_URL = "https://www.jobindex.dk/jobannonce/";
describe("parseDetailPage - jobindex-native (jd-*) shape", () => {
const job = parseDetailPage(NATIVE_PAGE, `${JOBANNONCE_URL}h1690934`, "h1690934");
test("extracts the contract fields", () => {
expect(job).toMatchObject({
id: "h1690934",
title: "Udvikler til Camunda/AWS",
company: "VELLIV",
location: "Ballerup",
deadline: "2026-09-13",
url: `${JOBANNONCE_URL}h1690934`,
});
});
test("converts the Danish long date to ISO", () => {
expect(job.deadline).toBe("2026-09-13");
});
test("extracts employment metadata and the description body", () => {
expect(job.employmentType).toBe("Fast");
expect(job.hours).toBe("Fuldtid");
expect(job.description).toContain("Camunda, AWS og automatisering");
});
});
describe("parseDetailPage - external ATS passthrough shape", () => {
const job = parseDetailPage(EXTERNAL_PAGE, `${JOBANNONCE_URL}h1690445`, "h1690445");
test("keeps the jobindex id and jobannonce URL, never the ATS og:url", () => {
expect(job.id).toBe("h1690445");
expect(job.url).toBe(`${JOBANNONCE_URL}h1690445`);
expect(job.url).not.toContain("hr-manager.net");
});
test("extracts the title and never reports the ATS brand as the company", () => {
expect(job.title).toBe("C#-udvikler til kritiske analyseløsninger i elnettet");
expect(job.company).toBeNull();
});
test("extracts the workplace widget location", () => {
expect(job.location).toBe("Fredericia");
});
test("finds the real deadline, not the CSS comment", () => {
expect(job.deadline).toBe("2026-09-21");
});
test("description is readable body text, not a stylesheet", () => {
expect(job.description).toContain("udbygningen af Danmarks kommende elnet");
expect(job.description).not.toContain("padding-bottom");
});
test("deadline is null when only the CSS comment mentions the label", () => {
const noDueWidget = EXTERNAL_PAGE.replace(
/<div class="frist emptyparent">[\s\S]*?<br><\/div>/,
"",
);
const parsed = parseDetailPage(noDueWidget, `${JOBANNONCE_URL}h9`, "h9");
expect(parsed.deadline).toBeNull();
});
});
+39
View File
@@ -0,0 +1,39 @@
import { join } from "path";
const CLI_PATH = join(import.meta.dir, "../src/cli.ts");
export interface CLIResult {
stdout: string;
stderr: string;
exitCode: number;
}
export async function runCLI(args: string[]): Promise<CLIResult> {
const proc = Bun.spawn(["bun", "run", CLI_PATH, ...args], {
stdout: "pipe",
stderr: "pipe",
});
const [stdout, stderr, exitCode] = await Promise.all([
new Response(proc.stdout).text(),
new Response(proc.stderr).text(),
proc.exited,
]);
return { stdout: stdout.trim(), stderr: stderr.trim(), exitCode };
}
export function parseJSON<T = unknown>(result: CLIResult): T {
if (result.exitCode !== 0) {
throw new Error(
`CLI exited with code ${result.exitCode}. stderr: ${result.stderr}`
);
}
try {
return JSON.parse(result.stdout) as T;
} catch {
throw new Error(
`Failed to parse JSON. stdout: ${result.stdout}\nstderr: ${result.stderr}`
);
}
}
+98
View File
@@ -0,0 +1,98 @@
import { describe, test, expect } from "bun:test";
import { parseJobCards, extractDivContent } from "../src/helpers";
// Minimal jobad-wrapper markup: parseJobCards splits on `jobad-wrapper-<id>`
// and reads the title from the <h4><a href>…</a></h4> and the company from the
// jix-toolbar-top__company link. We inject HTML entities to exercise decoding.
function card(id: string, title: string, company = "Acme A/S"): string {
return `<div id="jobad-wrapper-${id}" class="PaidJob">
<h4><a href="https://www.jobindex.dk/jobannonce/${id}">${title}</a></h4>
<div class="jix-toolbar-top__company">
<a href="https://www.jobindex.dk/virksomhed/acme">${company}</a>
</div>
</div>`;
}
describe("decodeHtmlEntities (via parseJobCards)", () => {
test("decodes hexadecimal numeric entities (&#xF8; -> ø)", () => {
const [c] = parseJobCards(card("h1", "Sm&#xF8;rrebro Chef"));
expect(c.title).toBe("Smørrebro Chef");
});
test("decodes uppercase-X hexadecimal entities (&#XE6; -> æ)", () => {
const [c] = parseJobCards(card("h2", "K&#XE6;re Kollega"));
expect(c.title).toBe("Kære Kollega");
});
test("still decodes decimal numeric entities (&#229; -> å) — regression", () => {
const [c] = parseJobCards(card("h3", "&#229;rhus Lead"));
expect(c.title).toBe("århus Lead");
});
test("decodes supplementary-plane code points with fromCodePoint (&#128512;)", () => {
const [c] = parseJobCards(card("h4", "Growth &#128512;"));
expect(c.title).toBe("Growth 😀");
});
test("decodes hex supplementary-plane code points (&#x1F600;)", () => {
const [c] = parseJobCards(card("h5", "Growth &#x1F600;"));
expect(c.title).toBe("Growth 😀");
});
test("decodes hex entities in the company name too", () => {
const [c] = parseJobCards(card("h6", "Engineer", "N&#xF8;rrebro ApS"));
expect(c.company).toBe("Nørrebro ApS");
});
});
describe("extractDivContent", () => {
test("extracts content from simple div", () => {
const html = '<div class="job-text">Simple text</div>';
expect(extractDivContent(html, "job-text")).toBe("Simple text");
});
test("extracts content with nested divs — the regression case", () => {
const html = `<div class="job-text">
<div class="highlight">First section</div>
<div class="details">Second section with important info</div>
</div>`;
expect(extractDivContent(html, "job-text")).toBe(
'\n <div class="highlight">First section</div>\n <div class="details">Second section with important info</div>\n ',
);
});
test("returns null when class not found", () => {
expect(extractDivContent("<div>no class</div>", "nonexistent")).toBeNull();
});
test("works with extra attributes on the div", () => {
const html = '<div id="desc" class="job-text" data-x="1">Content</div>';
expect(extractDivContent(html, "job-text")).toBe("Content");
});
test("handles deeply nested divs (3 levels)", () => {
const html = `<div class="job-text">
<div>
<div>Deep content</div>
</div>
</div>`;
expect(extractDivContent(html, "job-text")).toBe(
'\n <div>\n <div>Deep content</div>\n </div>\n ',
);
});
test("handles empty content", () => {
const html = '<div class="job-text"></div>';
expect(extractDivContent(html, "job-text")).toBe("");
});
test("handles br and other non-div tags", () => {
const html = '<div class="job-text">Line1<br>Line2<br>Line3</div>';
expect(extractDivContent(html, "job-text")).toBe("Line1<br>Line2<br>Line3");
});
test("escapes special regex characters in class name", () => {
const html = '<div class="job-text (special)">Content</div>';
expect(extractDivContent(html, "job-text (special)")).toBe("Content");
});
});
@@ -0,0 +1,34 @@
import { afterEach, describe, expect, test } from "bun:test";
import { apiFetch, htmlFetch } from "../src/helpers";
// A stalled upstream connection (accepted socket, no response) would otherwise
// hang the CLI forever - fetch has no default timeout. Assert both request
// wrappers carry an AbortSignal timeout. Fails on the pre-fix code (no signal).
const originalFetch = globalThis.fetch;
afterEach(() => {
globalThis.fetch = originalFetch;
});
describe("request timeout", () => {
test("apiFetch passes an AbortSignal timeout to fetch", async () => {
let init: RequestInit | undefined;
globalThis.fetch = (async (_url: string | URL | Request, i?: RequestInit) => {
init = i;
return new Response("{}", { status: 200 });
}) as unknown as typeof fetch;
await apiFetch("/search");
expect(init?.signal).toBeInstanceOf(AbortSignal);
});
test("htmlFetch passes an AbortSignal timeout to fetch", async () => {
let init: RequestInit | undefined;
globalThis.fetch = (async (_url: string | URL | Request, i?: RequestInit) => {
init = i;
return new Response("<html></html>", { status: 200 });
}) as unknown as typeof fetch;
await htmlFetch("https://www.jobindex.dk/x");
expect(init?.signal).toBeInstanceOf(AbortSignal);
});
});
@@ -0,0 +1,89 @@
import { afterEach, describe, expect, test } from "bun:test";
import { apiFetch, htmlFetch } from "../src/helpers";
// The portal contract requires backoff on 429/5xx. These tests pin the retry
// loop offline: a stubbed fetch counts attempts, and a stubbed setTimeout
// fires immediately so the exhaustion case does not sleep through the real
// 500ms -> 5s backoff schedule. apiFetch and htmlFetch carry separate copies
// of the loop, so both are exercised to keep them from drifting apart.
const originalFetch = globalThis.fetch;
const originalSetTimeout = globalThis.setTimeout;
afterEach(() => {
globalThis.fetch = originalFetch;
globalThis.setTimeout = originalSetTimeout;
});
function instantTimers() {
globalThis.setTimeout = ((fn: () => void) =>
originalSetTimeout(fn, 0)) as unknown as typeof setTimeout;
}
function stubFetch(responses: Array<() => Response>): { calls: number } {
const state = { calls: 0 };
globalThis.fetch = (async () => {
const i = Math.min(state.calls, responses.length - 1);
state.calls++;
return responses[i]();
}) as unknown as typeof fetch;
return state;
}
describe("htmlFetch retry/backoff", () => {
test("retries a 429 and succeeds on the next attempt", async () => {
instantTimers();
const state = stubFetch([
() => new Response("", { status: 429 }),
() => new Response("<html>ok</html>", { status: 200 }),
]);
const html = await htmlFetch("https://www.jobindex.dk/x");
expect(html).toContain("ok");
expect(state.calls).toBe(2);
});
test("does not retry a plain 4xx", async () => {
const state = stubFetch([() => new Response("", { status: 400 })]);
await expect(htmlFetch("https://www.jobindex.dk/x")).rejects.toThrow(/400/);
expect(state.calls).toBe(1);
});
test("gives up after the initial attempt plus six retries on persistent 5xx", async () => {
instantTimers();
const state = stubFetch([() => new Response("", { status: 500 })]);
await expect(htmlFetch("https://www.jobindex.dk/x")).rejects.toThrow(/500/);
expect(state.calls).toBe(7);
});
});
describe("apiFetch retry/backoff", () => {
test("retries a 429 and succeeds on the next attempt", async () => {
instantTimers();
const state = stubFetch([
() => new Response("", { status: 429 }),
() => new Response('{"ok":true}', { status: 200 }),
]);
const data = await apiFetch<{ ok: boolean }>("/x");
expect(data.ok).toBe(true);
expect(state.calls).toBe(2);
});
test("does not retry a plain 4xx", async () => {
const state = stubFetch([() => new Response("", { status: 400 })]);
await expect(apiFetch("/x")).rejects.toThrow(/400/);
expect(state.calls).toBe(1);
});
test("gives up after the initial attempt plus six retries on persistent 5xx", async () => {
instantTimers();
const state = stubFetch([() => new Response("", { status: 500 })]);
await expect(apiFetch("/x")).rejects.toThrow(/500/);
expect(state.calls).toBe(7);
});
});
@@ -0,0 +1,69 @@
import { describe, expect, test } from "bun:test";
import { parseSearchPage } from "../src/helpers";
// parseSearchPage had no tests at all: mutating total to stop using
// sr.hitcount survived the whole suite (review finding F35, 2026-08-19).
// The fixture mirrors the real Stash nesting documented in helpers.ts:
// jobsearch/result_app -> storeData -> searchResponse -> { hitcount, results[] }.
function stashPage(searchResponse: object): string {
const stash = { jobsearch: { result_app: { storeData: { searchResponse } } } };
return `<html><head><script>var Stash = ${JSON.stringify(stash)};</script></head></html>`;
}
const RESULT = {
tid: "h1689961",
headline: "Softwareudvikler",
company: { name: "Acme A/S", homeurl: "https://acme.example" },
area: "Aarhus",
firstdate: "2026-08-10",
apply_deadline: "2026-09-11T00:00:00",
};
describe("parseSearchPage", () => {
test("total comes from hitcount, not the page's result count", () => {
const page = stashPage({ hitcount: 435, results: [RESULT] });
const parsed = parseSearchPage(page);
expect(parsed.total).toBe(435);
expect(parsed.results).toHaveLength(1);
});
test("total falls back to the result count when hitcount is absent", () => {
const page = stashPage({ results: [RESULT, { ...RESULT, tid: "h2" }] });
expect(parseSearchPage(page).total).toBe(2);
});
test("maps the contract fields from a Stash result", () => {
const [job] = parseSearchPage(stashPage({ hitcount: 1, results: [RESULT] })).results;
expect(job).toMatchObject({
id: "h1689961",
title: "Softwareudvikler",
company: "Acme A/S",
location: "Aarhus",
date: "2026-08-10",
deadline: "2026-09-11",
url: "https://www.jobindex.dk/jobannonce/h1689961",
});
});
test("maps an ASAP posting's deadline to null (no stated deadline)", () => {
// apply_deadline_asap means "no fixed deadline, apply now". The /scrape
// schema defines null as exactly that, and every consumer (rank's expiry
// sweep, notion-sync's typed date column) does date arithmetic on this
// field - a bare "ASAP" string broke all of them on half of live results
// (review finding F12, 2026-08-19). lastdate present too: the flag wins.
const [job] = parseSearchPage(
stashPage({
hitcount: 1,
results: [{ ...RESULT, apply_deadline: undefined, apply_deadline_asap: true, lastdate: "2026-09-30" }],
}),
).results;
expect(job.deadline).toBeNull();
});
test("falls back to lastdate when apply_deadline is absent", () => {
const [job] = parseSearchPage(
stashPage({ hitcount: 1, results: [{ ...RESULT, apply_deadline: undefined, lastdate: "2026-09-30" }] }),
).results;
expect(job.deadline).toBe("2026-09-30");
});
});
+20
View File
@@ -0,0 +1,20 @@
{
"compilerOptions": {
"target": "ESNext",
"module": "ESNext",
"moduleResolution": "bundler",
"lib": ["ESNext"],
"types": ["bun-types"],
"strict": true,
"noImplicitAny": true,
"strictNullChecks": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"noFallthroughCasesInSwitch": true,
"skipLibCheck": true,
"outDir": "dist",
"rootDir": "."
},
"include": ["src/**/*", "tests/**/*"],
"exclude": ["node_modules", "dist"]
}