fix(jobdanmark-search): back off on 429/5xx in detail instead of failing on the first attempt (#460)

`detail` called fetch() directly rather than going through the CLI's own
request wrappers, so it had none of the three things apiFetch/apiPost
guarantee: no 429/5xx retry loop, a hand-inlined User-Agent that would
drift from the exported USER_AGENT, and a timeout no wrapper test covered.
A rate-limited detail page wrote API_ERROR and exited after ONE attempt;
jobnet, jobbank, jobindex, linkedin, and freehire all retry up to six
times on the same response. /scrape calls detail once per shortlisted
posting, so a burst that tripped jobdanmark's limiter dropped those
postings (no description, no deadline) while any other portal rode it out.

Demonstrated by driving the real command handler with a stubbed 429 and
instant timers: 1 fetch attempt and exit 1 before, 7 after (initial try
plus six retries, the contract's schedule).

Add htmlFetch to helpers.ts with the same backoff, timeout, and shared
User-Agent as the JSON wrappers - 404 returns null so detail keeps its
NOT_FOUND contract - and route detail through it. The retry-backoff,
user-agent, and request-timeout suites now cover all three wrappers, and
the new detail-backoff.test.ts exercises the handler path itself; its two
retry cases fail against the bare fetch().
This commit is contained in:
Ayobami Adegoke
2026-09-14 18:25:30 +02:00
committed by GitHub
parent c7bd494f11
commit c2cd71ddee
7 changed files with 238 additions and 21 deletions
@@ -1,11 +1,13 @@
import { afterEach, describe, expect, test } from "bun:test";
import { apiFetch, apiPost } from "../src/helpers";
import { apiFetch, apiPost, htmlFetch } from "../src/helpers";
// The portal contract requires backoff on 429/5xx. These tests pin the retry
// loop offline: a stubbed fetch counts attempts, and a stubbed setTimeout
// fires immediately so the exhaustion case does not sleep through the real
// 500ms -> 5s backoff schedule. apiFetch and apiPost carry separate copies of
// the loop, so both are exercised to keep them from drifting apart.
// 500ms -> 5s backoff schedule. apiFetch, apiPost, and htmlFetch carry separate
// copies of the loop, so all three are exercised to keep them from drifting
// apart. htmlFetch is the one `detail` uses: before it existed, detail called
// fetch() directly and a 429 failed on the first attempt (1 call, not 7).
const originalFetch = globalThis.fetch;
const originalSetTimeout = globalThis.setTimeout;
@@ -33,6 +35,7 @@ function stubFetch(responses: Array<() => Response>): { calls: number } {
const wrappers: Array<[string, () => Promise<{ ok: boolean }>]> = [
["apiFetch", () => apiFetch<{ ok: boolean }>("/x")],
["apiPost", () => apiPost<{ ok: boolean }>("/x", {})],
["htmlFetch", () => htmlFetch("https://jobdanmark.dk/job/x").then((html) => ({ ok: html !== null }))],
];
for (const [name, call] of wrappers) {
@@ -65,3 +68,12 @@ for (const [name, call] of wrappers) {
});
});
}
describe("htmlFetch 404", () => {
test("returns null without retrying so detail keeps its NOT_FOUND contract", async () => {
const state = stubFetch([() => new Response("", { status: 404 })]);
expect(await htmlFetch("https://jobdanmark.dk/job/missing")).toBeNull();
expect(state.calls).toBe(1);
});
});