* freehire-search: honor Retry-After on 429/503
apiGet retried a 429 on its own ~24s schedule and ignored Retry-After,
while freehire's limits are per minute and every refusal states how long
to wait, so a rate-limited run burned all six retries inside the window
it had just been refused for. Wait max(backoff, Retry-After), capped at 60s.
* freehire-search: bound the total retry wait; fail at once past it
* changelog: place the #632 entry under [Unreleased]
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Mads Lorentzen <madslorentzen_17@hotmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(freehire-search): reject --description-format on detail instead of ignoring it
The flag was on detail's known-flag list, but runDetail never reads it and
always prints the detail endpoint's HTML body stripped to plain text, so
detail <slug> --description-format markdown passed the unknown-flag guard,
exited 0 and printed text. Drop it from detail's list so it fails with
UNKNOWN_FLAG before any request; search keeps the flag.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* changelog: place the #648 entry under [Unreleased]
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Mads Lorentzen <madslorentzen_17@hotmail.com>
* fix(freehire-search): reject detail URLs on other hosts instead of asking the API for their last segment
normalizeSlug took the segment after "/jobs/" from ANY string, so a
Greenhouse apply link - which is what a freehire search result's own url
field carries - was reduced to its last path segment and detail
requested /api/v1/jobs/1, then reported NOT_FOUND for a URL that was
never a freehire posting. Every other portal CLI rejects an off-host
detail URL with BAD_ID (#447, #430, #499); freehire was the last one
trusting the path.
Parse URLs for real: freehire.me (apex or any subdomain, scheme optional)
or the host of FREEHIRE_API_URL for a self-hosted instance, plus a
/jobs/<slug> path, yields the slug; other hosts, look-alike and userinfo
hosts, and non-job freehire pages exit 1 with BAD_ID before any request.
Bare slugs are unchanged.
Five new normalizeSlug cases and a handler-level test that pastes the
search fixture's own apply URL into detail with fetch stubbed; the
off-host cases fail on master.
* changelog: place the #623 entry under [Unreleased]
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Mads Lorentzen <madslorentzen_17@hotmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
semantic_ratio and include_description are both reported in
meta.ignored_params. With include_description ignored, the agent endpoint
hydrates every full body from the database, and --no-description threw
that payload away client-side. Send --no-description to /api/v1/jobs/search,
which serves only the preview, and stop sending the two unread params.
The old value linked to freehire.me, which named the API rather than the
caller, so the API's operators could not reach this project before a
limit or field change. Use the owner/project form freehire's API docs ask
for, with this repo as the contact URL.
toResult passed posted_at through verbatim, so search and detail JSON
carried "date": "2026-07-06T00:00:00Z" while every other portal CLI
emits the date part only and /scrape stores it as
posted_date: "YYYY-MM-DD". The table and plain renderers already cut the
value to ten characters, so only JSON consumers - the /scrape path - saw
the raw timestamp.
Add contractDate(): the timestamp's date part, null for an absent or
non-date value. The two tests that pinned the raw timestamp now expect
the date and fail on master, plus three cases on the helper. SKILL.md
documents the shape.
* fix(jobnet-search): preserve full occupation identifiers in tables
The documented ESCO concept URI is 72 characters long, but the table renderer cut every identifier to 70, losing the last two UUID characters. Size the identifier column from its contents, retaining the existing minimum width and readable label alignment.
Exercise the actual documented occupations CLI with the README API response served over loopback. Table output must retain the same complete URI and UUID as JSON and plain output; no live portal requests.
* test(jobnet-search): cover Windows occupation fixture newlines
* docs(jobindex-search): align search examples with the actual CLI contract
The posted-today skill example omits the query that the shipped CLI requires, so copying it exits MISSING_REQUIRED. Supply a keyword and make the required flag explicit. Replace stale JSON-endpoint/card-parser search documentation with the implemented HTML Stash route and emitted field behavior.
Run the example read directly from SKILL.md through a real CLI subprocess with loopback HTTP fixtures. Keep a separate query/page routing control; no live portal requests or production-code changes.
* test(jobindex-search): cover Windows documentation newlines
* fix(portal-clis): reject missing Danish option values before dispatch
The Bunli parser discards known bare options before schema validation, silently dropping filters or applying defaults. Extend the existing pre-dispatch guards across the four Danish portals while preserving built-in switches, valued and equals-form arguments, and existing unknown-flag errors.
Actual CLI subprocess regressions deny and record every fetch to demonstrate that missing filters reach request construction on master. Valued-option and help controls keep accepted command forms covered without contacting any portal.
* docs(changelog): record fix for #573
* test(portal-clis): isolate valid flag controls from live services
The LinkedIn and Freehire positive controls spawned live search CLIs and only checked for the absence of BAD_ARG, so attempted portal requests still left a green suite. Use loopback responses and assert successful output plus the constructed freshness, pagination and result-limit contract.
LinkedIn cards are verbatim excerpts from portapulito's actual captured response in PR #153; fixture provenance and original hashes are retained. Four Danish validation suites already stop at required-argument guards and need no change. This isolates these controls rather than claiming a whole-CI egress policy.
* docs(changelog): record fix for #581
* fix(linkedin-search): preserve description paragraph and list breaks
Clean rendered description lines separately so whitespace normalization does not erase paragraph and list boundaries in JSON and plain output. Exercise the documented detail CLI against a loopback-served captured response, with field and closed-status controls.
Credits: portapulito diagnosed this behavior and supplied the captured guest-response fixture in upstream PR #153, which was closed as an accidental personalized-fork submission. This independent implementation revives only that correctness fix and its fixture evidence.
* docs(changelog): record fix for #567
* fix(jobbank-search): retain all documented repeated filter values
Declare the six array-capable search options repeatable in Bunli metadata. Without it, the CLI parser overwrites earlier occurrences before the handler receives the array, so documented OR filters only search the final value.
Exercise actual search CLI subprocesses through loopback HTTP and assert every RSS and total-count request retains repeated values. Cover all six filters, mixed equals/comma input, and unchanged single-value input; no live portal requests.
* docs(changelog): record fix for #577
External ads (isExternal: true) 404 on /find-job/{jobAdId} and hit the MitID login wall on /job/{jobAdId}. Route them to their employer-provided destination in jobAdUrl when an absolute http(s) URL is present.
When jobAdUrl is missing, empty, or carries an invalid/non-http scheme, fall back to https://jobnet.dk/find-job/{jobAdId} instead of emitting an empty string so downstream /scrape deduplication preserves unique URLs. Both url and jobAdUrl validate the scheme consistently.
Observation and test cases reported by NishilRathod in #557.
A live page wrote its Jobtype item as "Fuldtid </ li >". node-html-parser
consumes well-formed markup but does not recognise a closing tag with a
space after the "<" or the slash, so the fragment became a text node and
overviewValue kept it: employmentType: ["Fuldtid </ li >"], matching none
of the documented job-type vocabulary. The same cleanText feeds every
text field on both branches.
Strip only the closing-tag shape in cleanText before collapsing
whitespace. The parser decodes entities before cleanText runs, so author
text the page encoded as <div> or <T> arrives as a literal
<div> / <T> and must stay; a probe of what the parser leaks shows only
malformed closing shapes (</ li >, </ li>, < /li>). Two fixtures: the
stray tag in both the Jobtype item and a description paragraph (fails on
master), and an entity-encoded <div> that survives.
Observed by the maintainer in the #516 merge note.
/add-portal's contract says every search result carries at least id,
title, company, location, date, url, and the four other shipped CLIs
emit id. When #340 added the company/location/date/url aliases to these
two, id was left out: jobnet exposed the value only as jobAdId,
jobdanmark only as slug, so a consumer reading every portal's JSON
through one shape had to special-case both to call detail.
Purely additive: id equals jobAdId on jobnet and slug on jobdanmark, the
native keys stay, nothing is renamed. Each SKILL.md notes the alias. The
additive-contract test in each CLI now asserts id and its equality with
the native key; both fail on master.
The JSON-LD branch of detail emitted jobPosting.description verbatim, so
`detail --format plain` printed
`description: <p>Vi søger en udvikler til R&D.</p><ul><li>Python & Go</li>...`
while the rendered-HTML branch of the same command printed one clean
line per paragraph and bullet. Two description shapes from one command
depending on which page layout it hit, and the markup landed in
/scrape's stored snippet and /rank's agent context as-is. The portal
contract (/add-portal Step 4) asks for readable text: entities decoded,
tags stripped, paragraph breaks preserved.
Render the HTML through node-html-parser's structuredText - one line per
block element, entities decoded, tags gone - which is the shape the
fallback already produces. A plain-text description is unchanged, an
absent one stays "".
The JSON-LD test that pinned the raw <p>...</p> now expects the text,
plus two new cases; the HTML case fails on master.
Titles are indexed as the source ATS wrote them and toResult passed them
through untouched, so `detail intern-fullstack-amp-ai-innovation-noise-edaawqq2`
printed `Intern - Fullstack & AI Innovation ` and the same title reached
`search` output. In a 93-result /scrape batch, 1 title carried `&` and
3 ended in whitespace; /scrape Step 4.75 reads an undecoded entity in a
title as a half-working parser.
Run the title through the existing decodeHtmlEntities (already used for
detail descriptions) and trim it; an empty or whitespace-only title still
falls back to "(untitled)". Three new parsing.test.ts cases pin it; all
three fail on the old pass-through.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
formatSalary printed only the currency and amounts, although freehire's
enrichment records the pay period. `detail software-developer-trainee-codifi-fra635ux`
printed `INR 300000–300000` while the API returns `salary_period: "year"`
for that posting, so a yearly salary screened against a monthly figure
read as twelve times its real monthly value.
Append the period when freehire records one (`INR 300000–300000/year`,
`INR 12000/month`); a record without a period prints exactly as before.
Two new parsing.test.ts cases pin it; both fail on the old formatter.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
normalizeId took the first 6+-digit path segment from ANY URL, so a
Greenhouse or Lever apply link - the kind a posting's own page hands out
and a user pastes back into detail - was reduced to that number and the
handler fetched jobPosting/<number> from LinkedIn: whatever job carried
that id came back with exit 0, or NOT_FOUND if none did, never an error
about the input. Driving the real handler with a stubbed fetch,
https://boards.greenhouse.io/acme/jobs/4567890 requested
.../jobPosting/4567890. Every other portal CLI rejects an off-host detail
URL with BAD_ID (the #447 shape); linkedin was the one still trusting the
digits.
Parse URLs for real: a linkedin.com host (apex or any subdomain, scheme
optional) plus a /jobs/view/<slug->id path yields the id; other hosts,
look-alike and userinfo hosts, and linkedin.com profile or search URLs
exit 1 with the stderr-JSON BAD_ID contract before any request. Bare
ids, URNs, and slash-free title slugs are unchanged.
Four new normalizeId cases and a new detail-input.test.ts that drives
runDetail and the CLI with fetch stubbed; five cases fail on the old
pattern.
`detail` called fetch() directly rather than going through the CLI's own
request wrappers, so it had none of the three things apiFetch/apiPost
guarantee: no 429/5xx retry loop, a hand-inlined User-Agent that would
drift from the exported USER_AGENT, and a timeout no wrapper test covered.
A rate-limited detail page wrote API_ERROR and exited after ONE attempt;
jobnet, jobbank, jobindex, linkedin, and freehire all retry up to six
times on the same response. /scrape calls detail once per shortlisted
posting, so a burst that tripped jobdanmark's limiter dropped those
postings (no description, no deadline) while any other portal rode it out.
Demonstrated by driving the real command handler with a stubbed 429 and
instant timers: 1 fetch attempt and exit 1 before, 7 after (initial try
plus six retries, the contract's schedule).
Add htmlFetch to helpers.ts with the same backoff, timeout, and shared
User-Agent as the JSON wrappers - 404 returns null so detail keeps its
NOT_FOUND contract - and route detail through it. The retry-backoff,
user-agent, and request-timeout suites now cover all three wrappers, and
the new detail-backoff.test.ts exercises the handler path itself; its two
retry cases fail against the bare fetch().
detail fetched any http(s) input verbatim with no host check and, when
the path didn't match, silently used the whole input URL as the job id -
a non-posting page came back as a well-formed fake posting with exit 0.
buildUrl now requires a jobindex.dk host and a /jobannonce/<id> path,
rebuilds the fetch URL from the extracted id, and exits 1 with the
stderr-JSON BAD_ID contract otherwise; bare ids stay permissive
slash-free tokens per the jobnet precedent. Eight cases in the new
detail-input.test.ts; the five rejection/canonicalization cases fail
against the verbatim unguarded extraction.
The guard in the four bunli-based CLIs inspected only tokens starting
with `--`, so an undefined short flag bypassed it: bunli discarded it,
the search ran unfiltered, and the CLI exited 0. Live against jobnet,
`search -q "sygeplejerske"` returned all 18,179 ads as a successful
search against 667 for the real `--search-string` query - the same shape
as review finding F13 that motivated the guard.
Both dash forms are now checked. Declared shorts (jobindex's -q) and
bunli's built-in -h/-v stay valid. A negative number is rejected too:
bunli discards a `-`-prefixed token rather than consuming it as the
previous flag's value, so `--radius -5` silently fell back to the
default instead of failing its own min(1) schema; a value that must
begin with a dash uses the `--flag=value` form.
Fixes#426.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
The filter derefed item.text.toLowerCase() from a cast API response on
the same line that already guards g.items ?? [], so one item with a null
or missing text threw TypeError and the whole command exited 1 as
API_ERROR. The filter is extracted into an exported
filterAutocompleteGroups (the jobnet testability pattern), text is typed
nullable so the compiler enforces the guard, and an item without usable
text is skipped: it can never match the required non-empty query, so
downstream output never sees one. The null-text case was verified to
fail against the verbatim unguarded extraction with the production
TypeError. Closes out the #416/#418 audit.
date: job.publicationDate.slice(0, 10) trusted a TypeScript interface
claim nothing validates at runtime: apiFetch casts the JSON body, so one
ad with a null publication date threw TypeError inside the jobAds map
and the whole search exited 1 as API_ERROR. The neighboring
applicationDeadline field was already null-guarded with a 1900-01-01
sentinel. publicationDate is now typed nullable so the compiler enforces
the guard, and the ad degrades per-item to date: null. New test verified
to fail on the unfixed code with the exact production TypeError.
new Date(<unparseable>) yields an Invalid Date whose toISOString()
throws RangeError, and normalizeSearchItem runs inside an unguarded
items.map(), so one malformed RSS item killed the entire search with
{"error": "Invalid Date", "code": "API_ERROR"} and exit 1. The
un-CDATA'd fallback capture in parseRssItems can deliver exactly such a
value. An unparseable pubDate now degrades to the same shape as an
absent one (posted "", date null); every other item survives. Three
new cases pin the malformed shapes, each failing on the unfixed code.
parseInt truncated values before validation, so --jobage 0.5 became 0 and silently omitted LinkedIn's freshness filter. Require whole numbers of at least 1 for every numeric search flag and guard the behavior with CLI regression tests.
* feat(linkedin-search): add active status verification for job postings
* fix(linkedin-search): scope closed-posting detection to the top card, pin with tests (#280)
The first version matched five markers against the whole document, so
recruiter boilerplate quoting 'no longer accepting applications' in a
description flagged a live job CLOSED. Detection now stops where the
description markup begins and matches only the two markers real closed
pages carry (closed-job__flavor and the banner text, verified against
live guest pages); the three speculative phrases are dropped. Four new
fixture tests pin both directions plus the two description false-positive
cases - the false-positive pair fails on the unscoped version.
* feat(scrape): mark closed-at-source LinkedIn postings expired, never drop (#280)
/scrape Step 2 now consumes linkedin-search detail's isActive: a job whose
posting page renders the closed banner is written to seen_jobs.json with
status expired rather than silently dropped, per the /rank marking pattern -
the fix for the ghost-jobs class in #331. isActive: true is documented as
absence of the banner, not proof the posting is open.
---------
Co-authored-by: Navakanth Reddy Dumpa <navkanthr@gmail.com>
parseIntFlag used bare parseInt, so --jobage 0.5 truncated to 0, failed
the jobage > 0 guard in search.ts, and posted_within_days was silently
omitted from the outbound request while the CLI exited 0. Numeric flags
now accept whole numbers >= 1 only, mirroring the Danish CLIs'
z.coerce.number().int().min(1) contract, and reject everything else with
the stderr-JSON BAD_ARG error. Five new validation cases, each verified
to fail on the unfixed code.
A default search hydrates full bodies - ~73% of the payload, ~20k tokens
per query fed into agent context - while /scrape's own Step 2 says to
pre-filter by title before reading bodies. The flag keeps every other
field and drops the bodies (live 10-result search: ~58k -> ~10k chars);
hydration stays the default per the documented trade-off. The API
returns bodies regardless of include_description=false (verified live),
so the lean guarantee is enforced client-side. Review opportunity O1
(2026-08-19), approved as an enhancement.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every selector the old parser used (job-text, jix-info,
jix_robotjob--area, jix-toolbar-top__company) is gone from live pages:
detail returned CSS-comment text as the deadline, the external ATS URL
as its own id/url, null company/location/date, and a teaser description
- exit 0, on 4/5 live postings. The rewrite recognises both current
shapes (jobindex-native jd-* layout; external-ATS passthrough), always
keeps the jobindex id and jobannonce URL, anchors the deadline to a real
date in visible markup only (the label also lives in a CSS comment,
which is what the old regex captured), converts Danish long dates to
ISO, decodes Danish named entities, and returns company: null on
passthrough pages instead of the ATS brand. Live: 5/5 postings now
yield full descriptions and correct fields. Review finding F14
(2026-08-19).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The JSON-LD and rendered-HTML branches returned structurally different
records: the fallback emitted raw DD-MM-YYYY page text as datePosted,
free text (including the literal "Loebende") as validThrough, and a
hardcoded null addressLocality. Overview dates now convert to ISO,
Loebende maps to null, and the locality derives from the workplace
address via the same exported extractCity search uses. Review finding
F25 (2026-08-19).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
search guards the API's undisclosed-deadline sentinel and a test pins
it; detail dumped the raw response, so the same field for the same job
behaved two ways, and an undisclosed deadline stored via detail read as
126 years expired - /rank's sweep would retire the job on sight. All
three output formats now flow through prepareDetail. Review finding F33
(2026-08-19).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Silently discarded flags produced silently wrong results: jobdanmark
with --query (its real flag is --text) returned all 13,862 jobs as if
they matched, exit 0, empty stderr - indistinguishable from a real
result set. The four bunli CLIs get an argv preflight built from each
command's own options object; linkedin and freehire validate parsed
flags against per-command known sets. help/version still pass, and
add-portal.md's existing bogus-flag-exits-1 contract now holds for the
reference implementations contributors copy. One linkedin pin updated:
"--jobage-minutes -5" now fails as UNKNOWN_FLAG (the stray -5 token)
rather than BAD_ARG - same loud-failure invariant, earlier gate. Review
finding F13 (2026-08-19), decision approved by Mads.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The regex required class= before href= within one tag; LinkedIn's real
markup puts href first, so applyUrl was null on every live posting while
SKILL.md claimed the command returns an apply link. Fixing the regex
would only yield the job-view URL - a duplicate of url - so the field is
removed rather than repaired, and a test pins the removal. Review
finding F19 (2026-08-19), decision approved by Mads.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
coverImage, companyLogo, companyLogoSvgMarkup, overlayColor, and
silhouetteLogo were ~40% of a live payload - image keys, focal points
and overlay colours an agent can never act on, paid into context on
every /scrape query. A live 30-result response drops from ~30k to ~20k
chars. The #340 compatibility duplicates and slug (the detail command's
input) are kept deliberately. Review finding F3 (2026-08-19), decision
approved by Mads.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The feed's DD.MM.YYYY parenthetical passed through raw - documented, but
contradicting the /scrape contract, every other portal, and this CLI's
own detail command for the same job, and ambiguous to a date parser
(01.09.2026: 1 Sep or 9 Jan). The known shape converts to ISO; løbende
still maps to null; unrecognized shapes pass through for /rank's
defensive handling. Breaking for anything parsing the old format - the
README's own search example already showed ISO. Review finding F5
(2026-08-19), decision approved by Mads.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
apply_deadline_asap was emitted as the string "ASAP" on ~half of live
results - undocumented, contradicting the CLI's own README, and breaking
every consumer that does date arithmetic on the field (rank's sweep,
outcome's deadline check, notion-sync's typed date column). ASAP means
"no stated deadline", which the schema already defines null to mean.
The flag wins over any date field that happens to be present. Review
finding F12 (2026-08-19), decision approved by Mads.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The linkedin fixture was purpose-built for entity decoding and had no
<time> or location element, so removing the date extraction - a /scrape
contract field on a default-ON portal - survived the suite. jobindex's
parseSearchPage (the Stash parser behind every search) had zero tests,
so meta.total silently dropping hitcount survived too. Both mutations
now fail exactly the new tests. The ASAP deadline branch is deliberately
left to the F12 fix, which changes its behaviour to null. Review finding
F35 (2026-08-19).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The location regex required whitespace after the 4-digit postcode, but
live companyAddress values frequently read "2670, Greve" - those results
emitted location: null (7/30 in the review's live sample; 1/30 after this
fix), leaving /scrape's geography filter nothing to act on. Extraction is
now a helper with a comma fallback that requires a non-digit city start,
so a 4-digit street number never wins over the real postcode, and the
captured city is trimmed. Review finding F2 (2026-08-19).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
jobbank search output was missing the shared contract's date field; it now emits date as YYYY-MM-DD derived from posted (kept unchanged), null when the feed item has no pubDate. The inline result mapping is extracted into an exported normalizeSearchItem (behavior-preserving) so the derivation is pinned by tests.
Co-authored-by: oscarbol09 <80536682+oscarbol09@users.noreply.github.com>
Adds additive normalization so jobdanmark search output carries the cross-portal contract fields: company from companyName, location as the city after the postal code in companyAddress (null-safe - a missing or null address yields null instead of crashing the search), and date/deadline converted from DD-MM-YYYY to YYYY-MM-DD with safe passthrough on unexpected formats. Native fields unchanged.
Co-authored-by: oscarbol09 <80536682+oscarbol09@users.noreply.github.com>
Adds additive normalization so jobnet search output carries the cross-portal contract fields (company, location, date, deadline with the 1900-01-01 NotDisclosed sentinel mapped to null, and url). Emits the public /find-job/{jobAdId} route and corrects the skill's own stale /job/ documentation, which redirects anonymous visitors into the MitID login flow.
Co-authored-by: oscarbol09 <80536682+oscarbol09@users.noreply.github.com>
The last portal CLI still sending a full Chrome spoof after #283 and
4551346. Live-verified: search and detail endpoints serve identical
responses to Mozilla/5.0 (compatible; linkedin-search-cli/1.0).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
