freehire-search: lean search uses the web endpoint; drop unread params (#631)

semantic_ratio and include_description are both reported in
meta.ignored_params. With include_description ignored, the agent endpoint
hydrates every full body from the database, and --no-description threw
that payload away client-side. Send --no-description to /api/v1/jobs/search,
which serves only the preview, and stop sending the two unread params.
This commit is contained in:
Ilya Strelov authored and GitHub committed 2026-10-18 21:38:40 +02:00
1 parent ce9b656829
commit 0f6480d32e
6 files changed
+31 -27

No files matched your search

+2 -1
View File
@@ -196,6 +196,7 @@ dictionaries never guess). So:
live values (with counts) for a query before filtering. live values (with counts) for a query before filtering.
- The API retries 429/5xx with exponential backoff; an unreachable API exits - The API retries 429/5xx with exponential backoff; an unreachable API exits
non-zero with a clear message (best-effort service, see the dependency note above). non-zero with a clear message (best-effort service, see the dependency note above).
- `search` calls `/api/v1/agent/jobs/search` (public, like the rest). A self-hosted - `search` calls `/api/v1/agent/jobs/search` (public, like the rest); `--no-description`
calls the lighter `/api/v1/jobs/search`, which serves no full bodies. A self-hosted
instance older than that endpoint answers 404, and the CLI reports it as an error instance older than that endpoint answers 404, and the CLI reports it as an error
naming the endpoint — never as an empty result set. naming the endpoint — never as an empty result set.
+1 -1
View File
@@ -3,7 +3,7 @@
CLI for searching the [freehire.me](https://freehire.me) job aggregator across CLI for searching the [freehire.me](https://freehire.me) job aggregator across
**many markets** (tech-focused), via its public JSON API. **many markets** (tech-focused), via its public JSON API.
**Data source**: freehire.me REST API (`/api/v1/agent/jobs/search`, `/api/v1/jobs/facets`, `/api/v1/jobs/{slug}`). **Data source**: freehire.me REST API (`/api/v1/agent/jobs/search`, `/api/v1/jobs/search` for `--no-description`, `/api/v1/jobs/facets`, `/api/v1/jobs/{slug}`).
**Authentication**: None required — reads are public (only tracking mutations need a key, and those are out of scope here). **Authentication**: None required — reads are public (only tracking mutations need a key, and those are out of scope here).
**Dependencies**: None (plain `bun` + `fetch`). `bun install` is optional and only pulls dev type defs. **Dependencies**: None (plain `bun` + `fetch`). `bun install` is optional and only pulls dev type defs.
@@ -6,6 +6,12 @@ import { apiGet, toResult, writeError, type FreehireJob, type JobResult } from "
// follow-up `detail` per hit. // follow-up `detail` per hit.
const SEARCH_PATH = "/api/v1/agent/jobs/search" const SEARCH_PATH = "/api/v1/agent/jobs/search"
// The web search: identical query surface, but each hit carries only the index's
// truncated preview. --no-description uses it, because the agent endpoint reads
// every full body from the database whether or not the caller wants them - the
// lean mode would otherwise pay the expensive request and discard its payload.
const LEAN_SEARCH_PATH = "/api/v1/jobs/search"
/** How the API renders each result's full description. */ /** How the API renders each result's full description. */
export type DescriptionFormat = "markdown" | "text" | "html" export type DescriptionFormat = "markdown" | "text" | "html"
@@ -40,13 +46,7 @@ function buildQuery(opts: SearchOpts): URLSearchParams {
if (opts.query) p.set("q", opts.query) if (opts.query) p.set("q", opts.query)
p.set("limit", String(opts.limit)) p.set("limit", String(opts.limit))
p.set("offset", String((opts.page - 1) * opts.limit)) p.set("offset", String((opts.page - 1) * opts.limit))
p.set("semantic_ratio", "0") // keyword search; the semantic index is opt-in if (opts.includeDescription !== false) p.set("description_format", opts.descriptionFormat)
// The agent endpoint serves the index's truncated preview unless asked to
// rehydrate each hit from the database, so both params travel together -
// unless the caller opted out of hydration entirely (--no-description).
const hydrate = opts.includeDescription !== false
p.set("include_description", hydrate ? "true" : "false")
if (hydrate) p.set("description_format", opts.descriptionFormat)
if (opts.jobage > 0 && opts.jobage < 9999) p.set("posted_within_days", String(opts.jobage)) if (opts.jobage > 0 && opts.jobage < 9999) p.set("posted_within_days", String(opts.jobage))
if (opts.workMode) p.set("work_mode", opts.workMode) if (opts.workMode) p.set("work_mode", opts.workMode)
if (opts.company) p.set("company_slug", opts.company) if (opts.company) p.set("company_slug", opts.company)
@@ -112,22 +112,22 @@ function renderPlain(rows: JobResult[]): string {
export async function runSearch(opts: SearchOpts): Promise<number> { export async function runSearch(opts: SearchOpts): Promise<number> {
try { try {
const env = await apiGet<FreehireJob[]>(`${SEARCH_PATH}?${buildQuery(opts).toString()}`) const path = opts.includeDescription === false ? LEAN_SEARCH_PATH : SEARCH_PATH
const env = await apiGet<FreehireJob[]>(`${path}?${buildQuery(opts).toString()}`)
// A 404 here is a missing endpoint, not a missing job: a freehire instance // A 404 here is a missing endpoint, not a missing job: a freehire instance
// older than the agent search surface answers that way, and reporting it as // older than the agent search surface answers that way, and reporting it as
// an empty result set would hide the misconfiguration behind plausible output. // an empty result set would hide the misconfiguration behind plausible output.
if (!env) { if (!env) {
writeError( writeError(
`${SEARCH_PATH} not found — this freehire instance predates the agent search endpoint; upgrade it or unset FREEHIRE_API_URL to use the hosted API`, `${path} not found — this freehire instance predates the search endpoint; upgrade it or unset FREEHIRE_API_URL to use the hosted API`,
"SEARCH_FAILED", "SEARCH_FAILED",
) )
return 1 return 1
} }
let rows = (env.data ?? []).map(toResult) let rows = (env.data ?? []).map(toResult)
// The API currently returns description bodies regardless of // The web search still carries each hit's truncated preview, and the cost
// include_description=false (verified live 2026-08-19), and the cost this // this flag exists to avoid is the share of CLI output descriptions occupy
// flag exists to avoid is the ~73% of CLI output the bodies occupy in // in agent context - so the lean mode drops them client-side.
// agent context - so the lean mode strips them client-side either way.
if (opts.includeDescription === false) { if (opts.includeDescription === false) {
rows = rows.map((r) => ({ ...r, description: null })) rows = rows.map((r) => ({ ...r, description: null }))
} }
@@ -108,8 +108,10 @@ describe("runSearch (mocked fetch)", () => {
await runSearch({ ...searchOpts, query: "backend" }); await runSearch({ ...searchOpts, query: "backend" });
expect(new URL(mock.url()).pathname).toBe("/api/v1/agent/jobs/search"); expect(new URL(mock.url()).pathname).toBe("/api/v1/agent/jobs/search");
expect(requestedParams(mock).get("include_description")).toBe("true");
expect(requestedParams(mock).get("description_format")).toBe("markdown"); expect(requestedParams(mock).get("description_format")).toBe("markdown");
// Neither is read by the API (it lists both in meta.ignored_params).
expect(requestedParams(mock).get("include_description")).toBeNull();
expect(requestedParams(mock).get("semantic_ratio")).toBeNull();
}); });
test("skips description hydration when includeDescription is false", async () => { test("skips description hydration when includeDescription is false", async () => {
@@ -122,10 +124,11 @@ describe("runSearch (mocked fetch)", () => {
const out = captureStdout(); const out = captureStdout();
await runSearch({ ...searchOpts, query: "backend", includeDescription: false }); await runSearch({ ...searchOpts, query: "backend", includeDescription: false });
expect(requestedParams(mock).get("include_description")).toBe("false"); // The agent endpoint hydrates every body regardless of what it is asked, so
// the lean pass goes to the web search, which serves only the preview.
expect(new URL(mock.url()).pathname).toBe("/api/v1/jobs/search");
expect(requestedParams(mock).get("description_format")).toBeNull(); expect(requestedParams(mock).get("description_format")).toBeNull();
// The live API ignores include_description=false and sends bodies anyway // The preview is still a description, so the lean guarantee is enforced client-side.
// (verified 2026-08-19), so the lean guarantee is enforced client-side.
const parsed = JSON.parse(out.get()); const parsed = JSON.parse(out.get());
expect(parsed.results[0].description).toBeNull(); expect(parsed.results[0].description).toBeNull();
}); });
@@ -55,7 +55,7 @@ describe("named search flags require their documented values", () => {
expect(params.getAll("skills")).toEqual(["go", "kubernetes"]); expect(params.getAll("skills")).toEqual(["go", "kubernetes"]);
expect(params.get("company_slug")).toBe("acme"); expect(params.get("company_slug")).toBe("acme");
expect(params.get("work_mode")).toBe("remote"); expect(params.get("work_mode")).toBe("remote");
expect(params.get("include_description")).toBe("false"); expect(requests[0].pathname).toBe("/api/v1/jobs/search");
} finally { } finally {
server.stop(true); server.stop(true);
} }
@@ -15,7 +15,7 @@ Verified against the live API:
| Endpoint | Status | | Endpoint | Status |
|----------|--------| |----------|--------|
| `GET /api/v1/agent/jobs/search` | 200 | | `GET /api/v1/agent/jobs/search` | 200 |
| `GET /api/v1/jobs/search` | 200 (the web variant; not used by this skill) | | `GET /api/v1/jobs/search` | 200 (the web variant; used by `--no-description`) |
| `GET /api/v1/jobs/facets` | 200 | | `GET /api/v1/jobs/facets` | 200 |
| `GET /api/v1/jobs/{slug}` | 200 | | `GET /api/v1/jobs/{slug}` | 200 |
| `GET /api/v1/auth/me` | 401 (auth required — not used here) | | `GET /api/v1/auth/me` | 401 (auth required — not used here) |
@@ -34,15 +34,16 @@ jobs, returning `data: [job, …]` with `meta.total` = the estimated match count
It runs the **same query** as the web-facing `/api/v1/jobs/search` — same `q`, same It runs the **same query** as the web-facing `/api/v1/jobs/search` — same `q`, same
facets, same ranking, same pagination guard (`offset + limit ≤ 10000`) — and differs facets, same ranking, same pagination guard (`offset + limit ≤ 10000`) — and differs
in one respect: asked to, it replaces the search index's truncated `description` in one respect: it replaces the search index's truncated `description`
preview with the posting's **full** description read from the database. That is what preview with the posting's **full** description read from the database. That is what
lets a search of N roles stay one request instead of N + 1. lets a search of N roles stay one request instead of N + 1.
Two extra parameters control it: It always does so - there is no parameter to turn hydration off - so the skill's
`--no-description` mode calls `/api/v1/jobs/search` instead. One extra parameter
controls the rendering:
| Param | Maps to CLI flag | Notes | | Param | Maps to CLI flag | Notes |
|-------|------------------|-------| |-------|------------------|-------|
| `include_description` | (always `true`) | Without it the endpoint serves the index preview, same as the web search. |
| `description_format` | `--description-format` | `markdown` (the skill's default), `text`, or `html`. **An unrecognized value is not an error** — the API falls back to `html`, so the CLI validates the flag itself. | | `description_format` | `--description-format` | `markdown` (the skill's default), `text`, or `html`. **An unrecognized value is not an error** — the API falls back to `html`, so the CLI validates the flag itself. |
Hydration is best-effort per hit: a result whose row has vanished from the database Hydration is best-effort per hit: a result whose row has vanished from the database
@@ -56,8 +57,8 @@ error naming the path rather than as an empty result set.
## `GET /api/v1/jobs/search` ## `GET /api/v1/jobs/search`
The web variant of the same search — identical query surface, but `description` is The web variant of the same search — identical query surface, but `description` is
always the index's truncated preview. The skill does not call it; it is listed here always the index's truncated preview. The skill calls it for `--no-description`,
because the shared parameters below are documented against both. where reading every full body from the database would be wasted work.
Query parameters used by the skill: Query parameters used by the skill:
@@ -66,7 +67,6 @@ Query parameters used by the skill:
| `q` | `--query` / `-q` | Keyword full-text query. | | `q` | `--query` / `-q` | Keyword full-text query. |
| `limit` | `--limit` / `-n` | Page size. Default 25 in the CLI. | | `limit` | `--limit` / `-n` | Page size. Default 25 in the CLI. |
| `offset` | (derived) | `offset = (page - 1) * limit`. | | `offset` | (derived) | `offset = (page - 1) * limit`. |
| `semantic_ratio` | (fixed `0`) | Keyword search; the semantic index is opt-in. |
| `posted_within_days` | `--jobage` | Restrict to postings from the last N days. | | `posted_within_days` | `--jobage` | Restrict to postings from the last N days. |
| `regions` | `--region` | Repeatable; OR within the facet. Values like `global`, `eu`, `us`, `apac`, `latam`, `cis`. | | `regions` | `--region` | Repeatable; OR within the facet. Values like `global`, `eu`, `us`, `apac`, `latam`, `cis`. |
| `countries` | `--country` | Repeatable; ISO-3166 alpha-2 (lowercased). | | `countries` | `--country` | Repeatable; ISO-3166 alpha-2 (lowercased). |