feat(linkedin-search): report closed postings via isActive, wire into /scrape (adopts #280) (#383)

* feat(linkedin-search): add active status verification for job postings

* fix(linkedin-search): scope closed-posting detection to the top card, pin with tests (#280)

The first version matched five markers against the whole document, so
recruiter boilerplate quoting 'no longer accepting applications' in a
description flagged a live job CLOSED. Detection now stops where the
description markup begins and matches only the two markers real closed
pages carry (closed-job__flavor and the banner text, verified against
live guest pages); the three speculative phrases are dropped. Four new
fixture tests pin both directions plus the two description false-positive
cases - the false-positive pair fails on the unscoped version.

* feat(scrape): mark closed-at-source LinkedIn postings expired, never drop (#280)

/scrape Step 2 now consumes linkedin-search detail's isActive: a job whose
posting page renders the closed banner is written to seen_jobs.json with
status expired rather than silently dropped, per the /rank marking pattern -
the fix for the ghost-jobs class in #331. isActive: true is documented as
absence of the banner, not proof the posting is open.

---------

Co-authored-by: Navakanth Reddy Dumpa <navkanthr@gmail.com>
This commit is contained in:
Ayobami Adegoke
2026-08-29 11:12:26 +02:00
committed by GitHub
co-authored by Navakanth Reddy Dumpa
parent 730dcfb079
commit 3d296448bd
5 changed files with 88 additions and 0 deletions
@@ -39,6 +39,7 @@ export async function runDetail(opts: DetailOpts): Promise<number> {
job.employmentType ? `Employment: ${job.employmentType}` : "", job.employmentType ? `Employment: ${job.employmentType}` : "",
job.jobFunction ? `Function: ${job.jobFunction}` : "", job.jobFunction ? `Function: ${job.jobFunction}` : "",
job.industries ? `Industries: ${job.industries}` : "", job.industries ? `Industries: ${job.industries}` : "",
`Status: ${job.isActive ? "ACTIVE" : "CLOSED / EXPIRED"}`,
"", "",
job.description || "(no description)", job.description || "(no description)",
"", "",
@@ -63,6 +63,7 @@ export interface JobDetail extends JobCard {
employmentType: string | null employmentType: string | null
jobFunction: string | null jobFunction: string | null
industries: string | null industries: string | null
isActive: boolean
} }
/** /**
@@ -227,6 +228,21 @@ export function parseJobDetail(html: string, id: string): JobDetail {
criteria[clean(cm[1]).toLowerCase()] = clean(cm[2]) criteria[clean(cm[1]).toLowerCase()] = clean(cm[2])
} }
// Closed-state detection, scoped to the top card. A closed posting renders
// <figure class="closed-job closed-job__flavor topcard__flavor-row">
// <figcaption ...>No longer accepting applications</figcaption>
// </figure>
// there; that class and its visible text are the only markers real closed
// pages carry (verified against live guest pages, 2026-08-09). The search
// stops where the description markup begins: recruiter boilerplate quotes
// these phrases, and a false CLOSED talks a user out of a live job.
// Absence of the banner is absence of evidence, not proof the posting is
// open - markup drift or a consent-walled response also renders no banner -
// so isActive: true means only "no closed banner found".
const descStart = html.search(/class="(?:show-more-less-html__markup|description__text)/i)
const topcard = descStart === -1 ? html : html.slice(0, descStart)
const isActive = !/closed-job__flavor|no longer accepting applications/i.test(topcard)
return { return {
id, id,
title: title ? clean(title) : "(untitled)", title: title ? clean(title) : "(untitled)",
@@ -240,6 +256,7 @@ export function parseJobDetail(html: string, id: string): JobDetail {
employmentType: criteria["employment type"] ?? null, employmentType: criteria["employment type"] ?? null,
jobFunction: criteria["job function"] ?? null, jobFunction: criteria["job function"] ?? null,
industries: criteria["industries"] ?? null, industries: criteria["industries"] ?? null,
isActive,
} }
} }
@@ -86,6 +86,51 @@ describe("decodeHtmlEntities (via parseJobCards)", () => {
}); });
}); });
describe("parseJobDetail active-status detection", () => {
// Captured from a real closed guest posting (2026-08-09): the banner LinkedIn
// actually renders inside the top card. Its class and its visible text are the
// only closed markers that occur in the wild.
const closedBanner = `
<figure class="closed-job closed-job__flavor topcard__flavor-row">
<span class="closed-job__icon closed-job__icon--error-pebble lazy-load"></span>
<figcaption class="closed-job__flavor--closed">No longer accepting applications</figcaption>
</figure>`;
const page = (topcardExtra: string, description: string) => `
<h1 class="topcard__title">Data Engineer</h1>
<span class="topcard__flavor topcard__flavor--bullet">Berlin</span>
${topcardExtra}
<div class="show-more-less-html__markup">${description}</div>`;
test("a closed posting's top-card banner yields isActive: false", () => {
const job = parseJobDetail(page(closedBanner, "We build things."), "1");
expect(job.isActive).toBe(false);
});
test("an open posting yields isActive: true", () => {
const job = parseJobDetail(page("", "We are hiring!"), "2");
expect(job.isActive).toBe(true);
});
test("recruiter boilerplate in the description does not flag a live posting", () => {
// The review's false-positive case: the closed phrase appears in the
// *description text* of a job that is very much open.
const job = parseJobDetail(
page("", "Apply soon - once filled, this posting is no longer accepting applications."),
"3",
);
expect(job.isActive).toBe(true);
});
test("a closed-job class named in the description does not flag a live posting", () => {
const job = parseJobDetail(
page("", "Our design system documents a closed-job__flavor CSS class."),
"4",
);
expect(job.isActive).toBe(true);
});
});
describe("parseJobDetail dropped fields", () => { describe("parseJobDetail dropped fields", () => {
test("emits no applyUrl field", () => { test("emits no applyUrl field", () => {
// The extraction regex assumed class-before-href and never matched // The extraction regex assumed class-before-href and never matched
+10
View File
@@ -94,6 +94,16 @@ and URL. For jobs worth a deeper look, fetch full detail with that portal's `det
command (see its SKILL.md — do not guess flags) to extract **key requirements**, command (see its SKILL.md — do not guess flags) to extract **key requirements**,
**application deadline**, and a brief description snippet. **application deadline**, and a brief description snippet.
**Closed-at-source detection:** `linkedin-search detail` also returns `isActive`.
`false` means the posting page itself renders LinkedIn's "No longer accepting
applications" banner — the job died between being indexed and being fetched (expired
LinkedIn URLs redirect to *similar live jobs*, so a search hit can be a ghost). Mark
such a job, never silently drop it: write its entry to `seen_jobs.json` in Step 4 with
`"status": "expired"` and leave it out of the Step 5 presentation — an absent entry
looks identical to a job never seen, and the recorded status is what makes a later
ghost report self-triaging. `isActive: true` is only the absence of that banner, not
proof the posting is open; deadlines and dead URLs remain `/rank`'s job.
**From WebSearch results:** Use `WebFetch` on the posting URL and extract the same **From WebSearch results:** Use `WebFetch` on the posting URL and extract the same
fields manually. If it returns HTTP 403, retry with browser headers via curl per fields manually. If it returns HTTP 403, retry with browser headers via curl per
`.claude/skills/job-application-assistant/09-web-research.md` before giving up — most `.claude/skills/job-application-assistant/09-web-research.md` before giving up — most
+15
View File
@@ -40,6 +40,21 @@ per-file diff commands.
### Added ### Added
- **`linkedin-search detail` reports closed postings** (#280, adopted with the original
author's commit preserved) - a new `isActive` field: `false` when the posting page
renders LinkedIn's own "No longer accepting applications" top-card banner. Detection
is scoped to the top card and pinned by fixture tests in both directions, including
the false-positive case the review required (recruiter boilerplate quoting the closed
phrase in a *description* must not flag a live job - on the unscoped first version it
did, and the new tests fail there). Only the two markers real closed pages carry are
matched (`closed-job__flavor` and the banner text, verified against live guest
pages); three speculative phrases from the first version were dropped as
false-positive-only risk. `/scrape` Step 2 now consumes the signal: a closed-at-source
job is recorded in `seen_jobs.json` as `"status": "expired"` - marked, never silently
dropped, per the `/rank` pattern - which is the fix for the ghost-LinkedIn-jobs class
in #331 (an expired LinkedIn URL redirects to a *similar live job*, so a stored hit
can die unnoticed between scrape and click). `isActive: true` is documented as
absence of the banner, not proof the posting is open.
- **pypdf ATS text-layer fallback** - `/apply` Step 5d and `tools/verify_pdf.py` extract the CV PDF text layer with **pypdf** first (BSD, `pip install pypdf`) so Windows machines without Poppler still get a mechanical parseability check. Poppler `pdftotext -layout -enc UTF-8` remains the fallback; if both are missing the check still degrades to a visual keyword review. No extra cache or installer. `05-cv-templates.md` `framework_version` 1.4.2 → 1.4.3. - **pypdf ATS text-layer fallback** - `/apply` Step 5d and `tools/verify_pdf.py` extract the CV PDF text layer with **pypdf** first (BSD, `pip install pypdf`) so Windows machines without Poppler still get a mechanical parseability check. Poppler `pdftotext -layout -enc UTF-8` remains the fallback; if both are missing the check still degrades to a visual keyword review. No extra cache or installer. `05-cv-templates.md` `framework_version` 1.4.2 → 1.4.3.
- **CI now tests the full documented Python range** (#370) - the Python tool tests job - **CI now tests the full documented Python range** (#370) - the Python tool tests job
runs a 3.10-3.14 version matrix instead of pinning 3.12, so both the documented 3.10 runs a 3.10-3.14 version matrix instead of pinning 3.12, so both the documented 3.10