fix(linkedin-search): preserve description paragraph and list breaks (#567)

* fix(linkedin-search): preserve description paragraph and list breaks

Clean rendered description lines separately so whitespace normalization does not erase paragraph and list boundaries in JSON and plain output. Exercise the documented detail CLI against a loopback-served captured response, with field and closed-status controls.

Credits: portapulito diagnosed this behavior and supplied the captured guest-response fixture in upstream PR #153, which was closed as an accidental personalized-fork submission. This independent implementation revives only that correctness fix and its fixture evidence.

* docs(changelog): record fix for #567
This commit is contained in:
Rudy Mizrahi Celekli authored and GitHub committed 2026-10-18 06:55:15 +02:00
1 parent b2439a8011
commit 23876bfa95
5 files changed
+146 -1

No files matched your search

@@ -214,9 +214,12 @@ export function parseJobDetail(html: string, id: string): JobDetail {
extractDivContent(html, "description__text")
if (descHtml) {
const withBreaks = descHtml
.replace(/\s+/g, " ")
.replace(/<\s*br\s*\/?>/gi, "\n")
.replace(/<\/(p|li|ul|ol|div|h\d)>/gi, "\n")
description = decodeHtmlEntities(stripTags(withBreaks)).replace(/\n{3,}/g, "\n\n").trim() || null
// Clean each rendered line separately: stripTags collapses whitespace,
// so applying it to the whole description erased these boundaries.
description = withBreaks.split("\n").map(clean).join("\n").replace(/\n{3,}/g, "\n\n").trim() || null
}
// Job-criteria items: subheader label -> text value.
@@ -0,0 +1,74 @@
import { expect, test } from "bun:test";
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
const endpoint = "https://www.linkedin.com/jobs-guest/jobs/api/jobPosting/4345841651";
const cliPath = join(import.meta.dir, "../src/cli.ts");
const fixture = Bun.file(join(import.meta.dir, "fixtures/detail-orbis-excerpt.html"));
async function detail(format: "json" | "plain") {
const requests: string[] = [];
const server = Bun.serve({
hostname: "127.0.0.1",
port: 0,
fetch(request) {
requests.push(new URL(request.url).pathname);
return new Response(fixture, { headers: { "Content-Type": "text/html" } });
},
});
const dir = mkdtempSync(join(tmpdir(), "linkedin-description-"));
const preload = join(dir, "fetch.ts");
// Exercise the documented CLI in a fresh process, preserving its request
// options. Any request other than this posting fails before network access.
writeFileSync(preload, `
const originalFetch = globalThis.fetch;
globalThis.fetch = (input, init) => {
const url = input instanceof Request ? input.url : String(input);
if (url !== ${JSON.stringify(endpoint)}) throw new Error("Unexpected fixture request");
return originalFetch(${JSON.stringify(server.url.origin)} + new URL(url).pathname, init);
};
`);
try {
const proc = Bun.spawn([
process.execPath, "run", "--preload", preload, cliPath,
"detail", "4345841651", "--format", format,
], { stdout: "pipe", stderr: "pipe" });
const [stdout, stderr, exitCode] = await Promise.all([
new Response(proc.stdout).text(),
new Response(proc.stderr).text(),
proc.exited,
]);
expect(exitCode).toBe(0);
expect(stderr).toBe("");
expect(requests).toEqual(["/jobs-guest/jobs/api/jobPosting/4345841651"]);
return stdout;
} finally {
server.stop(true);
rmSync(dir, { recursive: true, force: true });
}
}
test("detail JSON preserves paragraph and list boundaries from a captured guest response", async () => {
const job = JSON.parse(await detail("json"));
expect(job.description).toContain("Europe\n");
expect(job.description).toContain("Designing and building agentic & multi-agent systems\nDeveloping");
expect(job.description).not.toContain("<p>");
});
test("detail plain output preserves the same description boundaries", async () => {
const output = await detail("plain");
expect(output).toContain("Designing and building agentic & multi-agent systems\nDeveloping");
expect(output).toContain("Status: CLOSED / EXPIRED");
});
test("description formatting retains the captured posting's other fields and text", async () => {
const job = JSON.parse(await detail("json"));
expect(job.id).toBe("4345841651");
expect(job.title).toBe("Agentic AI Engineer - Full remote - Europe");
expect(job.seniority).toBe("Mid-Senior level");
expect(job.employmentType).toBe("Contract");
expect(job.isActive).toBe(false);
expect(job.description).toContain("Designing and building agentic & multi-agent systems");
expect(job.description).toContain("Developing high-performance APIs that power AI agents at scale");
});
@@ -0,0 +1,20 @@
# LinkedIn detail fixture
`detail-orbis-excerpt.html` contains verbatim excerpts from the guest response for
posting `4345841651`, captured by **portapulito** through this CLI's `htmlFetch`
on 2026-07-14 and published in [upstream PR #153](https://github.com/MadsLorentzen/ai-job-search/pull/153).
That PR was closed because it was accidentally submitted to upstream instead of
the author's personalized fork; it was not merged.
Source: `.agents/skills/linkedin-search/cli/test/__fixtures__/detail-orbis.html`
at commit `5bc3afb94d19b670ecf09287c20d0c95e5d13ba0`.
Original SHA-256: `2125a346c22f471e3db08cd3890d0569b194703fe5f49b831cdde257e4aa4a59`.
Only the title, closed banner, description markup and criteria list are retained.
Their bytes are unmodified; recruiter cards, images, tracking links and unrelated
page sections are omitted. The description includes real adjacent paragraphs,
empty paragraphs and list items, rather than hand-written parser examples.
The CLI regression serves this saved response over loopback and redirects only
the exact expected guest endpoint to it in a child process. It makes no LinkedIn
requests and does not regenerate the fixture during tests.
@@ -0,0 +1,46 @@
<h2 class="top-card-layout__title font-sans text-lg papabear:text-xl font-bold leading-open text-color-text mb-0 topcard__title">Agentic AI Engineer - Full remote - Europe</h2>
<figure class="closed-job closed-job__flavor topcard__flavor-row">
<span class="closed-job__icon closed-job__icon--error-pebble lazy-load"></span>
<figcaption class="closed-job__flavor--closed">No longer accepting applications</figcaption>
</figure>
<div class="show-more-less-html__markup show-more-less-html__markup--clamp-after-5
relative overflow-hidden">
<p><strong>Agentic AI Engineer - Full remote - Europe</strong></p><p><br></p><p>Our SaaS client based in The Netherlands are in need of a number of AI Engineers who can design, build and deploy agentic systems to their customers.</p><p><br></p><p>This is a hands-on engineering role, focused on turning ideas into fast POCs and then scaling them into production-ready systems.</p><p><br></p><p><u>What you’ll be doing:</u></p><ul><li>Designing and building agentic &amp; multi-agent systems</li><li>Developing high-performance APIs that power AI agents at scale</li><li>Improving agent performance, responsiveness and reliability</li><li>Collaborating closely with Product to deliver quick POCs and iterate fast</li><li>Deploying AI systems into production using the Azure ecosystem</li><li>Making smart technical trade-offs and clearly explaining them to stakeholders</li></ul><p><br></p><p><u>Tech environment:</u></p><ul><li>Python / Kotlin</li><li>Azure (DevOps, backend services, AI services)</li><li>API-driven architectures</li><li>Agentic / multi-agent systems</li><li>Scalable backend &amp; cloud deployments</li></ul><p><br></p><p><u>What we’re looking for:</u></p><ul><li>Strong experience as an AI Engineer / Backend Engineer</li><li>Proven work with agentic or multi-agent systems</li><li>Solid API development experience at scale</li><li>Comfortable deploying and operating systems in Azure</li><li>Able to explain how agents perform, scale and improve particularly with technical and non-technical stakeholders</li><li>Someone who can deliver fast POCs and think long-term</li></ul><p><br></p><p>Location : Fully remote (Europe)</p><p>Duration : 3-month rolling contract</p><p>Headcount : 2</p><p>Rate : €375 - 400 a day</p><p><br></p><p>If this sounds like you, or you know someone who fits, drop me a message and I’ll share more details.</p>
</div>
<ul class="description__job-criteria-list">
<li class="description__job-criteria-item">
<h3 class="description__job-criteria-subheader">
Seniority level
</h3>
<span class="description__job-criteria-text description__job-criteria-text--criteria">
Mid-Senior level
</span>
</li>
<li class="description__job-criteria-item">
<h3 class="description__job-criteria-subheader">
Employment type
</h3>
<span class="description__job-criteria-text description__job-criteria-text--criteria">
Contract
</span>
</li>
<li class="description__job-criteria-item">
<h3 class="description__job-criteria-subheader">
Job function
</h3>
<span class="description__job-criteria-text description__job-criteria-text--criteria">
Information Technology
</span>
</li>
<li class="description__job-criteria-item">
<h3 class="description__job-criteria-subheader">
Industries
</h3>
<span class="description__job-criteria-text description__job-criteria-text--criteria">
Technology, Information and Media
</span>
</li>
</ul>