36 Commits
Author SHA1 Message Date
Mads LorentzenandClaude Fable 5.1 d7287ec834 chore(release): cut v1.7.2 (#507)
Rolls [Unreleased] into [1.7.2] - 2026-09-29 and moves the compare links.
Keeps an empty [Unreleased] heading above the release, per Keep a Changelog.
Thirty-two entries: seven added, one changed, twenty-four fixed.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-29 20:32:45 +02:00
Sahil Kumar SahuandClaude Opus 5.5 4f5520841f fix(freehire-search): decode HTML entities and trim whitespace in titles (#505)
Titles are indexed as the source ATS wrote them and toResult passed them
through untouched, so `detail intern-fullstack-amp-ai-innovation-noise-edaawqq2`
printed `Intern - Fullstack &amp; AI Innovation ` and the same title reached
`search` output. In a 93-result /scrape batch, 1 title carried `&amp;` and
3 ended in whitespace; /scrape Step 4.75 reads an undecoded entity in a
title as a half-working parser.

Run the title through the existing decodeHtmlEntities (already used for
detail descriptions) and trim it; an empty or whitespace-only title still
falls back to "(untitled)". Three new parsing.test.ts cases pin it; all
three fail on the old pass-through.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 20:24:56 +02:00
Sahil Kumar SahuandClaude Opus 5.5 9187f894a5 fix(freehire-search): keep the salary period so a yearly figure isn't read as monthly (#504)
formatSalary printed only the currency and amounts, although freehire's
enrichment records the pay period. `detail software-developer-trainee-codifi-fra635ux`
printed `INR 300000–300000` while the API returns `salary_period: "year"`
for that posting, so a yearly salary screened against a monthly figure
read as twelve times its real monthly value.

Append the period when freehire records one (`INR 300000–300000/year`,
`INR 12000/month`); a record without a period prints exactly as before.
Two new parsing.test.ts cases pin it; both fail on the old formatter.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 20:22:04 +02:00
9bc71de5f8 fix(robots-check): decode curl output as UTF-8 so non-ASCII responses stop reading as UNCONFIRMED on Windows (#506)
_fetch() ran curl with text=True and no encoding, so Python decoded the
response with the Windows ANSI code page. On a byte that code page leaves
undefined (0x81 in cp1252 and cp1254, 0x9e in cp1254) subprocess's pipe
reader thread died, stdout came back None, rpartition() raised
AttributeError, and the gate printed "UNCONFIRMED (AttributeError)". That
fails closed, but it blocked the browser-header retry 09-web-research.md
permits on hosts such as tr.indeed.com (a UTF-8 CJK Disallow rule) and
kap.org.tr (a UTF-8 Turkish 404 page).

Pin encoding='utf-8', errors='replace' on the call (RFC 9309: robots.txt
is UTF-8), as verify_pdf.py and verify_layout.py already do. Decision
rules and the fail-closed handling of real read failures are unchanged.

The regression tests swap curl for a child process that prints the real
bytes, so the actual subprocess decode path runs with _fetch's own
kwargs, and pin cp1254 when no encoding is passed so a UTF-8 host cannot
pass the bug by luck. Both fail on master.

Co-authored-by: Can Mugan <217335573+canmugan@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 20:18:48 +02:00
Yang 97e2575795 fix(tests): escape the regex in test_verify_pdf's docstring and compile-check tests/*.py (#503)
FindNonAsciiDateRangesTests' docstring (from #498) quotes the regex `\s*`
in a non-raw string. `\s` is an invalid escape sequence, so Python 3.12+
prints a SyntaxWarning for it (CPython gh-98401) when
tests/test_verify_pdf.py is compiled. It is in this repo's own CI log
for a45bbd1 on 3.12, 3.13 and 3.14, at tests/test_verify_pdf.py:75, while
the run stays green. Under `-W error::SyntaxWarning` the module fails to
import and the whole test_verify_pdf suite drops out of the run.

Escaped in the docstring. ToolsCompileWithoutWarnings (#480) only compiled
tools/*.py, which is why this got through; it now compiles tests/*.py too.
With the unescaped docstring it fails naming tests/test_verify_pdf.py, and
it passes with the fix.

Verification on Python 3.10, 3.11, 3.12, 3.13 and 3.14, from a clean
bytecode cache: `python -W error::SyntaxWarning -m unittest discover -s
tests -t .` OK (501 tests, the same count as master),
`tools/lint_skills.py`, `tools/check_framework_version.py`,
`tools/security_guards.py`.
2026-09-29 20:15:19 +02:00
Ayobami Adegoke a45bbd1a2a feat(verify_pdf): add --ascii-dates, the non-folding check for the documented date-range rule (#385) (#498)
* feat(verify_pdf): add --ascii-dates, the non-folding check for the documented date-range rule (#385)

05-cv-templates.md records a real Workday import that dropped a role's
end date because `2016--2024` reaches the text layer as 2016<U+2013>2024,
and tells Step 5d to confirm every entry's years are joined by an ASCII
hyphen. Nothing executed that. The natural probe, --contains "2016-2024",
failed identically for "en-dashed" and "absent" before #458 and passes on
both since, because the fold maps the en-dash back to "-" on purpose.

Add --ascii-dates: scan the raw text layer (never the fold) for a year
joined to any Unicode dash or the minus sign and fail naming each hit
with its line and code point. A year on either side of the dash is
enough (Mar 2016 - Jul 2016, 2016 - Present); a numeric range with no
year is not a date. The dump is still written before the check fails.

Wire it in: Step 5d's extraction command carries the flag and the
"Dates recognizable" item points at it, keeping the bare single-year
half as a read-through; the guide's "add this to the step 5d checks"
paragraph names the command (framework_version 1.4.4 -> 1.4.5); the
upstream-only CI assertion runs it on the stock CV.

Demonstrated on the stock template compiled with lualatex through both
extractors: a \cventry{2016--2024} build fails with U+2013 while
--contains "2016-2024" passes on it; the \cventry{2016-2024} build passes
both; the untouched stock CV passes the CI assertion.

Ten new test_verify_pdf.py cases. Proposed by 9scorp4 in Discussion #385.

* fix(verify_pdf): --ascii-dates only joins a year and a dash across horizontal whitespace

`\s*` between the year and the dash also matched a newline, so a heading
ending in an ASCII date ("Heading 1988-1994\n- note") joined the dash
that merely opened the next line and was reported as a U+2013 range; the
mirror shape ("Six Sigma -\n2016 onwards") did the same. Both are real
text-layer shapes: a dash used as a bullet, a wrapped prose line.

Match horizontal whitespace only ([^\S\n]*). Every existing case still
passes; three fixtures pin the two newline shapes and the same-line
spaced dash that must still be caught.

Found in review by yang2632, with the regex and the fixture.
2026-09-28 19:06:41 +02:00
Ayobami Adegoke 8cbab6823e fix(linkedin-search): reject detail URLs on other hosts instead of fetching the posting with that number (#500)
normalizeId took the first 6+-digit path segment from ANY URL, so a
Greenhouse or Lever apply link - the kind a posting's own page hands out
and a user pastes back into detail - was reduced to that number and the
handler fetched jobPosting/<number> from LinkedIn: whatever job carried
that id came back with exit 0, or NOT_FOUND if none did, never an error
about the input. Driving the real handler with a stubbed fetch,
https://boards.greenhouse.io/acme/jobs/4567890 requested
.../jobPosting/4567890. Every other portal CLI rejects an off-host detail
URL with BAD_ID (the #447 shape); linkedin was the one still trusting the
digits.

Parse URLs for real: a linkedin.com host (apex or any subdomain, scheme
optional) plus a /jobs/view/<slug->id path yields the id; other hosts,
look-alike and userinfo hosts, and linkedin.com profile or search URLs
exit 1 with the stderr-JSON BAD_ID contract before any request. Bare
ids, URNs, and slash-free title slugs are unchanged.

Four new normalizeId cases and a new detail-input.test.ts that drives
runDetail and the CLI with fetch stubbed; five cases fail on the old
pattern.
2026-09-27 19:28:16 +02:00
Yang a62af6a068 fix(verify-layout): escape the docstring's LaTeX macro so the tool stops emitting SyntaxWarning (#480)
`tools/verify_layout.py`'s module docstring explains which documents trigger
the Poppler `-bbox` crash, and names the macro whose absence causes it:
`\hypersetup{pdftitle=...}`. In a non-raw docstring `\h` is an invalid escape
sequence, so compiling the module prints

    SyntaxWarning: "\h" is an invalid escape sequence.
    Such sequences will not work in the future. Did you mean "\\h"?

on Python 3.12+ (CPython gh-98401 promoted the long-standing
DeprecationWarning to a SyntaxWarning). The 3.15 language reference still
documents the sequence as a SyntaxWarning, with a SyntaxError only "in a
future Python version".

This is on the real path, not a constructed one: /apply Step 5b runs
`python3 tools/verify_layout.py <pdf>` once per document, so the first
invocation after any source change prints the warning into the middle of the
layout report a user is reading to judge their CV. It is also visible in this
repo's own CI log, where `python -m unittest discover -s tests -t . -v` prints
it above the test results.

Reproduce on master:

    python3 -W error::SyntaxWarning -c "compile(open('tools/verify_layout.py',
      encoding='utf-8').read(), 'tools/verify_layout.py', 'exec')"

raises on master; after this change it is silent.

Escaping (rather than making the docstring raw) keeps the surrounding prose
unchanged, and the single-backtick markup matches the other inline literals in
the same docstring. The rendered text still reads `\hypersetup{...}`.

The new `ToolsCompileWithoutWarnings` case is a pure `compile()` of the source
text of every `tools/*.py` inside `warnings.catch_warnings(record=True)` and
asserts no `SyntaxWarning` or `DeprecationWarning` (3.10 and 3.11 report the
same escape as a DeprecationWarning) - no cache file, no temp file, so it runs
the same on Windows (py_compile's atomic rename over an open
NamedTemporaryFile is what the first version tripped on). It fails on master
and passes here on every Python in the CI matrix, and it guards the next
docstring that quotes a macro.

Verification on Python 3.10, 3.11, 3.12, 3.13 and 3.14: `python3 -m unittest discover -s
tests -t .` (488 tests OK; master is 487), `python3 -W error::SyntaxWarning`
compile of every `tools/*.py`, `tests/test_changelog_structure.py`.
2026-09-27 19:28:00 +02:00
Ayobami Adegoke 120f476a08 fix(rank): preserve Unicode identity in tracker matching (#486) 2026-09-21 08:41:22 +02:00
Mads LorentzenandClaude Fable 5.1 ba4f2397a3 fix(tools): write UTF-8 on stdout so non-Latin postings stop crashing on Windows (#490)
A piped stdout on Windows defaults to the ANSI code page (cp1252 on most
Western installs). rank_state.py printed titles and companies with
ensure_ascii=False, so one Cyrillic, CJK, Devanagari, Polish or Turkish
posting ended /rank with UnicodeEncodeError. job_key --audit, salary_lookup,
convert_salary_excel and verify_layout failed the same way.

Each tool now reconfigures stdout and stderr to UTF-8 at entry. The new
tests run every tool in a child process with a cp1252 stdout forced through
PYTHONIOENCODING, so Linux CI reproduces the Windows failure.


Claude-Session: https://claude.ai/code/session_01WPXraLZ4i7UW4xGn9tVcox

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-21 08:37:18 +02:00
Ayobami Adegoke 2e8600d69d fix(rank): reject invalid score dimensions before saving results (#488) 2026-09-21 08:26:43 +02:00
Ayobami Adegoke 456f2bfdbf fix(scrape): distinguish non-Latin companies in fallback job keys (#487) 2026-09-21 08:24:15 +02:00
Ayobami Adegoke 6a308c5cc8 fix(rank): handle UTF-8 BOM in tracker CSV headers (#485) 2026-09-21 08:21:04 +02:00
Ayobami Adegoke 362ef6a522 test(salary): exercise real workbook conversion through the CLI (#489) 2026-09-21 08:20:50 +02:00
Jakob Stender Guldberg 5bb5f79f7c feat(add-template): keep a registered template's intermediates in build/ (#473) (#474)
* feat(add-template): keep a registered template's intermediates in build/

A template registered via /add-template declared a bare compile command, so
every new template scattered .aux/.log/.out beside its source, and Step 4 had to
enumerate six LaTeX extensions to clean up after its test compile.

- The elicited LaTeX command now redirects intermediates to build/ with the
  portable -output-directory (not the MiKTeX-only -aux-directory), creates the
  folder (TeX Live does not), and moves the PDF back beside the source where
  /apply reads it. A leading rm -f ensures a failed compile, which skips the mv,
  never leaves a stale PDF there.
- Toolchains with nothing to redirect (typst compile) keep their command, and
  registration never blocks on the rule. If the redirect alone breaks a
  template (e.g. \include from a subfolder), it is dropped and recorded under
  Known pitfalls.
- The manifest example shows the pattern, Step 4 cleanup deletes build/
  instead of listing extensions, and the ACTIVE-TEMPLATE block tells /apply to
  run the command from the output directory and remove build/ in Step 5e.
- build/ is gitignored.

Stock templates are untouched, so /apply Step 5e stays.

* docs(add-template): drop the MiKTeX aside from the build directory rule

* test(add-template): pin the build directory compile command
2026-09-21 08:20:10 +02:00
Ayobami Adegoke 09435eb1a5 fix(apply): run the page-count check Step 5b claimed Step 5d already ran (#476)
Step 5b's prose said "Page count is not checked here - that is
verify_pdf.py --pages's job, and Step 5d already runs it", and
verify_layout.py's docstring declines to measure page count for the same
reason. Step 5d's only verify_pdf.py call is --dump-text, and no step in
the workflow passed --pages at all (only the upstream-only CI assertion on
the stock examples does), so the hard 2-page CV and 1-page cover letter
limits were enforced by nothing but the visual PDF read - the "measure
first, then look" failure 5b was written to stop.

5b now runs verify_pdf.py --pages 2 on the CV and --pages 1 on the cover
letter ahead of verify_layout.py, names the ACTIVE-TEMPLATE page limit as
the substitute for a custom template, and the deferral sentence points at
those lines instead of at 5d.

tests/test_apply_page_count.py pins the invocations, their counts, their
order relative to the layout measurement, and that no prose defers the
check to a step that does not run it; all four cases fail on master.
2026-09-16 21:09:33 +02:00
Ayobami Adegoke b91c6125ec fix(salary): print the privacy footnote only when a row rendered N/A* (#475)
format_entry appended "* N/A = Too few employees to publish (privacy)"
under every category table, including one where every row has an index,
so the output asserted a suppression that never happened - the residual
noted on #470. Set a flag in the N/A* branch and print the footnote only
when it fired; a table with a suppressed row renders exactly as before.

Two FormatEntryTests cases pin both directions; the "omitted" case fails
on master.
2026-09-16 21:06:23 +02:00
Ayobami Adegoke 968fb1bd7e fix(salary): pair a bare Count/Index column pair instead of splitting it into two categories (#470)
The pairing loop required a non-empty derived category name on both
sides, but a header with no category word - "Count" + "Index", Danish
"Antal" + "Lønindeks" - strips to an empty name, so the simplest layout
the README advertises ("auto-pairs count/index columns") came out as two
unrelated standalone categories:

  {"count": {"count": 500}, "index": {"index": 108.5}}

salary_lookup then rendered a "Count  500  N/A*" row above an
"Index  -  108.5" row, and its footnote read the N/A* as "too few
employees to publish (privacy)" - a false statement about a company whose
headcount is in the file, shown during /apply's salary step. Any suffix
("Antal alle") made pairing work, which is why the shipped tests, all
suffixed, never saw it.

Pair on equal derived names, empty included, and give the nameless pair
the README's top-level category name (all_employees). A bare "Antal" with
no bare index column still stays a standalone count; named pairs
alongside are untouched.

Four new cases in test_convert_salary_excel.py, one of them rendering the
converter's output through salary_lookup.format_entry; all four fail
against the old pairing rule.
2026-09-16 06:47:45 +02:00
Oscar Madera e92f7d9065 feat(setup): add documents/projects/ ingestion for independent projects (Path A) (#468) 2026-09-16 06:47:15 +02:00
Jakob Stender Guldberg c93609cd22 fix(gmail-sync,outcome): keep free-form tracker notes free of CSV-breaking characters (#454) (#455)
* fix(gmail-sync): strip CSV-breaking characters from the email subject

Step 7a interpolated the raw subject line of a received email into the
`notes` column of job_search_tracker.csv. No writer here emits a quoted
tracker field, so an unescaped comma splits the row - for csv.DictReader
just as much as for a naive split, which matters because
tools/rank_state.py is the repo's only machine reader and uses exactly
that. `notes` is column 10 of 14, so a subject as ordinary as
"Re: Your application, Data Scientist" shifted cv_file,
cover_letter_file and source a column left. A line break is worse: it
ends the row and starts a second one.

The rule now sits on the append instruction itself rather than in a
general note a writer can miss. The subject survives verbatim in the
archive's outcome.md, which is Markdown and carries no such constraint.

/outcome Step 4 (outcome.md:195) is also free-form and has the same
exposure, but its text is model-authored in a turn the user is watching
rather than copied from third-party mail unattended. Left out
deliberately, to be filed separately.

* fix(outcome): keep the Step 4 tracker note free of CSV-breaking characters

/outcome Step 4 appended "a short dated note" to `notes` with no
constraint on its content, the same exposure /gmail-sync Step 7a had:
nothing quotes a tracker field, so `rejected, no feedback given` shifts
cv_file, cover_letter_file and source a column left under
csv.DictReader, and a line break ends the row. The append instruction
now requires a note with no commas, double quotes or line breaks.

Folded in at the maintainer's request on #455 so one entry and one rule
cover both free-form writers. The CSV-safety tests move out of
test_gmail_sync_command.py into test_tracker_notes_csv_safe.py, where a
CASES table pins the rule on each writer's append line.
2026-09-16 06:45:55 +02:00
Souptik Chakraborty 1b65f7198a fix(verify-layout): name both causes of a broken bounding-box extractor (#451) (#465)
verify_layout.py's skipped: message blamed only the xpdf-based pdftotext that
Git for Windows puts ahead of Poppler in PATH. A real Poppler can abort too:
26.0x before 26.05 crashes -bbox/-bbox-layout/-htmlmeta on a PDF whose Info
dictionary carries an empty string in any field, and hyperref writes exactly
that for every field it does not set. A lualatex/pdflatex document built with
hyperref and no \hypersetup{pdftitle=...} - an ordinary /add-template CV
template, not a malformed one - hits this with a working Poppler installed,
and the old message sent the reader to check their PATH when nothing was
wrong with it.

Named both causes in the raised message, the code comment above it, and the
module docstring. Behavior is unchanged: either cause still degrades to
skipped: exit 2, not exit 1, since a broken extractor is still not a broken
document.

Added test_poppler_abort_on_empty_info_string_names_that_cause_too, mocking
subprocess.run to raise CalledProcessError the way a real Poppler 26.0x abort
does (exit 1, a libc++abi out_of_range trace on stderr) rather than the
xpdf case's exit 99 - the two failures are the same exception type with
different exit codes and stderr text, so the message has to actually
distinguish them, not just catch the one class. Negative control: this test
fails against the pre-fix message with "'Poppler aborted' not found in
'...a pdftotext without -bbox is usually the xpdf build...'".

Thanks to @main-sounds-audio for the report and the isolated repro (which
field, which Poppler modes, five runs of five) that pinned this to Poppler's
own upstream regression (issue #1699, fixed in 26.05.0) rather than an
extraction-library swap.

Fixes #451
2026-09-16 06:43:20 +02:00
Oscar Madera 88d1b61047 feat(apply): verify posting source host against installed portals and known ATS apexes (#431) (#467)
- Add source host verification rule to apply.md Step 1 before drafting
- Check posting URL host against installed portals and six official ATS apexes
- Enforce fail-closed look-alike parsing (prefix, suffix, userinfo spoofing)
- Plainly flag unverified third-party hosts in evaluation output
- Add tests/test_apply_host_check.py and update CHANGELOG.md
2026-09-16 06:43:16 +02:00
Mads LorentzenandClaude Fable 5.1 27eb57ae93 docs(readme): say up front that Claude Code needs a paid plan or API credits (#456)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WPXraLZ4i7UW4xGn9tVcox
2026-09-14 18:50:00 +02:00
Mads LorentzenandClaude Fable 5.1 6de14ea85b changelog: fork-reconcile note on the #458 entry, PR number on the #463 entry
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WPXraLZ4i7UW4xGn9tVcox
2026-09-14 18:36:12 +02:00
NathanandNathan No-ot 9484a61831 fix(ci): skip the pristine-template guard in test_setup_command on forks (#463)
TemplatesStillCarryThePlaceholders asserts 05-cv-templates.md and 06-cover-letter-templates.md still carry their [FIRST_NAME]/[YOUR_*] tokens. /setup's documented Step 3.5/3.6 personalisation replaces exactly those tokens, so python-tests failed permanently on any personalized fork. Same @unittest.skipIf on GITHUB_REPOSITORY (defaulting to upstream when unset) as test_placeholder_integrity.py received in #407; that fix landed three days before this guard, which did not pick up the pattern.

Co-authored-by: Nathan No-ot <244263078+Nnoot02@users.noreply.github.com>
2026-09-14 18:35:23 +02:00
Ayobami Adegoke 73d52e0991 fix(verify_pdf): fold LaTeX's typographic substitutions before --contains; guard T1 fontenc for pdflatex (#385, #384) (#458)
`normalize_text()` folded whitespace only, so `--contains` compared what a
user types against what LaTeX renders. The stock CV compiled with the
documented lualatex command turns `'` into U+2019 and `--` into U+2013, so
`--contains "Master's degree"` and `--contains "2016-2024"` both reported
the keyword missing from a document that plainly contains it, through both
extractors. The documented remedy for a missing keyword is to add it, which
is the one thing the ATS section forbids.

Fold both sides at comparison time: NFC, then curly apostrophes and quotes
to ASCII, en/em dashes to `-`, no-break space to space. `--dump-text` still
writes the raw layer - that is what an ATS parses, and the date-range rule
in 05-cv-templates.md needs the raw en-dash visible there.

Separately, pdflatex without T1 font encoding stores accents decomposed
(`e` + U+0300). NFC repairs the pdftotext side of that, but pypdf reads the
same layer as `Z¨ urich` with a spacing accent, which no fold recovers.
moderncv 2.5 loads T1 itself under pdflatex; the apt-packaged 2.3.1 does
not - reproduced by compiling the template against moderncv v2.3.1 with
pdflatex (before: U+0308/U+0300 in pdftotext, `Z¨ urich` in pypdf; after:
U+00FC/U+00E8 in both). The template and the guide's preamble gain
`\ifpdftex\usepackage[T1]{fontenc}\fi`; the lualatex text layer is
byte-identical before and after.

Tests: ten new cases in test_verify_pdf.py (the fold-through and
normalize_text ones fail on the whitespace-only code) and a
test_latex_guidance.py guard that the fontenc line exists and stays inside
the pdflatex branch. framework_version 1.4.3 -> 1.4.4 on 05-cv-templates.md.

Reported and diagnosed by 9scorp4 in Discussions #385 and #384.
2026-09-14 18:30:50 +02:00
Ayobami Adegoke c2cd71ddee fix(jobdanmark-search): back off on 429/5xx in detail instead of failing on the first attempt (#460)
`detail` called fetch() directly rather than going through the CLI's own
request wrappers, so it had none of the three things apiFetch/apiPost
guarantee: no 429/5xx retry loop, a hand-inlined User-Agent that would
drift from the exported USER_AGENT, and a timeout no wrapper test covered.
A rate-limited detail page wrote API_ERROR and exited after ONE attempt;
jobnet, jobbank, jobindex, linkedin, and freehire all retry up to six
times on the same response. /scrape calls detail once per shortlisted
posting, so a burst that tripped jobdanmark's limiter dropped those
postings (no description, no deadline) while any other portal rode it out.

Demonstrated by driving the real command handler with a stubbed 429 and
instant timers: 1 fetch attempt and exit 1 before, 7 after (initial try
plus six retries, the contract's schedule).

Add htmlFetch to helpers.ts with the same backoff, timeout, and shared
User-Agent as the JSON wrappers - 404 returns null so detail keeps its
NOT_FOUND contract - and route detail through it. The retry-backoff,
user-agent, and request-timeout suites now cover all three wrappers, and
the new detail-backoff.test.ts exercises the handler path itself; its two
retry cases fail against the bare fetch().
2026-09-14 18:25:30 +02:00
Ayobami Adegoke c7bd494f11 fix(jobindex-search): reject non-jobindex detail URLs instead of fetching them verbatim (#447) (#448)
detail fetched any http(s) input verbatim with no host check and, when
the path didn't match, silently used the whole input URL as the job id -
a non-posting page came back as a well-formed fake posting with exit 0.
buildUrl now requires a jobindex.dk host and a /jobannonce/<id> path,
rebuilds the fetch URL from the extracted id, and exits 1 with the
stderr-JSON BAD_ID contract otherwise; bare ids stay permissive
slash-free tokens per the jobnet precedent. Eight cases in the new
detail-input.test.ts; the five rejection/canonicalization cases fail
against the verbatim unguarded extraction.
2026-09-10 20:36:54 +02:00
Jakob Stender Guldberg c776e3f2b6 fix(outcome,interview): glob the full <company>_<role> stem in the CV fallback (#444)
When a tracker row's cv_file/cover_letter_file columns are empty, /outcome
and /interview fell back to a company-prefix glob (cv/main_<company>*.tex).
/apply names drafts main_<company>_<role><CV_EXT>, so two roles at one
company both match that glob. /outcome copied whichever the filesystem
returned first into the archive as cv_draft.tex - the file whose stated
purpose is to record what was actually submitted - and its own "leave an
existing archived file" rule then made the wrong copy permanent.

Both fallbacks now glob cv/main_<company>_<role>.* and
cover_letters/cover_<company>_<role>.*, deriving the stem by the Subfolder
naming rule in documents/README.md rather than restating it, and skip with
a note instead of widening the search. Dropping the hardcoded .tex also
makes a template registered by /add-template findable.

The dot before the extension wildcard matters: a bare trailing * also
absorbs a longer role, so ML Engineer and ML Engineer II at one company
would collide the same way the company-prefix glob did.
2026-09-10 20:32:37 +02:00
cbd8a991ab feat(layout): measure compiled PDF layout instead of eyeballing it (#378)
* feat(layout): measure compiled PDF layout instead of eyeballing it

/apply Step 5b asks for layout properties and executes none of them: they are
checked by reading the rendered page, which is exactly how they get missed.

The failure that motivates this is silent under every existing check. A moderncv
\cventry renders as a tabular, so an entry is one unbreakable block; when it does
not fit in the space left, the whole entry moves to the next page and leaves a
hole behind. The document still compiles, still reports the expected page count,
and still passes tools/verify_pdf.py. Observed in the wild at 273pt, roughly 19
blank lines, mid-page, on a CV whose visual read looked fine.

tools/verify_layout.py reports per page where the text starts and stops, bottom
whitespace as a share of page height, and the largest gap between lines, then
exits 1 on a hole over 100pt, a non-final page ending more than 25% early, body
text colliding with the page-number footer, a final page more than 35% empty, or
an entry header or section heading stranded at a page break.

Page count is deliberately not checked here - verify_pdf.py --pages already does
that, and two implementations of one rule drift. Geometry comes from Poppler
pdftotext -bbox, already a dependency; a missing Poppler exits 2 with "skipped:"
rather than failing the run.

Tests build synthetic Page/Line geometry, so the suite needs neither Poppler nor
a LaTeX toolchain and runs on the existing 3.10-3.14 matrix.

* fix(layout): survive Windows encoding and a pdftotext without -bbox

Review found two failures on the repo's primary platform:

subprocess.run(..., text=True) decoded pdftotext's UTF-8 output with the
Windows ANSI codepage and crashed on the stock cv/main_example.pdf. It now
passes encoding="utf-8" with errors="replace" at the call site, matching the
fix verify_pdf.py already carries from #369.

Git for Windows ships an xpdf-based pdftotext that shadows Poppler in a
default PATH and has no -bbox flag; it exited 99, the CalledProcessError
escaped, and the run ended in exit 1 - indistinguishable from a real layout
problem, which would send /apply chasing a phantom hole. That now routes to
the existing "skipped:" exit 2 path with a message naming the likely cause,
covered by three tests on the extractor-failure path.

Also from review:

- .claude/settings.json and security_guards.py gain the verify_layout.py
  permission entries. apply.md Step 5b runs the tool on every /apply, so
  without them every run prompts.
- The docstring and CHANGELOG no longer call Poppler a dependency
  verify_pdf.py relies on. Since #369 verify_pdf prefers pypdf and Poppler is
  the fallback; word bboxes have no pypdf equivalent, so this is the one step
  that still wants it, and that is now what the text says.
- Docstring and apply.md state that the thresholds are calibrated for the
  stock moderncv and cover.cls geometry, and that the shipped example CV
  fails the thin-final-page rule by design.
- largest_gap documents that it measures top-to-top, so a tall line inflates
  the gap by its own height - over-detection, the safe direction.
- The test module imports via tools.verify_layout like test_verify_pdf.py
  instead of sys.path.insert.

* fix(verify_layout): quote only the first stderr line in the skip message (#378)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013fqqLgQSnwgWkv98twQhHi

---------

Co-authored-by: Mads Lorentzen <madslorentzen_17@hotmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-09 19:26:49 +02:00
8c81edc330 fix(scrape): make seen_jobs.json keys a pure function of the posting (#441)
* fix(scraper): make seen_jobs.json keys a pure function of the posting, and clean up the drift

The dedup key was prose only ("<url_or_company_title_key>"), so different
/scrape runs slugified company+title differently and the state file
accumulated two failures: keys carrying "&", "/", "," and ":" that break
the archive-folder path /apply and /outcome derive from company+role
(documents/README.md's subfolder rule exists because of exactly this),
and the same posting stored twice under two different truncations of a
long title (two Deloitte entries, one job, one URL).

tools/job_key.py makes the key a pure, deterministic function of
company+title+url: a strict allowlist slug, length-capped with a hash of
the full slug so truncation never collides across runs, and a fallback
to the portal's numeric job id when a non-Latin title slugifies to
nothing (a real prior entry, "securion_", would have collided with
every future non-Latin posting from that company).

--audit finds both failure classes in an existing seen_jobs.json without
guessing at a fix: malformed keys (real damage), a legacy three-part
company_title_location shape (harmless but not what the current rule
produces, so it silently re-duplicates on the next scrape), and
duplicate URLs. Ran it against this workspace's file and re-keyed the 15
entries it found - 7 malformed, 8 legacy-shape - verified byte-for-byte
against the pre-cleanup copy that no entry's data changed, only its key.

tests/test_job_key.py (16 tests) covers the slugify rules, the
truncation-hash behavior, both non-Latin fallback paths, and the audit
CLI's exit codes.

* fix(scrape): call the key helper from Step 4 instead of slugifying ad hoc

The helper added in the previous commit is only load-bearing if the spec
calls it. Step 4 described the key as prose ("<url_or_company_title_key>"),
which is what let each run slugify its own way. Step 4 now names the
command, and the schema shows the key's provenance.

Renumbers the trailing list item; no other behaviour in the step changes.

* docs(changelog): record the job-key rule under Unreleased

* fix(scrape): preserve dedup continuity across key rule

* changelog: note that existing seen_jobs.json entries need no migration (#441)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013fqqLgQSnwgWkv98twQhHi

---------

Co-authored-by: nox <nox@Mac.home>
Co-authored-by: Mads Lorentzen <madslorentzen_17@hotmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-08 16:25:15 +02:00
Mads LorentzenandClaude Fable 5.1 ccf786bdf2 changelog: move the #437 and #439 entries out of the released 1.7.1 section
Both branches predate the v1.7.1 cut, so their hunks' context pointed at the
old [Unreleased] "### Added" header, which the cut had rolled into [1.7.1].
The rebases applied cleanly and landed the entries inside the release. They
belong under [Unreleased].

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013fqqLgQSnwgWkv98twQhHi
2026-09-07 18:45:36 +02:00
Oscar Madera 6b07b13bd2 feat(commands): add GitHub repository project extraction to /expand (#437) 2026-09-07 18:43:59 +02:00
Oscar Madera 6176e6aaca feat(outcome): add stale sweep branch for batch-resolving quiet applications (#439)
* feat(outcome): add stale sweep branch for batch-resolving quiet applications

- Add /outcome stale [N] and /outcome sweep [N] to Step 0
- Offer stale sweep in Step 1.3 when rows are quiet 60+ days
- Add Step 2c Stale Sweep Branch with interactive all/select/skip confirmation
- Batch-resolve qualifying open applications to canonical no_response
- Update tracker status and append dated notes; update archive outcome.md
- Add Rule 9 to Important Rules
- Add tests/test_outcome_stale.py and update CHANGELOG.md

* test(outcome): drop auxiliary candidate filter from spec test per review
2026-09-07 18:41:16 +02:00
f8c606fb3d fix(jobnet-search): fallback to search endpoint on detail 404 for external ads (#432) (#435)
* fix(jobnet-search): fallback to search endpoint on detail 404 for external ads (#432)

* fix(jobnet-search): mark external detail fallback degraded and omit unknown fields

* changelog: add the #432 entry for the external-ad fallback

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013fqqLgQSnwgWkv98twQhHi

---------

Co-authored-by: Mads Lorentzen <madslorentzen17@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-07 18:38:15 +02:00
Instinctandnox ab5732138a fix(web-research): fail loudly when $SCRATCHPAD is unset instead of writing into the repo (#440)
* fix(web-research): fail loudly when $SCRATCHPAD is unset

The two runnable snippets in 09-web-research.md both start with
`cd "$SCRATCHPAD"`, but nothing in the repo ever sets that variable.
With it unset the command expands to `cd ""`, which succeeds and leaves
the shell in the current directory, so `page.html` and the extracted
text land wherever the command was run from. In practice that is the
repo checkout, which is exactly what the paragraph directly beneath the
curl block forbids: "Write to the session scratchpad directory, never
into the repo."

Guarding with `${SCRATCHPAD:?...}` turns a silent write into the repo
into an immediate, self-explaining failure. The message names where the
value comes from so the reader can set it and re-run.

* docs(changelog): record the $SCRATCHPAD guard under Unreleased

---------

Co-authored-by: nox <nox@Mac.home>
2026-09-07 18:34:46 +02:00
62 changed files with 3774 additions and 126 deletions

No files matched your search

+2 -1
View File
@@ -126,7 +126,8 @@ bun run .agents/skills/freehire-search/cli/src/cli.ts detail <slug|url> [--forma
`slug` is the `id` from a `search` result (e.g. `golang-zensar-2bxu6dxm`). You may
also pass a full `https://freehire.me/jobs/<slug>` URL. Returns the full (HTML-stripped)
description, skills, region/country, and — when the posting is enriched — seniority,
category, employment type, and salary.
category, employment type, and salary. `salary` keeps its period when freehire records
one (`INR 300000–300000/year`, `INR 12000/month`); compare pay only in one period.
Use it for a posting you already have a slug for — a tracked application, a shared
link, or a closed posting search no longer lists. Re-fetching a hit that `search`
@@ -107,6 +107,7 @@ export interface FreehireJob {
salary_min?: number
salary_max?: number
salary_currency?: string
salary_period?: string // e.g. "year", "month"
}
}
@@ -144,11 +145,15 @@ export interface JobDetailResult extends JobResult {
description: string | null
}
/** Reshape a freehire job into the contract search-result fields. */
/**
* Reshape a freehire job into the contract search-result fields. The title comes
* from the source ATS as indexed, which can carry HTML entities and stray
* whitespace ("Intern - Fullstack &amp; AI Innovation "), so it is decoded and trimmed.
*/
export function toResult(j: FreehireJob): JobResult {
return {
id: j.public_slug,
title: j.title || "(untitled)",
title: decodeHtmlEntities(j.title || "").trim() || "(untitled)",
company: j.company || null,
company_slug: j.company_slug || null,
location: j.location || null,
@@ -176,12 +181,17 @@ export function toDetail(j: FreehireJob): JobDetailResult {
}
}
/** Human-readable salary line from the enrichment fields, or null when absent. */
/**
* Human-readable salary line from the enrichment fields, or null when absent.
* The period is kept when freehire records one ("INR 300000–300000/year"):
* without it a yearly figure is indistinguishable from a monthly one.
*/
function formatSalary(e: FreehireJob["enrichment"]): string | null {
if (e.salary_min == null && e.salary_max == null) return null
const cur = e.salary_currency ? `${e.salary_currency} ` : ""
if (e.salary_min != null && e.salary_max != null) return `${cur}${e.salary_min}–${e.salary_max}`
return `${cur}${e.salary_min ?? e.salary_max}`
const per = e.salary_period ? `/${e.salary_period}` : ""
if (e.salary_min != null && e.salary_max != null) return `${cur}${e.salary_min}–${e.salary_max}${per}`
return `${cur}${e.salary_min ?? e.salary_max}${per}`
}
function numericEntity(cp: number): string {
@@ -49,6 +49,24 @@ describe("toResult — reshape into the portal-skill contract", () => {
expect(r.date).toBeNull();
expect(r.work_mode).toBeNull();
});
test("decodes HTML entities and trims whitespace in the title", () => {
// Real title from the index (freehire slug intern-fullstack-amp-ai-innovation-noise-edaawqq2).
expect(toResult(job({ title: "Intern - Fullstack &amp; AI Innovation " })).title).toBe(
"Intern - Fullstack & AI Innovation",
);
expect(toResult(job({ title: "R&#38;D Engineer" })).title).toBe("R&D Engineer");
expect(toResult(job({ title: "SDE Intern - Frontend " })).title).toBe("SDE Intern - Frontend");
});
test("a whitespace-only title still falls back to (untitled)", () => {
expect(toResult(job({ title: " " })).title).toBe("(untitled)");
expect(toResult(job({ title: "" })).title).toBe("(untitled)");
});
test("detail inherits the cleaned title", () => {
expect(toDetail(job({ title: "Data &amp; Analytics Intern" })).title).toBe("Data & Analytics Intern");
});
});
describe("toDetail — adds cleaned description + enrichment", () => {
@@ -68,6 +86,18 @@ describe("toDetail — adds cleaned description + enrichment", () => {
expect(d.salary).toBe("EUR 90000–120000");
});
test("keeps the salary period, so a yearly figure is not read as monthly", () => {
const d = toDetail(
job({ enrichment: { salary_min: 300000, salary_max: 300000, salary_currency: "INR", salary_period: "year" } }),
);
expect(d.salary).toBe("INR 300000–300000/year");
});
test("a single-bound salary keeps its period too", () => {
const d = toDetail(job({ enrichment: { salary_min: 12000, salary_currency: "INR", salary_period: "month" } }));
expect(d.salary).toBe("INR 12000/month");
});
test("null enrichment fields when the enrichment object is empty", () => {
const d = toDetail(job({ enrichment: {} }));
expect(d.seniority).toBeNull();
@@ -1,7 +1,7 @@
import { defineCommand, option } from "@bunli/core"
import { z } from "zod"
import { parse } from "node-html-parser"
import { BASE_URL, normalizeSlug, writeError } from "../helpers.js"
import { BASE_URL, htmlFetch, normalizeSlug, writeError } from "../helpers.js"
import { extractCity, toContractDate } from "./search.js"
interface JsonLdJobPosting {
@@ -241,26 +241,15 @@ export const detail = defineCommand({
const url = `${BASE_URL}/job/${slug}`
try {
const response = await fetch(url, {
headers: {
"Accept": "text/html,application/xhtml+xml",
"User-Agent": "Mozilla/5.0 (compatible; jobdanmark-cli/1.0)",
},
signal: AbortSignal.timeout(15000),
})
// htmlFetch carries the portal contract's 429/5xx backoff, the request
// timeout, and the shared User-Agent; a bare fetch() here had none.
const html = await htmlFetch(url)
if (response.status === 404) {
if (html === null) {
writeError("Job not found", "NOT_FOUND")
process.exit(1)
}
if (!response.ok) {
writeError(`API request failed: ${response.status} ${response.statusText}`, "API_ERROR")
process.exit(1)
}
const html = await response.text()
if (signal.aborted) return
const output = parseJobPostingFromHtml(html, slug, url)
@@ -64,6 +64,45 @@ export async function apiPost<T>(path: string, body: unknown): Promise<T> {
throw new Error("API request failed after max retries")
}
/**
* Fetch a rendered jobdanmark.dk page as text, with the same 429/5xx backoff,
* request timeout, and User-Agent as apiFetch/apiPost. `detail` reads HTML
* rather than the JSON API; it used to call fetch() directly with none of the
* three, so a rate-limited detail page failed on the first 429 while every
* other portal's detail command retried. Returns null on 404 so the caller
* keeps its own NOT_FOUND contract.
*/
export async function htmlFetch(url: string): Promise<string | null> {
const maxRetries = 6
let delay = 500
for (let attempt = 0; attempt <= maxRetries; attempt++) {
const response = await fetch(url, {
headers: {
"Accept": "text/html,application/xhtml+xml",
"User-Agent": USER_AGENT,
},
signal: AbortSignal.timeout(15000),
})
if (response.status === 429 || response.status >= 500) {
if (attempt === maxRetries) {
throw new Error(`API request failed: ${response.status} ${response.statusText}`)
}
const jitter = Math.floor(Math.random() * 500)
await new Promise((resolve) => setTimeout(resolve, delay + jitter))
delay = Math.min(delay * 2, 5000)
continue
}
if (response.status === 404) {
return null
}
if (!response.ok) {
throw new Error(`API request failed: ${response.status} ${response.statusText}`)
}
return response.text()
}
throw new Error("API request failed after max retries")
}
export function writeError(error: string, code: string): void {
process.stderr.write(JSON.stringify({ error, code }) + "\n")
}
@@ -0,0 +1,134 @@
import { afterEach, describe, expect, test } from "bun:test";
import { detail } from "../src/commands/detail";
// The portal contract requires backoff on 429/5xx, and `/scrape` calls
// `detail` once per shortlisted posting - a burst that trips the rate limiter
// is exactly when it matters. The handler used to call fetch() directly with
// no retry loop: on a 429 it wrote API_ERROR and exited after ONE attempt,
// while every other portal's detail command retried. These tests drive the
// real command handler (not the wrapper in isolation) with a stubbed fetch,
// instant timers, and process.exit turned into a throw so the exit path can
// be asserted. On the pre-fix handler the first test sees 1 call and an exit.
const originalFetch = globalThis.fetch;
const originalSetTimeout = globalThis.setTimeout;
const originalExit = process.exit;
const originalLog = console.log;
const originalStderrWrite = process.stderr.write;
afterEach(() => {
globalThis.fetch = originalFetch;
globalThis.setTimeout = originalSetTimeout;
process.exit = originalExit;
console.log = originalLog;
process.stderr.write = originalStderrWrite;
});
const JSON_LD_PAGE = `<!doctype html><html><head>
<script type="application/ld+json">{"@context":"https://schema.org","@type":"JobPosting",
"title":"Data Engineer","datePosted":"2026-09-01","hiringOrganization":{"@type":"Organization","name":"Acme"},
"description":"Build pipelines."}</script></head><body></body></html>`;
function instantTimers() {
globalThis.setTimeout = ((fn: () => void) =>
originalSetTimeout(fn, 0)) as unknown as typeof setTimeout;
}
function stubFetch(responses: Array<() => Response>): { calls: number } {
const state = { calls: 0 };
globalThis.fetch = (async () => {
const i = Math.min(state.calls, responses.length - 1);
state.calls++;
return responses[i]();
}) as unknown as typeof fetch;
return state;
}
function captureOutput(): { stdout: string[]; stderr: string[] } {
const out = { stdout: [] as string[], stderr: [] as string[] };
console.log = ((...args: unknown[]) => out.stdout.push(args.join(" "))) as typeof console.log;
process.stderr.write = ((chunk: string | Uint8Array) => {
out.stderr.push(String(chunk));
return true;
}) as typeof process.stderr.write;
return out;
}
function firstStderrJson(out: { stderr: string[] }): unknown {
const firstLine = out.stderr.join("").trim().split("\n")[0];
return JSON.parse(firstLine);
}
class ExitCalled extends Error {
constructor(public code: number | undefined) {
super(`process.exit(${code})`);
}
}
function throwingExit() {
process.exit = ((code?: number) => {
throw new ExitCalled(code);
}) as unknown as typeof process.exit;
}
async function runDetail(slug: string): Promise<{ exit: number | null }> {
const handler = (detail as unknown as { handler: (ctx: unknown) => Promise<void> }).handler;
try {
await handler({ flags: { format: "json" }, positional: [slug], signal: new AbortController().signal });
return { exit: null };
} catch (err) {
if (err instanceof ExitCalled) return { exit: err.code ?? 0 };
throw err;
}
}
describe("detail backoff on the real handler path", () => {
test("retries a 429 and returns the posting on the next attempt", async () => {
instantTimers();
throwingExit();
const out = captureOutput();
const state = stubFetch([
() => new Response("", { status: 429, statusText: "Too Many Requests" }),
() => new Response(JSON_LD_PAGE, { status: 200 }),
]);
const result = await runDetail("data-engineer-acme");
expect(result.exit).toBeNull();
expect(state.calls).toBe(2);
const parsed = JSON.parse(out.stdout.join("\n")) as { title: string; slug: string };
expect(parsed.title).toBe("Data Engineer");
expect(parsed.slug).toBe("data-engineer-acme");
expect(out.stderr.join("")).toBe("");
});
test("gives up after the initial attempt plus six retries and exits 1 with API_ERROR", async () => {
instantTimers();
throwingExit();
const out = captureOutput();
const state = stubFetch([() => new Response("", { status: 503, statusText: "Service Unavailable" })]);
const result = await runDetail("data-engineer-acme");
expect(result.exit).toBe(1);
expect(state.calls).toBe(7);
const err = firstStderrJson(out) as { code: string; error: string };
expect(err.code).toBe("API_ERROR");
expect(err.error).toMatch(/503/);
});
test("a 404 is not retried and still reports NOT_FOUND", async () => {
throwingExit();
const out = captureOutput();
const state = stubFetch([() => new Response("", { status: 404 })]);
const result = await runDetail("gone");
expect(result.exit).toBe(1);
expect(state.calls).toBe(1);
// The handler's own catch block sees the throwing process.exit stub and
// writes a second line - a test artifact, not CLI behaviour. The first
// stderr line is the contract.
expect(firstStderrJson(out)).toEqual({ error: "Job not found", code: "NOT_FOUND" });
});
});
@@ -1,5 +1,5 @@
import { afterEach, describe, expect, test } from "bun:test";
import { apiFetch, apiPost } from "../src/helpers";
import { apiFetch, apiPost, htmlFetch } from "../src/helpers";
// A stalled upstream connection (accepted socket, no response) would otherwise
// hang the CLI forever - fetch has no default timeout. Assert both request
@@ -21,6 +21,17 @@ describe("request timeout", () => {
expect(init?.signal).toBeInstanceOf(AbortSignal);
});
test("htmlFetch passes an AbortSignal timeout to fetch", async () => {
let init: RequestInit | undefined;
globalThis.fetch = (async (_url: string | URL | Request, i?: RequestInit) => {
init = i;
return new Response("<html></html>", { status: 200 });
}) as unknown as typeof fetch;
await htmlFetch("https://jobdanmark.dk/job/x");
expect(init?.signal).toBeInstanceOf(AbortSignal);
});
test("apiPost passes an AbortSignal timeout to fetch", async () => {
let init: RequestInit | undefined;
globalThis.fetch = (async (_url: string | URL | Request, i?: RequestInit) => {
@@ -1,11 +1,13 @@
import { afterEach, describe, expect, test } from "bun:test";
import { apiFetch, apiPost } from "../src/helpers";
import { apiFetch, apiPost, htmlFetch } from "../src/helpers";
// The portal contract requires backoff on 429/5xx. These tests pin the retry
// loop offline: a stubbed fetch counts attempts, and a stubbed setTimeout
// fires immediately so the exhaustion case does not sleep through the real
// 500ms -> 5s backoff schedule. apiFetch and apiPost carry separate copies of
// the loop, so both are exercised to keep them from drifting apart.
// 500ms -> 5s backoff schedule. apiFetch, apiPost, and htmlFetch carry separate
// copies of the loop, so all three are exercised to keep them from drifting
// apart. htmlFetch is the one `detail` uses: before it existed, detail called
// fetch() directly and a 429 failed on the first attempt (1 call, not 7).
const originalFetch = globalThis.fetch;
const originalSetTimeout = globalThis.setTimeout;
@@ -33,6 +35,7 @@ function stubFetch(responses: Array<() => Response>): { calls: number } {
const wrappers: Array<[string, () => Promise<{ ok: boolean }>]> = [
["apiFetch", () => apiFetch<{ ok: boolean }>("/x")],
["apiPost", () => apiPost<{ ok: boolean }>("/x", {})],
["htmlFetch", () => htmlFetch("https://jobdanmark.dk/job/x").then((html) => ({ ok: html !== null }))],
];
for (const [name, call] of wrappers) {
@@ -65,3 +68,12 @@ for (const [name, call] of wrappers) {
});
});
}
describe("htmlFetch 404", () => {
test("returns null without retrying so detail keeps its NOT_FOUND contract", async () => {
const state = stubFetch([() => new Response("", { status: 404 })]);
expect(await htmlFetch("https://jobdanmark.dk/job/missing")).toBeNull();
expect(state.calls).toBe(1);
});
});
@@ -1,5 +1,5 @@
import { afterEach, describe, expect, test } from "bun:test";
import { apiFetch, apiPost, USER_AGENT } from "../src/helpers";
import { apiFetch, apiPost, htmlFetch, USER_AGENT } from "../src/helpers";
// Bun's fetch injects an anonymous default User-Agent (Bun/1.3.10) when code
// sets none. This CLI should say who is asking, in the honest style jobindex
@@ -46,3 +46,17 @@ describe("apiPost user agent", () => {
expect(headerValue(init?.headers, "Content-Type")).toBe("application/json");
});
});
describe("htmlFetch user agent", () => {
test("sends the shared User-Agent and asks for HTML", async () => {
let init: RequestInit | undefined;
globalThis.fetch = (async (_url: string | URL | Request, i?: RequestInit) => {
init = i;
return new Response("<html></html>", { status: 200 });
}) as unknown as typeof fetch;
await htmlFetch("https://jobdanmark.dk/job/x");
expect(headerValue(init?.headers, "User-Agent")).toBe(USER_AGENT);
expect(headerValue(init?.headers, "Accept")).toContain("text/html");
});
});
@@ -60,23 +60,38 @@ function stripTags(html: string): string {
}
/**
* Extract job ID from URL or return as-is if already an ID
* Parse a detail invocation's <id|url> into a canonical fetch target, or null.
*
* This is the gate between a stored (untrusted) URL and a network fetch, so it
* must never trust the raw string: the previous version fetched any http(s)
* URL verbatim and, when the path didn't match, used the whole input URL as
* the id - a non-posting page (a redirect target, a look-alike host, the
* homepage) came back as a well-formed fake posting with exit 0 (#447). A URL
* input now needs a jobindex.dk host (apex or subdomain) and a
* /jobannonce/<id> path, and the fetch URL is rebuilt from the extracted id -
* the canonical short form the bare-id path always used. A bare id stays a
* permissive scheme- and slash-free token (the jobnet precedent): the server
* 404s unknowns loudly, which is the honest failure. Exported for tests.
*/
function extractIdFromUrl(url: string): string {
// Match IDs like h1647303, r13677312, etc.
const match = url.match(/\/jobannonce\/([a-zA-Z]\d+)/)
if (match) return match[1]
return url
}
function buildUrl(idOrUrl: string): { url: string; id: string } {
if (idOrUrl.startsWith("http")) {
const id = extractIdFromUrl(idOrUrl)
return { url: idOrUrl, id }
export function buildUrl(idOrUrl: string): { url: string; id: string } | null {
const trimmed = idOrUrl.trim()
if (/^https?:\/\//i.test(trimmed)) {
let host: string
try {
host = new URL(trimmed).hostname.toLowerCase()
} catch {
return null
}
if (host !== "jobindex.dk" && !host.endsWith(".jobindex.dk")) return null
// Match IDs like h1647303, r13677312, etc.
const match = trimmed.match(/\/jobannonce\/([a-zA-Z]\d+)/)
if (!match) return null
return { url: `${BASE_URL}/jobannonce/${match[1]}`, id: match[1] }
}
// It's a bare ID
const url = `${BASE_URL}/jobannonce/${idOrUrl}`
return { url, id: idOrUrl }
if (/^[a-zA-Z0-9_-]+$/.test(trimmed)) {
return { url: `${BASE_URL}/jobannonce/${trimmed}`, id: trimmed }
}
return null
}
const DANISH_MONTHS: Record<string, string> = {
@@ -262,7 +277,15 @@ export const detail = defineCommand({
process.exit(1)
}
const { url, id } = buildUrl(idArg)
const parsed = buildUrl(idArg)
if (!parsed) {
writeError(
`Could not parse a jobindex job id or jobannonce URL from "${idArg}"`,
"BAD_ID",
)
process.exit(1)
}
const { url, id } = parsed
try {
const html = await htmlFetch(url)
@@ -0,0 +1,52 @@
import { describe, expect, test } from "bun:test";
import { buildUrl } from "../src/commands/detail";
// buildUrl is the gate between a stored (untrusted) URL and a network fetch.
// It must yield a canonical jobindex fetch target or null (-> BAD_ID) - never
// the raw input. The unguarded version fetched any http(s) URL verbatim and,
// when the path didn't match, used the whole input URL as the id, so a
// non-posting page came back as a well-formed fake posting with exit 0 (#447).
describe("jobindex detail input parsing", () => {
test("canonical URL with title slug", () => {
expect(buildUrl("https://www.jobindex.dk/jobannonce/h1647303/senior-data-engineer")).toEqual({
url: "https://www.jobindex.dk/jobannonce/h1647303",
id: "h1647303",
});
});
test("trailing slash and query string variants", () => {
expect(buildUrl("https://www.jobindex.dk/jobannonce/r13677312/")?.id).toBe("r13677312");
expect(buildUrl("https://www.jobindex.dk/jobannonce/h1647303?utm_source=x")?.id).toBe("h1647303");
});
test("jobindex subdomains and bare apex are accepted", () => {
expect(buildUrl("https://it.jobindex.dk/jobannonce/h1647303")?.id).toBe("h1647303");
expect(buildUrl("https://jobindex.dk/jobannonce/h1647303")?.id).toBe("h1647303");
});
test("a bare id builds the canonical URL (server 404s unknowns loudly)", () => {
expect(buildUrl("h1647303")).toEqual({
url: "https://www.jobindex.dk/jobannonce/h1647303",
id: "h1647303",
});
});
test("an off-host URL is rejected, not fetched", () => {
expect(buildUrl("https://evil.example/jobannonce/h1647303")).toBeNull();
});
test("look-alike and userinfo hosts are rejected", () => {
expect(buildUrl("https://jobindex.dk.evil.example/jobannonce/h1647303")).toBeNull();
expect(buildUrl("https://www.jobindex.dk@evil.example/jobannonce/h1647303")).toBeNull();
});
test("an own-host URL without a jobannonce id is rejected (the fake-posting repro)", () => {
expect(buildUrl("https://www.jobindex.dk/")).toBeNull();
});
test("garbage bare input is rejected", () => {
expect(buildUrl("not a slug!")).toBeNull();
expect(buildUrl("ftp://www.jobindex.dk/jobannonce/h1")).toBeNull();
});
});
@@ -1,6 +1,7 @@
import { defineCommand, option } from "@bunli/core"
import { z } from "zod"
import { apiFetch, normalizeJobId, writeError, stripHtml } from "../helpers.js"
import type { JobAdRaw, SearchApiResponse } from "./search.js"
export interface DetailApiResponse {
id: string
@@ -8,13 +9,14 @@ export interface DetailApiResponse {
body: string
publicationDateTime: string
unpublicationDateTime: string | null
approvalStatus: string
views: number
approvalStatus: string | null
views: number | null
createdDateTime: string
updatedDateTime: string
isAnonymousEmployer: boolean
isAnonymousEmployer: boolean | null
hasLogo: boolean
logoUrl: string | null
isExternal?: boolean
employer: {
cvrNumber: string | null
pNumber: string | null
@@ -22,7 +24,7 @@ export interface DetailApiResponse {
hasCompanyLogo: boolean
}
job: {
type: string
type: string | null
address: {
streetName: string | null
city: string | null
@@ -31,21 +33,21 @@ export interface DetailApiResponse {
countryCode: string
countryName: string
}
noFixedWorkplace: boolean
isLimitedPeriod: boolean
isDisabilityFriendly: boolean
isPartTime: boolean
noFixedWorkplace: boolean | null
isLimitedPeriod: boolean | null
isDisabilityFriendly: boolean | null
isPartTime: boolean | null
employmentDate: string | null
conceptUriDa: string | null
preferredLabelDa: string | null
driversLicenses: unknown[]
classifications: unknown[]
shifts: unknown[]
isFavorite: boolean
isFavorite: boolean | null
}
application: {
deadlineDate: string | null
availablePositions: number
availablePositions: number | null
contactPersons: Array<{
firstNames: string | null
lastName: string | null
@@ -53,12 +55,73 @@ export interface DetailApiResponse {
}>
url: string | null
urlText: string | null
isApplicationDeadlineASAP: boolean
isApplicationDeadlineASAP: boolean | null
}
organisationTypeId: number | null
user: string | null
}
/**
* Maps a raw JobAd from the search endpoint to a DetailApiResponse.
* Used as a fallback when /FindJob/JobAdDetails/<id> returns 404 for external ads (#432).
*/
export function mapSearchAdToDetail(raw: JobAdRaw & { jobAdUrl?: string | null; jobAnnouncementTypeName?: string | null }): DetailApiResponse {
const street = raw.workPlaceAddress ? raw.workPlaceAddress.trim() : null
return {
id: raw.jobAdId,
title: raw.title,
body: raw.description ?? "",
publicationDateTime: raw.publicationDate ?? "",
unpublicationDateTime: null,
approvalStatus: null,
views: null,
createdDateTime: raw.publicationDate ?? "",
updatedDateTime: raw.publicationDate ?? "",
isAnonymousEmployer: null,
hasLogo: Boolean(raw.hasLogo),
logoUrl: raw.logoUrl ?? null,
isExternal: true,
employer: {
cvrNumber: raw.cvr ?? null,
pNumber: null,
name: raw.hiringOrgName ?? "",
hasCompanyLogo: Boolean(raw.hasLogo),
},
job: {
type: raw.jobAnnouncementTypeName ?? (raw.workHourPartTime != null ? (raw.workHourPartTime ? "PartTime" : "FullTime") : null),
address: {
streetName: street && street.length > 0 ? street : null,
city: raw.postalDistrictName ?? raw.municipality ?? null,
postalCode: raw.postalCode ? String(raw.postalCode) : null,
municipality: raw.municipality ?? null,
countryCode: raw.country === "Danmark" ? "DK" : (raw.country || "DK"),
countryName: raw.country || "Danmark",
},
noFixedWorkplace: null,
isLimitedPeriod: null,
isDisabilityFriendly: null,
isPartTime: raw.workHourPartTime != null ? Boolean(raw.workHourPartTime) : null,
employmentDate: null,
conceptUriDa: raw.conceptUriDa ?? null,
preferredLabelDa: raw.occupation ?? null,
driversLicenses: [],
classifications: [],
shifts: [],
isFavorite: raw.isFavorite != null ? Boolean(raw.isFavorite) : null,
},
application: {
deadlineDate: raw.applicationDeadline ?? null,
availablePositions: null,
contactPersons: [],
url: raw.jobAdUrl && raw.jobAdUrl.trim().length > 0 ? raw.jobAdUrl.trim() : null,
urlText: null,
isApplicationDeadlineASAP: raw.applicationDeadlineStatus ? raw.applicationDeadlineStatus === "NotDisclosed" : null,
},
organisationTypeId: null,
user: null,
}
}
/**
* Normalize a raw detail response before any output format sees it.
*
@@ -99,30 +162,53 @@ export const detail = defineCommand({
process.exit(1)
}
let data: DetailApiResponse | null = null
try {
const data = prepareDetail(
data = prepareDetail(
await apiFetch<DetailApiResponse>(`/FindJob/JobAdDetails/${id}`, {
incrementViews: "false",
}),
)
if (signal.aborted) return
if (flags.format === "json") {
console.log(JSON.stringify(data, null, 2))
} else if (flags.format === "table") {
outputTable(data)
} else {
outputPlain(data)
}
} catch (err) {
const message = err instanceof Error ? err.message : String(err)
if (message.includes("404") || message.includes("Not Found")) {
writeError("Job ad not found", "NOT_FOUND")
// Fallback for external ads: JobAdDetails returns 404 for ads with isExternal: true,
// but /FindJob/Search returns the full ad object including HTML description (#432).
try {
const searchResult = await apiFetch<SearchApiResponse>("/FindJob/Search", {
searchString: id,
resultsPerPage: "5",
pageNumber: "1",
orderType: "PublicationDate",
})
const match = searchResult.jobAds?.find((ad) => ad.jobAdId === id)
if (match) {
process.stderr.write("note: detail endpoint returned 404; retrieved external posting summary from search endpoint\n")
data = prepareDetail(mapSearchAdToDetail(match))
}
} catch {
// If fallback search fails, fall through to NOT_FOUND
}
if (!data) {
writeError("Job ad not found", "NOT_FOUND")
process.exit(1)
}
} else {
writeError(message, "API_ERROR")
process.exit(1)
}
process.exit(1)
}
if (signal.aborted || !data) return
if (flags.format === "json") {
console.log(JSON.stringify(data, null, 2))
} else if (flags.format === "table") {
outputTable(data)
} else {
outputPlain(data)
}
},
})
@@ -131,13 +217,13 @@ function outputTable(data: DetailApiResponse): void {
console.log(`ID: ${data.id}`)
console.log(`Title: ${data.title}`)
console.log(`Employer: ${data.employer.name}`)
console.log(`Type: ${data.job.type}`)
console.log(`Type: ${data.job.type ?? "-"}`)
console.log(`City: ${data.job.address.city ?? "-"}`)
console.log(`Postal: ${data.job.address.postalCode ?? "-"}`)
console.log(`Country: ${data.job.address.countryName}`)
console.log(`Published: ${data.publicationDateTime}`)
console.log(`Deadline: ${data.application.deadlineDate ?? "-"}`)
console.log(`Positions: ${data.application.availablePositions}`)
console.log(`Positions: ${data.application.availablePositions ?? "-"}`)
console.log(`Apply URL: ${data.application.url ?? "-"}`)
}
@@ -152,7 +238,7 @@ export function formatDetailPlain(data: DetailApiResponse): string {
`Location: ${data.job.address.city ?? "-"}, ${data.job.address.countryName}`,
`Published: ${data.publicationDateTime}`,
`Deadline: ${data.application.deadlineDate ?? "-"}`,
`Positions: ${data.application.availablePositions}`,
`Positions: ${data.application.availablePositions ?? "-"}`,
]
if (data.application.url) {
@@ -0,0 +1,88 @@
import { describe, expect, test } from "bun:test"
import { mapSearchAdToDetail } from "../src/commands/detail"
import type { JobAdRaw } from "../src/commands/search"
describe("mapSearchAdToDetail (Issue #432 external ad fallback)", () => {
const sampleAd: JobAdRaw & { jobAdUrl?: string; jobAnnouncementTypeName?: string } = {
jobAdId: "ext-123",
title: "AI Technical Artist",
hiringOrgName: "Tactile Games",
occupation: "Programmør og systemudvikler",
conceptUriDa: "http://data.star.dk/esco/occupation/8b6456a3-ae9a-45a0-a65b-fed797521753",
jobAnnouncementTypeName: "Almindelige vilkår",
workHourPartTime: false,
jobAdUrl: "https://job-boards.eu.greenhouse.io/tactilegames/jobs/4890782101",
hasLogo: true,
logoUrl: "/bff/logo/123",
workPlaceAddress: " Trekronergade 26 ",
cvr: "32319882",
description: "<p>Great job opening at Tactile.</p>",
applicationDeadline: "2026-12-05T00:00:00+01:00",
applicationDeadlineStatus: "ExpirationDate",
country: "Danmark",
municipality: "København",
postalCode: 2500,
postalDistrictName: "Valby",
publicationDate: "2026-09-05T00:00:00+02:00",
isExternal: true,
isSeen: false,
isFavorite: false,
}
test("maps all key fields correctly to DetailApiResponse format", () => {
const detail = mapSearchAdToDetail(sampleAd)
expect(detail.id).toBe("ext-123")
expect(detail.title).toBe("AI Technical Artist")
expect(detail.body).toBe("<p>Great job opening at Tactile.</p>")
expect(detail.publicationDateTime).toBe("2026-09-05T00:00:00+02:00")
expect(detail.isExternal).toBe(true)
expect(detail.views).toBeNull()
expect(detail.approvalStatus).toBeNull()
expect(detail.isAnonymousEmployer).toBeNull()
expect(detail.employer.name).toBe("Tactile Games")
expect(detail.employer.cvrNumber).toBe("32319882")
expect(detail.employer.hasCompanyLogo).toBe(true)
expect(detail.job.type).toBe("Almindelige vilkår")
expect(detail.job.address.streetName).toBe("Trekronergade 26")
expect(detail.job.address.city).toBe("Valby")
expect(detail.job.address.postalCode).toBe("2500")
expect(detail.job.address.municipality).toBe("København")
expect(detail.job.address.countryCode).toBe("DK")
expect(detail.job.address.countryName).toBe("Danmark")
expect(detail.job.isPartTime).toBe(false)
expect(detail.job.noFixedWorkplace).toBeNull()
expect(detail.job.isLimitedPeriod).toBeNull()
expect(detail.job.isDisabilityFriendly).toBeNull()
expect(detail.job.preferredLabelDa).toBe("Programmør og systemudvikler")
expect(detail.job.conceptUriDa).toBe("http://data.star.dk/esco/occupation/8b6456a3-ae9a-45a0-a65b-fed797521753")
expect(detail.application.deadlineDate).toBe("2026-12-05T00:00:00+01:00")
expect(detail.application.availablePositions).toBeNull()
expect(detail.application.url).toBe("https://job-boards.eu.greenhouse.io/tactilegames/jobs/4890782101")
expect(detail.application.isApplicationDeadlineASAP).toBe(false)
})
test("handles empty or whitespace address gracefully", () => {
const detail = mapSearchAdToDetail({
...sampleAd,
workPlaceAddress: " ",
postalDistrictName: null,
municipality: null,
postalCode: null,
})
expect(detail.job.address.streetName).toBeNull()
expect(detail.job.address.city).toBeNull()
expect(detail.job.address.postalCode).toBeNull()
expect(detail.job.address.municipality).toBeNull()
})
test("flags undisclosed deadline as ASAP", () => {
const detail = mapSearchAdToDetail({
...sampleAd,
applicationDeadlineStatus: "NotDisclosed",
})
expect(detail.application.isApplicationDeadlineASAP).toBe(true)
})
})
@@ -5,15 +5,42 @@ export interface DetailOpts {
format: "json" | "plain"
}
/** Accept a raw job ID, a job-view URL, or a job URN. */
const LINKEDIN_HOST = /(^|\.)linkedin\.com$/i
/**
* Accept a raw job ID, a job URN, a LinkedIn job-view URL (any linkedin.com
* host, with or without scheme), or a bare title slug ending in the ID.
*
* A URL on any other host is rejected. The previous pattern took the first
* 6+-digit path segment from ANY URL, so a Greenhouse or Lever apply link -
* the kind a posting's own page hands out - fetched whatever LinkedIn
* posting happened to carry that number and printed it with exit 0. Host and
* path are read through real URL parsing, so look-alike hosts
* (linkedin.com.evil.io) and userinfo tricks (linkedin.com@evil.io) fall on
* the reject side too.
*/
export function normalizeId(input: string): string | null {
const urn = input.match(/urn:li:jobPosting:(\d+)/)
const trimmed = input.trim()
const urn = trimmed.match(/urn:li:jobPosting:(\d+)/)
if (urn) return urn[1]
const url = input.match(/-(\d{6,})(?:[\/?]|$)/) || input.match(/\/(\d{6,})(?:[\/?]|$)/)
if (url) return url[1]
const bare = input.match(/^\d{6,}$/)
if (bare) return input
return null
if (/^\d{6,}$/.test(trimmed)) return trimmed
const hasScheme = /^https?:\/\//i.test(trimmed)
if (hasScheme || /^[a-z0-9.-]+\.[a-z]{2,}(\/|$)/i.test(trimmed)) {
let parsed: URL
try {
parsed = new URL(hasScheme ? trimmed : `https://${trimmed}`)
} catch {
return null
}
if (!LINKEDIN_HOST.test(parsed.hostname)) return null
const path = parsed.pathname.match(/\/jobs\/view\/(?:[^/]*-)?(\d{6,})\/?$/)
return path ? path[1] : null
}
// A scheme- and slash-free slug such as "software-engineer-1234567890".
const slug = trimmed.match(/^[a-z0-9-]*-(\d{6,})$/i)
return slug ? slug[1] : null
}
export async function runDetail(opts: DetailOpts): Promise<number> {
@@ -0,0 +1,72 @@
import { afterEach, describe, expect, test } from "bun:test";
import { runDetail } from "../src/commands/detail";
import { DETAIL_URL } from "../src/helpers";
import { runCLI } from "./helpers";
// The real command path, not the parser in isolation: `detail` must never
// send a request for a URL that is not a LinkedIn job view. Before the host
// check, a Greenhouse apply link pasted from a posting was reduced to its
// 7-digit path segment and the handler fetched LinkedIn posting 4567890 -
// an unrelated job, printed with exit 0. fetch is stubbed throughout; no
// test here touches the network.
const originalFetch = globalThis.fetch;
const originalStderrWrite = process.stderr.write;
afterEach(() => {
globalThis.fetch = originalFetch;
process.stderr.write = originalStderrWrite;
});
function recordingFetch(): string[] {
const requested: string[] = [];
globalThis.fetch = (async (input: string | URL | Request) => {
requested.push(String(input));
return new Response("", { status: 404 });
}) as unknown as typeof fetch;
return requested;
}
function captureStderr(): string[] {
const out: string[] = [];
process.stderr.write = ((chunk: string | Uint8Array) => {
out.push(String(chunk));
return true;
}) as typeof process.stderr.write;
return out;
}
describe("detail input on the real handler path", () => {
test("a foreign job-board URL is rejected with BAD_ID before any request", async () => {
const requested = recordingFetch();
const stderr = captureStderr();
const rc = await runDetail({ id: "https://boards.greenhouse.io/acme/jobs/4567890", format: "json" });
expect(rc).toBe(1);
expect(requested).toEqual([]);
expect(JSON.parse(stderr.join("").trim())).toMatchObject({ code: "BAD_ID" });
});
test("a LinkedIn job URL builds the fetch URL from the extracted id", async () => {
const requested = recordingFetch();
captureStderr();
const rc = await runDetail({
id: "https://dk.linkedin.com/jobs/view/data-scientist-9876543210/?trackingId=x",
format: "json",
});
expect(rc).toBe(1); // the stub answers 404 -> NOT_FOUND, after the request was made
expect(requested).toEqual([`${DETAIL_URL}/9876543210`]);
});
});
describe("detail input through the CLI", () => {
test("exits 1 with the stderr-JSON BAD_ID contract for a foreign URL", async () => {
const result = await runCLI(["detail", "https://jobs.lever.co/acme/1234567"]);
expect(result.exitCode).toBe(1);
expect(JSON.parse(result.stderr)).toMatchObject({ code: "BAD_ID" });
});
});
@@ -267,6 +267,30 @@ describe("normalizeId", () => {
expect(normalizeId("https://dk.linkedin.com/jobs/view/data-scientist-9876543210/")).toBe("9876543210");
});
test("accepts a scheme-less linkedin.com job URL and a bare title slug", () => {
expect(normalizeId("www.linkedin.com/jobs/view/1234567890")).toBe("1234567890");
expect(normalizeId("software-engineer-1234567890")).toBe("1234567890");
});
test("rejects a job URL on any other host instead of extracting its digits", () => {
// The old pattern took the first 6+-digit path segment from any URL, so an
// ATS apply link fetched an unrelated LinkedIn posting with that number.
expect(normalizeId("https://boards.greenhouse.io/acme/jobs/4567890")).toBeNull();
expect(normalizeId("https://jobs.lever.co/acme/1234567")).toBeNull();
expect(normalizeId("https://example.com/jobs/view/1234567890")).toBeNull();
});
test("rejects look-alike and userinfo hosts", () => {
expect(normalizeId("https://linkedin.com.evil.io/jobs/view/1234567890")).toBeNull();
expect(normalizeId("https://notlinkedin.com/jobs/view/1234567890")).toBeNull();
expect(normalizeId("https://www.linkedin.com@evil.io/jobs/view/1234567890")).toBeNull();
});
test("rejects a linkedin.com URL that is not a job view", () => {
expect(normalizeId("https://www.linkedin.com/in/someone-1234567890/")).toBeNull();
expect(normalizeId("https://www.linkedin.com/jobs/search/?currentJobId=1234567890")).toBeNull();
});
test("returns null for non-job URLs and invalid strings", () => {
expect(normalizeId("https://www.linkedin.com/feed/")).toBeNull();
expect(normalizeId("not-a-url")).toBeNull();
+6 -4
View File
@@ -75,9 +75,11 @@ Collect:
1. **Name** - short kebab-case identifier (e.g. `awesome-cv`, `classic-serif`). Must not collide with an existing folder in `templates/`.
2. **Source extension** - the main file's extension (`.tex`, `.typ`, ...), inferred from the provided source file.
3. **Compile command** - the full command `/apply` and Step 4's test compile will run, using `<file>` (no extension) as the placeholder for the output basename:
- **`.tex` source**: infer the engine the same way as before - if the source uses `fontspec` or loads font files by path, it requires `xelatex` or `lualatex`; tell the user this rather than letting them pick `pdflatex`. Render as `lualatex -interaction=nonstopmode <file>.tex` (or the appropriate engine).
- **`.tex` source**: infer the engine the same way as before - if the source uses `fontspec` or loads font files by path, it requires `xelatex` or `lualatex`; tell the user this rather than letting them pick `pdflatex`. Render as `rm -f <file>.pdf && mkdir -p build && lualatex -interaction=nonstopmode -output-directory=build <file>.tex && mv build/<file>.pdf ./` (or the appropriate engine).
- **`.typ` source**: default to `typst compile <file>.typ <file>.pdf` - Typst has a single binary, no engine choice.
- **Anything else**: no built-in guidance; ask the user for the exact compile command.
**Build directory rule:** if the toolchain can redirect its intermediate files, the declared command sends them to a `build/` folder beside the source, and still leaves the PDF next to the source, where `/apply` reads it. For LaTeX, use `-output-directory=build`. It also moves the PDF, hence the `mv`. TeX Live does not create the directory, hence the `mkdir -p`. A failed compile skips the `mv`, hence the leading `rm -f`, so a failed LaTeX compile never leaves a stale PDF where `/apply` inspects it. After a failed compile, read the log in `build/`. If Step 4 fails only because of the redirect (e.g. `\include` from a subfolder), drop the redirect but keep the leading `rm -f`, and record why under "Known pitfalls". If the toolchain writes nothing but the PDF (`typst compile`) or has no redirect option, keep the command as it is - never block registration on this rule.
4. **Fonts** - which font(s) the template uses and where they come from:
- **Bundled font files** (`.ttf`/`.otf` shipped with the template): copy them into the template folder in Step 3 and record the relative path used to load them (LaTeX `\fontspec` `Path`, Typst `#import`/font path, or equivalent).
- **System / distribution fonts**: record the font name and note that the user's machine must have it installed.
@@ -113,7 +115,7 @@ Write into it:
## Compile command
cd <output dir> && <the full declared command, e.g. lualatex -interaction=nonstopmode <file>.tex or typst compile <file>.typ <file>.pdf>
cd <output dir> && <the full declared command, e.g. rm -f <file>.pdf && mkdir -p build && lualatex -interaction=nonstopmode -output-directory=build <file>.tex && mv build/<file>.pdf ./ or typst compile <file>.typ <file>.pdf>
## Style rules
@@ -138,7 +140,7 @@ Never register a template without a successful test compile. Templates that "loo
```
3. If the compile fails: show the user the relevant error lines, diagnose (missing font file, wrong engine/command, missing class or package), fix what you can (e.g. font path values), and re-compile. If the fix needs input only the user has (a missing font file, a license-restricted class), ask for it and wait.
4. On success, confirm a PDF was produced and Read it to check the layout renders sensibly (no overlapping text, fonts loaded, page count matches the declared page limit for the dummy content). Record any surprises in the manifest's "Known pitfalls".
5. Delete the scratch source file, the scratch PDF, and any other intermediate files the compile command produced (LaTeX toolchains typically leave `_compile_test.aux`/`.log`/`.out`/`.fls`/`.fdb_latexmk`/`.synctex.gz`; other toolchains may leave nothing beyond the PDF — check what actually landed in the folder and remove all `_compile_test.*` byproducts).
5. Delete the scratch source file, the scratch PDF, any `build/` folder a test compile created, and any other `_compile_test.*` byproduct left in the template folder.
Do not proceed to Step 5 until the test compile passes.
@@ -161,7 +163,7 @@ Insert (or replace, if one exists) this block immediately after the file's H1 ti
> - **Template skeleton:** `templates/<type>/<name>/template<source-extension>` — use this as the structural reference instead of the stock template
> - **Manifest:** `templates/<type>/<name>/TEMPLATE.md` — read this for style rules and known pitfalls before drafting
> - **Source extension:** `<source-extension>` (not `.tex` unless the template's own toolchain is LaTeX)
> - **Compile command:** `<the full declared command>` (not the command named in the stock guidance below — `/apply`'s compile step must use this instead)
> - **Compile command:** `<the full declared command>` (not the command named in the stock guidance below — `/apply`'s compile step must use this instead). Run it from the output directory. If it redirects to `build/`, the log is in `build/`, and `/apply`'s Step 5e cleanup must delete that `build/` folder too
> - **Fonts:** <font summary, including any path note for bundled fonts>
> - **Page limit:** exactly <N> page(s)
> - **Output file:** `cv/main_<company>_<role><source-extension>` / `cover_letters/cover_<company>_<role><source-extension>`; copy any class/package/font files the template needs into the output directory, or reference them by relative path
+45 -8
View File
@@ -44,13 +44,29 @@ python salary_lookup.py "<Company Name>" --json
If the posting specifies a city, add `--city "<City>"` to narrow results. Parse the JSON output and include the salary benchmark in the evaluation. If the tool is not configured or returns an error, skip the salary benchmark.
### Source Host Verification (when input is a URL)
Before proceeding to drafting, inspect the posting URL's hostname to verify provenance (#431). Classify the host into one of three categories:
1. **Installed portal board:** the host matches any configured job portal in `.agents/skills/` (e.g. `jobindex.dk`, `linkedin.com`, `jobnet.dk`, `jobbank.dk`, `jobdanmark.dk`, `freehire.me`, or any portal added by `/add-portal`).
2. **Known official ATS apex:** the host matches or is a valid subdomain of one of the six standard ATS domains:
- `greenhouse.io`
- `lever.co`
- `myworkdayjobs.com` (or `workday.com`)
- `ashbyhq.com`
- `smartrecruiters.com`
- `workable.com`
*Look-alike parsing:* the host must match the apex exactly or end with `.<apex>`. Look-alike prefix tricks (e.g. `evil-greenhouse.io`), suffix spoofing (e.g. `job-boards.greenhouse.io.evil.com`), userinfo tricks (`https://greenhouse.io@evil.com/`), and unfamiliar subdomains fail closed and must not be classified as an official ATS.
3. **Neither (Unverified host):** name the host plainly in the evaluation output as unverified (`⚠ Unverified source host: <hostname> - not an installed portal board or known ATS apex`). Alert the user to verify the employer and link legitimacy before committing time and tokens to drafting.
Present the evaluation to the user with:
1. **Skills match** - which required/preferred skills match vs. gaps
2. **Experience match** - how work history maps to the role
3. **Behavioral/culture match** - how behavioral profile fits the role/company culture
4. **Salary benchmark** - salary index for the company (if available)
5. **Overall fit score** and recommendation (strong fit / moderate fit / weak fit)
1. **Source host verification** - installed portal board, official ATS, or ⚠ unverified source host (named plainly)
2. **Skills match** - which required/preferred skills match vs. gaps
3. **Experience match** - how work history maps to the role
4. **Behavioral/culture match** - how behavioral profile fits the role/company culture
5. **Salary benchmark** - salary index for the company (if available)
6. **Overall fit score** and recommendation (strong fit / moderate fit / weak fit)
After presenting the evaluation, ask the user:
> "Should I proceed with drafting the CV and cover letter for this role?"
@@ -229,7 +245,26 @@ If either compile fails, fix the error and re-compile until clean.
### 5b. Inspect layout
Read both PDFs via the Read tool and verify:
**Measure first, then look.** A visual read catches gross breakage but cannot tell you that a page is 40% empty, and the failure below survives both a clean compile and a correct page count:
```bash
python tools/verify_pdf.py cv/main_<company>_<role>.pdf --pages 2
python tools/verify_pdf.py cover_letters/cover_<company>_<role>.pdf --pages 1
python tools/verify_layout.py cv/main_<company>_<role>.pdf
python tools/verify_layout.py cover_letters/cover_<company>_<role>.pdf
```
The two `--pages` lines are the page-count check: exactly 2 pages for the CV and exactly 1 for the cover letter (the hard limits in `05-cv-templates.md` and `06-cover-letter-templates.md`), exit 1 otherwise. With a custom template active, substitute its declared **Page limit** from the `ACTIVE-TEMPLATE` block. Nothing else runs this check - `verify_layout.py` deliberately leaves page count to it, and Step 5d's extraction call passes no `--pages` - so if these lines are skipped, the page budget is enforced by nothing but the visual read below.
The layout script reports, per page, where the text starts and stops, bottom whitespace as a share of page height, and the largest vertical gap between lines. It exits 1 on: a hole over 100pt (~7 blank lines), a non-final page ending more than 25% early, body text colliding with the page-number footer, a final page more than 35% empty, and an entry header or section heading stranded at a page break. Page count is **not** checked here — that is `verify_pdf.py --pages`'s job, and the two `--pages` lines above run it.
The hole check is the one a visual read misses. A moderncv `\cventry` renders as a `tabular`, so it is an **unbreakable block**: when it does not fit in the space left, the whole entry jumps to the next page and leaves a hole behind, while the document still compiles and still reports the right page count. Fix it by shortening the entry that follows the hole, not by stretching the page.
If Poppler is missing, or the `pdftotext` first in PATH is the xpdf build Git for Windows ships (no `-bbox`), the script exits 2 with `skipped:` — note the degraded mode in the Step 6 report and rely on the visual inspection alone. Exit 2 is never a layout verdict.
The thresholds are calibrated for the stock moderncv and `cover.cls` geometry; a template registered via `/add-template` may report a phantom hole above a footer the 90pt band does not cover.
Then read both PDFs via the Read tool and verify:
**CV (`cv/main_<company>_<role>.pdf`):**
- [ ] Exactly 2 pages (not 1, not 3)
@@ -263,9 +298,11 @@ An ATS parser reads the PDF's embedded **text layer**, not the rendered page —
**1. Extract the text layer:**
```bash
python tools/verify_pdf.py cv/main_<company>_<role>.pdf --dump-text cv/main_<company>_<role>.txt
python tools/verify_pdf.py cv/main_<company>_<role>.pdf --ascii-dates --dump-text cv/main_<company>_<role>.txt
```
`--ascii-dates` is the date-range check from `05-cv-templates.md` ("Date fields must be ASCII ranges"): it exits 1 if the raw text layer has a year joined to a Unicode dash - the en-dash LaTeX makes from `--`, which a Workday import dropped together with the date - naming each hit and its code point. It reads the raw layer, not the folded comparison `--contains` uses, which is why `--contains "2016-2024"` cannot catch this. The dump is written before the check runs, so a failure still leaves the `.txt`. Fix a hit in the `<CV_EXT>` date argument (a single ASCII hyphen), then re-run 5a-5c and re-extract.
The command prints `extractor: pypdf` or `extractor: pdftotext`. Record that name in the Step 6 report. Read the `.txt` file. If that tool is unavailable, the Poppler fallback is:
```bash
@@ -277,7 +314,7 @@ cd cv && pdftotext -layout -enc UTF-8 main_<company>_<role>.pdf main_<company>_<
- [ ] **Text extracted at all**, with no garbage runs: no `(cid:NNN)` markers, no `�` replacement characters, no stretches of missing text that are visible in the PDF
- [ ] **Email and phone survive as literal text.** Icon fonts extract as glyph names (the stock template's contact line extracts as `MOBILE-ALT [+XX ...] • Envelope [your.email@...]`) — that noise is harmless, but the actual address and digits must be present. A contact detail carried only by an icon or a hyperlink target (like the `LinkedIn` link text) is invisible to an ATS; the email must be printed as text.
- [ ] **Reading order matches the visual order** — section headings appear in the same sequence as on the page, and lines from different sections are not interleaved. The stock banking template is single-column and safe; custom templates registered via `/add-template` with sidebars or multi-column layouts are where this breaks.
- [ ] **Dates recognizable** — each role and degree has its years present in the extraction.
- [ ] **Dates recognizable and ASCII-joined** — each role and degree has its years present in the extraction with a start *and* an end, and `--ascii-dates` above exited 0. A bare single year (`2016`) is the half the flag cannot see: it imports as a start with no end, so read for it.
Failures here are template-level problems: fix them in the `<CV_EXT>` source (e.g. print the email as text rather than icon-only), then re-run 5a–5c and re-extract. If a custom template's layout fundamentally scrambles extraction order, tell the user prominently — they may be trading ATS compatibility for looks.
+16 -2
View File
@@ -55,6 +55,7 @@ Look up the GitHub username from `01-candidate-profile.md`. If a GitHub URL or u
2. For each repository found:
- Fetch the repository README
- Note: name, description, primary language(s), topics/tags, any frameworks or libraries mentioned in the README
- If the repository represents an independent technical project (not an empty stub or uncustomized fork), extract a project summary (problem domain, tech stack, and demonstrable technical results) for consideration under Independent Projects
3. Also retrieve the full repository list if available (to catch unpinned repos)
If no GitHub username or URL is found in the profile, skip this source and note it was skipped.
@@ -108,19 +109,25 @@ After enriching all items, build a deduplicated competency map. Group findings i
**Domain Knowledge** (subject matter expertise: geophysics, ML, NLP, etc.)
**Methods and Practices** (agile, version control, reproducibility, testing, etc.)
**Soft / Behavioral** (leadership, communication, collaboration signals from references and project descriptions)
**Independent Projects & Portfolio** (distinct technical projects from GitHub with problem domain, tech stack, and key technical milestone)
For each competency, record:
- The competency name
- The source item it came from (e.g. "Coursera — Deep Learning Specialisation", "GitHub — repo-name", "Reference letter — Jens Jensen")
- Whether it came from direct lookup (A), inference (B), or both
For each project, record:
- Project name
- One-line summary: problem tackled, tech stack used, and verifiable outcome/impact
- Source (e.g. "GitHub — repo-name")
Remove anything already present in `01-candidate-profile.md` or `02-behavioral-profile.md`.
---
## Step 4: Present Grouped Summary
Present all new competencies for the user's review before writing anything. Format:
Present all new competencies and project additions for the user's review before writing anything. Format:
```
## /expand found [N] new competency signals across [M] sources
@@ -131,6 +138,11 @@ Source: [Course/cert name — Provider]
+ [Competency 2]
...
**PROJECTS & PORTFOLIO**
Source: [GitHub — repo-name]
+ [Project Name]: [Problem, stack, and outcome]
...
**GITHUB — [repo-name]**
Source: README + inferred from tech stack
+ [Competency 1]
@@ -169,6 +181,7 @@ Wait for the user's response before writing anything.
Apply only the confirmed items. Use the Edit tool to add to the relevant sections of each file — do not rewrite entire files.
### Additions to `01-candidate-profile.md`
- Independent projects → append to the `## Independent Projects` section formatted as `- **[Project Name]**: [Description with stack and outcome] *(GitHub — repo-name)*`
- Technical skills (primary and secondary) → append to the Technical Skills section
- Domain knowledge → append to the Domain Knowledge or Technical Skills section (match the existing structure)
- Methods and practices → append appropriately
@@ -189,7 +202,7 @@ After writing, present:
## /expand Complete
### Added to 01-candidate-profile.md
[List each competency added, with source]
[List each competency and independent project added, with source]
### Added to 02-behavioral-profile.md
[List each behavioral signal added, with source]
@@ -214,3 +227,4 @@ After writing, present:
- **User confirms before writing.** The full competency map is shown and confirmed before a single file is touched.
- **Behavioral signals are labeled.** Anything inferred from tone, language, or indirect signals is marked as inferred so it is reviewed critically.
- **GitHub is fully scanned.** All public repositories are checked, not just pinned ones — unpinned repos often contain significant competency signals.
- **Portfolio & projects grounded in code.** Independent projects added to the profile must reflect real projects found in public GitHub repositories — never fabricated project claims.
+3 -1
View File
@@ -124,7 +124,9 @@ Approving the whole batch in one reply is expected UX - the requirement is that
For every row the user approved:
1. **Tracker (`job_search_tracker.csv`):** update the matched row's `status` column per the Step 5 table, and append to `notes`: `<date> gmail-sync: <signal> ("<email subject>")`. Never restructure the CSV, reorder rows, or touch unrelated rows - same rule `/outcome` follows. The rewrite touches only `status`, `notes` (and `date` when the drafted-rule below fires): preserve every other field of the row, parsed or not, so the `deadline` column written by `/apply` Step 6b - or any column added in the future - is never blanked by a status sync.
1. **Tracker (`job_search_tracker.csv`):** update the matched row's `status` column per the Step 5 table, and append to `notes`: `<date> gmail-sync: <signal> ("<email subject>")`, **with every comma, double quote and line break deleted from the subject first**. No writer here emits a quoted tracker field and no reader unquotes one, so an unescaped comma splits the row identically for a naive split and for the `csv.DictReader` the shipped reader actually uses (`tools/rank_state.py`): `cv_file`, `cover_letter_file` and `source` each shift a column left. A line break is worse - it ends the row and starts a second one. The double quote is stripped as cheap insurance for the day something does quote a field; on today's readers it is harmless. The subject is a human-readable breadcrumb here, not data anything reads back - item 2 below keeps it verbatim in `outcome.md`, which is Markdown and carries no such constraint. This matters more than it looks: `/gmail-sync` is the only tracker writer that copies *third-party* text, and the only one that runs unattended, so nobody is watching the row it edits.
Never restructure the CSV, reorder rows, or touch unrelated rows - same rule `/outcome` follows. The rewrite touches only `status`, `notes` (and `date` when the drafted-rule below fires): preserve every other field of the row, parsed or not, so the `deadline` column written by `/apply` Step 6b - or any column added in the future - is never blanked by a status sync.
**If the matched row was still `drafted`,** also set `date` to the email's date. The employer replying proves the user submitted by hand without running `/outcome`, so the drafting date now in that column is wrong. The email's date is an upper bound on the real submission date, tight for an ack and loose for a rejection weeks later, which is why Step 6 shows it and lets the user supply the actual date instead.
2. **`outcome.md`:** tick the relevant stage checkbox (adding the date in parentheses) or update `Status`/`Date resolved` per the table. Append a dated entry to `## Notes`, never overwrite existing Notes history:
+1 -1
View File
@@ -25,7 +25,7 @@ v1 preps for a **specific application**. Generic no-target practice is out of sc
- `job_posting.md` - the exact posting the user applied to
- `cv_draft.tex` and `cover_letter.tex` - what was actually submitted. **These are what the interviewer read**; every talking point must be consistent with their claims.
- `outcome.md` - the stage reached so far and any recorded feedback from earlier stages. Feedback from stage N is the highest-value input for stage N+1 prep.
2. **Fallbacks** (the application may predate `/outcome`): posting via WebFetch on the tracker row's `source` URL, or ask the user to paste it; CV via `cv/main_<company>*.tex` and cover letter via `cover_letters/cover_<company>_*.tex`. State plainly which context is missing rather than guessing - and suggest `/outcome <company>` to build the archive for next time.
2. **Fallbacks** (the application may predate `/outcome`): posting via WebFetch on the tracker row's `source` URL, or ask the user to paste it; CV via `cv/main_<company>_<role>.*` and cover letter via `cover_letters/cover_<company>_<role>.*`, deriving `<company>_<role>` by the **Subfolder naming** rule in `documents/README.md`. **Never widen those globs to the company alone**: with two roles at one company it would prep you from the sibling role's documents. State plainly which context is missing rather than guessing - and suggest `/outcome <company>` to build the archive for next time.
3. **Ask the user what this interview is** (skip anything `outcome.md` already records): stage (phone screen / technical / case / final round), date, format (phone, video, onsite), and who is interviewing (names and titles, if known).
4. **Read the frameworks once** - do not re-read them in later steps:
- `.claude/skills/job-application-assistant/07-interview-prep.md`
+51 -3
View File
@@ -22,6 +22,8 @@ Follow these steps **in order**.
- `followup` → enter the follow-up branch (Step 2b) over every quiet open application, using the default threshold of **10 days**
- `followup <N>`, e.g. `/outcome followup 14` → follow-up branch with an N-day threshold
- `followup <company>`, e.g. `/outcome followup acme` → draft a follow-up for that application now, regardless of threshold
- `stale` or `sweep` → enter the stale application sweep branch (Step 2c) over open applications quiet for **60+ days**
- `stale <N>` or `sweep <N>`, e.g. `/outcome stale 90` → stale sweep branch with an N-day threshold
---
@@ -33,7 +35,7 @@ Follow these steps **in order**.
```
**If the file exists and its header does not end in `,deadline`, append `,deadline` to the header line only** - no data row is touched. Legacy rows then read as an empty deadline. This is the one edit to an existing tracker this command may make outside a matched row, and Step 4's "never restructure the CSV" governs that row, not this header line.
2. **With an argument:** match rows case-insensitively on company (and role, if given). One match → proceed. Several → list them and ask. None → the application was made outside the workflow; collect company, role, date applied, channel, and posting URL from the user and add a tracker row.
3. **Without an argument:** list all rows whose status is not final (see **Tracker status vocabulary** below) as a numbered table (company, role, date applied, current status, deadline, days quiet, follow-ups sent) and ask which to update. The two derived columns come straight from existing data: **days quiet** counts from the row's `date` or the latest dated entry in `notes`, whichever is more recent; **follow-ups sent** counts the `followed up YYYY-MM-DD` markers in `notes`. If any open row is 10+ days quiet with fewer than two follow-ups sent, add one line under the table: "Some of these have gone quiet - want a follow-up draft? (Step 2b)". If every row is resolved, say so and stop.
3. **Without an argument:** list all rows whose status is not final (see **Tracker status vocabulary** below) as a numbered table (company, role, date applied, current status, deadline, days quiet, follow-ups sent) and ask which to update. The two derived columns come straight from existing data: **days quiet** counts from the row's `date` or the latest dated entry in `notes`, whichever is more recent; **follow-ups sent** counts the `followed up YYYY-MM-DD` markers in `notes`. If any open row is 10+ days quiet with fewer than two follow-ups sent, add one line under the table: "Some of these have gone quiet - want a follow-up draft? (Step 2b)". If any open rows are 60+ days quiet, also offer: "You have applications quiet for 60+ days — run `/outcome stale` to batch-resolve them (Step 2c)." If every row is resolved, say so and stop.
**`drafted` rows are listed but never counted as quiet** - nothing was sent, so nobody is late replying. List them under their own heading ("Drafted, not yet submitted"), leave **days quiet** and **follow-ups sent** blank, and keep them out of the follow-up offer above.
@@ -110,11 +112,56 @@ If the user decides not to send, log nothing.
---
## Step 2c: Stale Sweep Branch (batch-resolve quiet applications)
Enter this branch from the `stale` or `sweep` argument (Step 0), or from the suggestion under the open-pipeline table in Step 1.3. In an extended job hunt, applications that received no response accumulate and clutter the tracker, `/html-report` funnel metrics, and `/notion-sync`. This branch operationalizes batch-cleaning old quiet applications while keeping the user in full control.
**Candidates.** An application qualifies when its tracker `status` is open and submitted (`applied` or `interview`), the threshold has passed since its `date` (or since the latest dated entry in `notes`, whichever is more recent), and its status is neither final nor `drafted` (`drafted` applications were never submitted and cannot receive a response). Parse dates defensively — skip unparseable rows with a note.
**Threshold.** The default threshold is **60 days** quiet. If the user specified an integer `<N>` (e.g. `/outcome stale 90` or `/outcome sweep 45`), use N days instead.
**Presentation.** If no open applications exceed the threshold, report:
> "No open applications exceed the <N>-day quiet threshold. Your tracker is up to date!"
and stop.
Otherwise, present qualifying applications as a numbered table:
```
## Stale Applications ([K] quiet for [N]+ days)
| # | Company | Role | Date Applied | Days Quiet | Follow-ups Sent | Current Status | Proposed Status |
|---|---------|------|--------------|------------|-----------------|----------------|-----------------|
| 1 | Acme | SWE | 2026-05-10 | 118 | 2 | applied | no_response |
| 2 | Beta | MLE | 2026-06-01 | 96 | 1 | applied | no_response |
```
Then ask:
> **How would you like to resolve these applications?**
>
> - **`all`** — Mark all [K] applications as `no_response` and update archives
> - **`select`** — Specify which numbers to resolve (e.g. "1, 3" or "1-4")
> - **`skip`** — Cancel without making any changes
Wait for the user's explicit response before writing anything.
**Execution.** For each application the user confirms:
1. **Update Tracker:** update the row's `status` column to `no_response` (using the canonical spelling from **Tracker status vocabulary**). Append `stale resolved no_response (YYYY-MM-DD)` to `notes`. Follow Step 4's rule: never restructure the CSV, preserve all other columns intact.
2. **Update Archive:** derive `documents/applications/<company>_<role>/` per the **Subfolder naming** rule. If the folder exists, update or write `outcome.md` with:
- `**Status:** no_response`
- `**Date resolved:** YYYY-MM-DD`
- Append to `## Notes`: `- Stale resolution: marked no_response after [N] days quiet (YYYY-MM-DD)`
**Calibration Handoff.** If 3 or more applications were resolved in this sweep, continue to Step 5 to offer calibration handoff. Otherwise present a summary of resolved applications and stop.
---
## Step 3: Archive the Application Materials
Create or update `documents/applications/<company>_<role>/`. All content here is personal data - the folder is already gitignored (`documents/applications/**`), so nothing needs redacting.
1. **`cv_draft.tex` and `cover_letter.tex`** - copy (never move) the submitted files. Locate them via the tracker row's `cv_file`/`cover_letter_file` columns; if those are empty, look for `cv/main_<company>*.tex` and `cover_letters/cover_<company>_*.tex`. If a file already exists in the archive, leave it - the archived version is what was actually submitted. If no draft files exist (application made outside `/apply`), skip with a note.
1. **`cv_draft.tex` and `cover_letter.tex`** - copy (never move) the submitted files. Locate them via the tracker row's `cv_file`/`cover_letter_file` columns; if those are empty, look for `cv/main_<company>_<role>.*` and `cover_letters/cover_<company>_<role>.*`, deriving `<company>_<role>` by the **Subfolder naming** rule in `documents/README.md`. **Never widen those globs to the company alone** - two roles at one company both match it, and the first hit wins silently. If a file already exists in the archive, leave it - the archived version is what was actually submitted. If nothing matches (application made outside `/apply`), skip with a note rather than widening the search: a sibling role's CV recorded as what you submitted is worse than no file at all.
2. **`job_posting.md`** - if it already exists, leave it. Otherwise try WebFetch on the tracker row's `source` URL and save the posting text, retrying a 403 with browser headers per `.claude/skills/job-application-assistant/09-web-research.md`. If the URL is dead (postings expire fast - this is exactly why the archive matters), ask the user to paste the posting, or write a stub noting the posting is unavailable. **Never reconstruct a posting from memory.**
3. **`outcome.md`** - write or update it in exactly the format documented in `documents/README.md`, so `/setup` Path A parses it without special cases:
@@ -145,7 +192,7 @@ Update rules: tick stage checkboxes as they are reached (add the date in parenth
## Step 4: Update the Tracker
Update the matched row's `status` column using the canonical spellings from **Tracker status vocabulary** above (e.g. `drafted` → `applied` → `interview` → `offer` → `hired` / `rejected` / `no_response` / `offer_declined` / `withdrawn`) and append a short dated note to the `notes` column. Never restructure the CSV, reorder rows, or touch other rows. The rewrite touches only the `status` and `notes` columns: preserve every other field of the row, parsed or not, so a value the row carries - the `deadline` written by `/apply` Step 6b, or any column added in the future - is never blanked by a status update.
Update the matched row's `status` column using the canonical spellings from **Tracker status vocabulary** above (e.g. `drafted` → `applied` → `interview` → `offer` → `hired` / `rejected` / `no_response` / `offer_declined` / `withdrawn`) and append a short dated note to the `notes` column, **containing no commas, double quotes or line breaks**. No writer here emits a quoted tracker field, so a comma in the note shifts `cv_file`, `cover_letter_file` and `source` a column left for `csv.DictReader` (`tools/rank_state.py`) as much as for a naive split, and a line break ends the row - `rejected, no feedback given` is exactly the sentence that corrupts it; write `rejected - no feedback given`. `/gmail-sync` Step 7a applies the same rule to the email subjects it appends. Never restructure the CSV, reorder rows, or touch other rows. The rewrite touches only the `status` and `notes` columns: preserve every other field of the row, parsed or not, so a value the row carries - the `deadline` written by `/apply` Step 6b, or any column added in the future - is never blanked by a status update.
**Moving a row off `drafted`:** rows written by `/apply` Step 6b carry the date the documents were drafted, not the date they were sent. Whenever this step advances such a row to any other status - `applied`, or straight to `interview` or `rejected` when the user reports an outcome for something they submitted without recording it - overwrite its `date` column with the actual submission date. The `date` column is read as "applied on" by `/notion-sync` and drives `/html-report`'s year/season grouping and this command's own days-quiet count, so leaving the draft date in place would misreport the application.
@@ -193,3 +240,4 @@ If the recorded status is `hired`, congratulate the user warmly first - this is
6. **Follow-ups: draft only, never send.** The follow-up branch produces text for the user to send themselves. It never emails, messages, or submits anything, and it must not be wired to tools that do.
7. **Follow-ups: no new claims.** Every substantive statement in a follow-up or thank-you note comes from the archived submitted materials. Rule 3 applies with no exceptions.
8. **Maximum two follow-ups per application.** After the second silent follow-up, the honest move is recording the resolution, not persistence.
9. **Stale sweep: user confirms before writing.** The stale sweep branch never marks applications as no_response automatically. It always presents the qualifying candidate list and waits for explicit user confirmation (all, select, or skip).
+6 -2
View File
@@ -20,7 +20,7 @@ If `$ARGUMENTS` is empty or does not contain a recognized scope keyword, ask:
>
> - **`profile`** — Clears candidate data from the skill files (profile, behavioral, STAR examples, profile statements, personalized evaluation criteria, search queries). The framework structure, scoring framework, and writing rules are preserved. Use this to re-run `/setup` from scratch.
>
> - **`documents`** — Deletes all files you've placed in the `documents/` folder (CV PDFs, LinkedIn export, diplomas, references, pasted job postings, past applications). The folder structure and `README.md` are preserved.
> - **`documents`** — Deletes all files you've placed in the `documents/` folder (CV PDFs, LinkedIn export, diplomas, references, project summaries, pasted job postings, past applications). The folder structure and `README.md` are preserved.
>
> - **`all`** — Both of the above.
>
@@ -86,7 +86,7 @@ cv/main_example.tex. This scope covers skill files only.
### If scope includes `documents`:
Use Glob to list all files present in `documents/cv/`, `documents/linkedin/`, `documents/diplomas/`, `documents/references/`, `documents/postings/`, and `documents/applications/`. Present as:
Use Glob to list all files present in `documents/cv/`, `documents/linkedin/`, `documents/diplomas/`, `documents/references/`, `documents/projects/`, `documents/postings/`, and `documents/applications/`. Present as:
```
## Documents reset will delete:
@@ -103,6 +103,9 @@ documents/diplomas/
documents/references/
- [filename] or "(empty)"
documents/projects/
- [filename] or "(empty)"
documents/postings/
- [filename] or "(empty)"
@@ -246,6 +249,7 @@ rm -f documents/cv/*
rm -f documents/linkedin/*
rm -f documents/diplomas/*
rm -f documents/references/*
rm -f documents/projects/*
rm -f documents/postings/*
rm -rf documents/applications/*/
```
+9 -4
View File
@@ -29,7 +29,7 @@ visibility cannot be determined — warn now and wait:
Wait for the user's confirmation before showing the path prompt. A private origin, no
origin, or a non-fork remote needs no warning — continue silently.
Then, before greeting the user, scan the `documents/` folder. Use Glob with `documents/**/*` and count files per subfolder (`cv/`, `linkedin/`, `diplomas/`, `references/`, `applications/`).
Then, before greeting the user, scan the `documents/` folder. Use Glob with `documents/**/*` and count files per subfolder (`cv/`, `linkedin/`, `diplomas/`, `references/`, `projects/`, `applications/`).
Then welcome the user with a single message that lists three paths. The wording changes based on what was found.
@@ -57,7 +57,7 @@ Then welcome the user with a single message that lists three paths. The wording
>
> Three ways to start:
>
> **Path A: Documents folder** (best signal if you have several materials) - Drop your CV / LinkedIn export / diplomas / reference letters in the `documents/` folder, then say "go". I'll read everything and build your profile from it. See `documents/README.md` for the folder layout.
> **Path A: Documents folder** (best signal if you have several materials) - Drop your CV / LinkedIn export / diplomas / reference letters / project summaries in the `documents/` folder, then say "go". I'll read everything and build your profile from it. See `documents/README.md` for the folder layout.
>
> **Path B: Single CV import** - Paste or @-mention a single CV/resume here. I'll extract it and ask follow-up questions for what's missing.
>
@@ -86,6 +86,7 @@ Use Glob with `documents/**/*` to scan the full tree. Print:
**linkedin/**: [list files, or "(empty)"]
**diplomas/**: [list files, or "(empty)"]
**references/**: [list files, or "(empty)"]
**projects/**: [list files, or "(empty)"]
**applications/**: [list subfolders with their files, or "(empty)"]
I will read these and cross-reference before proposing any changes.
@@ -109,7 +110,7 @@ Hold this content in context throughout Path A. Do not re-read.
### Step A3: Parse Documents
Read each document found in Step A1. Process subfolders in this order: `cv/`, `linkedin/`, `diplomas/`, `references/`, `applications/`.
Read each document found in Step A1. Process subfolders in this order: `cv/`, `linkedin/`, `diplomas/`, `references/`, `projects/`, `applications/`.
**`cv/` documents:** name, contact (email, phone, LinkedIn, GitHub), education (degree, institution, dates, thesis), work experience (title, company, dates, location, bullets), skills, languages (with any stated proficiency), publications, awards, profile/summary.
@@ -119,6 +120,8 @@ Read each document found in Step A1. Process subfolders in this order: `cv/`, `l
**`references/` documents:** referee name, title, organization; full text of the letter (extract specific quotes); competency language used.
**`projects/` documents:** project name, summary/description, problem domain, tech stack (languages, frameworks, tools), key technical challenges and architectural decisions, measurable outcomes/metrics (e.g. users, performance, stars, impact).
**`applications/<company>_<role>/` subfolders:**
- `job_posting.md`: role title, company, required skills, experience level, sector, role type
- `cover_letter.tex`: opening structure, body structure, bullet style, closing, recurring phrases
@@ -157,12 +160,13 @@ If no inconsistencies, state "No cross-reference issues found." and continue.
For each skill file, compare extracted document content against the current file content from Step A2. Build two buckets.
**Additive changes:** entirely new content not in the skill file in any form. Examples: a certification not in `01-candidate-profile.md`, a new endorsement skill, a referee not yet listed, a new behavioral quote from a reference letter, a new award.
**Additive changes:** entirely new content not in the skill file in any form. Examples: a certification not in `01-candidate-profile.md`, a new independent project not in `01-candidate-profile.md`, a new endorsement skill, a referee not yet listed, a new behavioral quote from a reference letter, a new award.
**Conflicting changes:** content that touches something already in a skill file but disagrees. Examples: a different date range for an existing job, a different job title for the same role, a different graduation date than what is recorded.
**Inference rules** (apply when populating from inferred sources):
- **`01-candidate-profile.md` (`## Independent Projects`):** Source is `projects/` documents. Extract structured project entries formatted as `- **[PROJECT_NAME]**: [DESCRIPTION with tech stack and measurable outcome]`. Ground all claims in the document text.
- **`02-behavioral-profile.md`:** Source is LinkedIn About + recommendation letters. Extract recurring themes, adjectives, phrases about how the candidate works. Add only to "Strongest Behavioral Traits", "How [Candidate] Works Best", or "Management Style Preferences" sections. Do not overwrite existing scored assessments. Always label inferred additions: *[Inferred from LinkedIn About / Reference letter - review before relying on this]*
- **`03-writing-style.md`:** Source is `cover_letter.tex` files. Extract recurring patterns. Add as observations under "## Patterns Observed in Past Applications". Do not modify existing rules. Only add if 2+ cover letters show a genuine pattern.
- **`04-job-evaluation.md`:** Source is `job_posting.md` + `outcome.md` pairs. If an application reached interview or offer: note role type and sector as a confirmed strong-fit signal. If 2+ applications repeat a no-response or rejection pattern: note it. Add findings under "## Calibration from Past Applications". Do not modify the existing scoring framework.
@@ -193,6 +197,7 @@ Present the full change set before writing anything.
### 01-candidate-profile.md
- [ ] New certification: [title], [issuer], [date] - extracted from LinkedIn
- [ ] New independent project: [PROJECT_NAME] - [description, tech stack, key outcome]
- [ ] New reference: [name, title, company]
Quote: "[relevant quote]"
+4
View File
@@ -12,8 +12,12 @@
"Bash(python3 salary_lookup.py:*)",
"Bash(python tools/rank_state.py:*)",
"Bash(python3 tools/rank_state.py:*)",
"Bash(python tools/job_key.py:*)",
"Bash(python3 tools/job_key.py:*)",
"Bash(python tools/verify_pdf.py:*)",
"Bash(python3 tools/verify_pdf.py:*)",
"Bash(python tools/verify_layout.py:*)",
"Bash(python3 tools/verify_layout.py:*)",
"Bash(pdftotext:*)"
]
}
@@ -1,5 +1,5 @@
---
framework_version: 1.4.3
framework_version: 1.4.5
---
# CV Templates and Tailoring Guide
@@ -42,6 +42,13 @@ Expected output: `Output written on main_<company>_<role>.pdf (2 pages, ...)`. A
\renewcommand*{\sectionstyle}[1]{{\sectionfont\color{color1}#1}}
\usepackage[utf8]{inputenc}
% pdflatex fallback only (the documented engine is lualatex, which skips this
% branch). Without T1 font encoding pdflatex builds accented letters with
% \accent, and the PDF text layer stores them decomposed - `e` + U+0300 rather
% than U+00E8 - so an ATS keyword match on "Genève" fails while the page looks
% right. moderncv 2.5 loads T1 itself under pdflatex; 2.3.1 (Debian/Ubuntu apt)
% does not. \ifpdftex comes from iftex, which every moderncv version loads.
\ifpdftex\usepackage[T1]{fontenc}\fi
% moderncv loads hyperref itself in an \AtEndPreamble hook, so \hypersetup
% must go in an \AtEndPreamble of our own: on moderncv < 2.4 a top-level
% \usepackage{hyperref} clashes with the class's own
@@ -277,7 +284,8 @@ What to check in the extraction:
- **Contact details as literal text.** The stock template's fontawesome contact icons extract as glyph names (`MOBILE-ALT`, `Envelope`) - harmless noise, because the actual address and number are printed beside them. The failure mode is a contact detail carried *only* by an icon or a hyperlink (like the `LinkedIn` link text, whose URL is not in the text layer): invisible to an ATS. The email address must always appear as printed text.
- **No garbled output.** `(cid:NNN)` markers or `�` characters mean a font is embedded without a Unicode mapping - an ATS sees the same garbage. This shows up with unusual fonts in custom templates, not with the stock moderncv setup under lualatex.
- **Reading order.** The stock banking style is single-column, so extraction order matches visual order. Custom templates (via `/add-template`) with sidebars or multi-column layouts can interleave unrelated lines; if extraction order is scrambled, the user is trading ATS compatibility for looks and should be told.
- **Keyword coverage.** Match the posting's required/preferred terms against the extracted text, in the posting's language. Prefer the posting's exact term over a synonym when it is truthfully applicable - ATS matching is often literal. Never add a keyword the profile does not support.
- **Keyword coverage.** Match the posting's required/preferred terms against the extracted text, in the posting's language. Prefer the posting's exact term over a synonym when it is truthfully applicable - ATS matching is often literal. Never add a keyword the profile does not support. `verify_pdf.py --contains` folds both sides for whitespace, Unicode normalization (NFC) and LaTeX's typographic substitutions before comparing - `'` reaches the text layer as U+2019 and `--` as U+2013, so `--contains "Master's degree"` and `--contains "2016-2024"` match what the template actually renders. The dumped `.txt` is never folded: it is the raw layer the ATS sees, which is why the date-range check below reads the dump, not `--contains`.
- **Accents intact (pdflatex fallback).** Under pdflatex without T1 font encoding the text layer stores accented letters decomposed (`e` + combining grave instead of `è`); pypdf reads that as `Gen` `eve` with a stray spacing accent, and neither form matches a typed keyword. The stock template guards this with `\ifpdftex\usepackage[T1]{fontenc}\fi`; keep the line in tailored CVs and custom templates that may be compiled with pdflatex. It is a no-op under lualatex.
### Date fields must be ASCII ranges (confirmed ATS import failure)
@@ -302,7 +310,7 @@ Two independent causes, both easy to avoid:
Where a genuine range exists, use it even when a single year would be factually accurate - a degree written `1995` is true but imports worse than `1992-1995`. Do not invent a start date you do not have; a lone graduation year is fine, just expect it to be typed in by hand.
**Add this to the step 5d checks**: after extracting the text layer, confirm every experience entry shows a start *and* an end separated by an ASCII hyphen. Because the failure is silent and invisible in the PDF, the candidate otherwise discovers it only while filling in the application form.
**Cause 1 is mechanical in step 5d**: `python tools/verify_pdf.py cv/main_<company>_<role>.pdf --ascii-dates` scans the raw text layer for a year joined to a Unicode dash and exits 1 naming each hit with its code point (`U+2013 in 'Role Title 2016–2024'`). It deliberately reads the raw layer: the fold that lets `--contains "2016-2024"` match what the template renders maps that same en-dash back to `-`, so `--contains` can never see this. A year on either side of the dash is enough, so `Mar 2016 – Jul 2016` and `2016 – Present` are caught too; a numeric range with no year (`EUR 600k–1M`) is left alone. **Cause 2 stays a read-through check**: after extracting the text layer, confirm every experience entry shows a start *and* an end. Because both failures are silent and invisible in the PDF, the candidate otherwise discovers them only while filling in the application form.
## Page Budget - Hard 2-Page Limit
@@ -1,5 +1,5 @@
---
framework_version: 1.1.0
framework_version: 1.1.1
---
# Web Research and Fetching
@@ -48,7 +48,7 @@ Two details worth knowing, both covered by `tests/test_robots_check.py`:
### The retry: curl with browser headers
```bash
cd "$SCRATCHPAD" && curl -sSL --max-time 45 -o page.html -w "HTTP %{http_code} size=%{size_download}\n" \
cd "${SCRATCHPAD:?set this to the session scratchpad directory from your system prompt}" && curl -sSL --max-time 45 -o page.html -w "HTTP %{http_code} size=%{size_download}\n" \
-H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36' \
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8' \
-H 'Accept-Language: en-GB,en;q=0.9' \
@@ -65,7 +65,7 @@ Write to the session scratchpad directory, never into the repo. `--compressed` i
`WebFetch` converts to markdown for you; curl does not. Strip the tags:
```bash
cd "$SCRATCHPAD" && python3 -c "
cd "${SCRATCHPAD:?set this to the session scratchpad directory from your system prompt}" && python3 -c "
import re, html
h = open('page.html', encoding='utf-8', errors='replace').read()
h = re.sub(r'(?is)<(script|style|noscript|svg)[^>]*>.*?</\1>', ' ', h)
+17 -5
View File
@@ -5,7 +5,7 @@ description: >
(LinkedIn, local job boards, and any skills added with /add-portal). Deduplicates
across runs. Triggers on: job scrape, find jobs, search jobs, new jobs, job search,
scrape jobs, /scrape
allowed-tools: Read, Write, Edit, Glob, Grep, Bash(bun --version), Bash(bun run .agents/skills/*/cli/src/cli.ts *), WebFetch, WebSearch, Agent, AskUserQuestion
allowed-tools: Read, Write, Edit, Glob, Grep, Bash(bun --version), Bash(bun run .agents/skills/*/cli/src/cli.ts *), Bash(python tools/job_key.py:*), Bash(python3 tools/job_key.py:*), WebFetch, WebSearch, Agent, AskUserQuestion
---
# Job Scraper
@@ -117,7 +117,10 @@ site for the role and store that URL instead, or drop the candidate rather than
fragment link.
For every candidate:
- Skip if the URL or company+title combo already exists in `seen_jobs.json`
- Skip if the URL matches any existing `seen_jobs.json` entry, regardless of
that entry's key. This preserves dedup continuity for postings stored under
the pre-helper key rule while new entries use the canonical key from Step 4.
- Otherwise, skip if the company+title combo already exists in `seen_jobs.json`
- Skip if the company+role already appears in `job_search_tracker.csv`
### Step 2.5: Mass-Posting Detection (within this run)
@@ -138,11 +141,19 @@ For each new job, do a rapid fit check (NOT the full evaluation from `04-job-eva
### Step 4: Deduplicate & Store
1. Add ALL fetched jobs (new and skipped) to `seen_jobs.json` with structure:
1. Derive each entry's key with the helper, never by slugifying in the moment:
```bash
python3 tools/job_key.py --company "<company>" --title "<title>" --url "<url>"
```
It prints one line: the canonical key for that posting. The key must be a pure function of the posting, because two runs that slugify differently store the same job twice and defeat the dedup this step exists to provide. The helper also length-caps long titles and disambiguates the cap with a hash of the full slug, so a truncated title is stable across runs and two different long titles never collide. `python3 tools/job_key.py --audit` reports entries in an existing state file that predate this rule; it only reports, and never rewrites keys, since a rewritten key breaks the tracker's own company+role matching.
2. Add ALL fetched jobs (new and skipped) to `seen_jobs.json` with structure:
```json
{
"seen": {
"<url_or_company_title_key>": {
"<key from tools/job_key.py>": {
"title": "...",
"company": "...",
"url": "...",
@@ -168,7 +179,8 @@ The `source` field records which mechanism produced the entry: `cli` for Step 1b
`posted_date` is the posting's own publication date, taken from the `date` field Step 2's contract already guarantees on every portal CLI's search output. Step 1b uses that date to scope the run to the last 14 days and then drops it, so nothing downstream can distinguish a posting published yesterday from one published two years ago - `first_seen` is when this scraper first saw the entry, not when the employer posted it. Persisting it makes Step 1b's window auditable after the run and gives `/rank` a freshness signal to weigh, instead of rediscovering the date and recording it in prose that nothing reads. That gap landed for real: a freehire-search posting dated 2024-05-13 was scraped and ranked Strong Fit at position 1 of 133, its own scoring note observing the listing "may be long stale" with nothing able to act on it. `null` means the portal returned no date for that result (the CLIs emit `date: null` when a listing omits it); a missing key means the entry predates this field - **never infer a posting date** from either, and never backfill by guessing.
2. Only present jobs NOT already in the seen list or tracker.
3. Only present jobs NOT already in the seen list (matched by URL or
company+title) or tracker.
### Step 4.5: Generate Referral Contact Links (High & Medium Fit Only)
+3
View File
@@ -73,6 +73,8 @@ jobs:
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
with:
python-version: ${{ matrix.python-version }}
- name: Install Excel integration test dependency
run: python -m pip install openpyxl==3.1.5
- run: python -m unittest discover -s tests -t . -v
dependency-review:
@@ -174,6 +176,7 @@ jobs:
run: |
python3 tools/verify_pdf.py cv/main_example.pdf \
--pages 2 \
--ascii-dates \
--contains '[your.email@example.com]' \
--contains 'Professional Experience' \
--contains 'Achievement'
+2
View File
@@ -17,6 +17,7 @@ venv/
*.synctex.gz
*.fls
*.fdb_latexmk
build/
# Personal data (never commit these)
salary_data.json
@@ -73,6 +74,7 @@ documents/cv/**
documents/linkedin/**
documents/diplomas/**
documents/references/**
documents/projects/**
# Also where /interview saves its prep packs (interview_prep_<stage>.md): these
# name the employers applied to, quote what was submitted, and set out the
# candidate's weak points.
+416 -1
View File
@@ -13,6 +13,420 @@ per-file diff commands.
## [Unreleased]
## [1.7.2] - 2026-09-29
### Added
- **`verify_pdf.py --ascii-dates` makes the documented date-range rule executable**
(`tools/verify_pdf.py`, `/apply` Step 5d, `05-cv-templates.md`, CI) - "Date fields must be
ASCII ranges" records a real Workday import that dropped a role's end date because
`2016--2024` reaches the text layer as `2016<U+2013>2024`, and told Step 5d to confirm every
entry's years are joined by an ASCII hyphen. Nothing executed that: the natural probe,
`--contains "2016-2024"`, failed identically for "en-dashed" and "absent" before #458 and
passes on both since, because the fold maps the en-dash back to `-` on purpose. The new flag
reads the raw layer (never the fold), fails on a year joined to any Unicode dash or the
minus sign, and names each hit with its line and code point. A year on either side is
enough (`Mar 2016 – Jul 2016`, `2016 – Present`); a numeric range with no year is not a
date. Step 5d's extraction command now carries the flag (the dump is still written before
the check fails), the "Dates recognizable" checklist item points at it and keeps the bare
single-year half as a read-through, the guide's "add this to the step 5d checks" paragraph
names the command, and the upstream-only CI assertion runs it on the stock CV. Demonstrated
on the stock template: a `\cventry{2016--2024}` build fails with `U+2013`, the
`\cventry{2016-2024}` build passes, and `--contains "2016-2024"` passes on both. Only
horizontal whitespace may sit between the year and the dash, so a year ending one line is
not joined to a dash that opens the next (a bullet, a wrapped prose line) - yang2632's
catch in review, with the regex and fixture. Thirteen new `test_verify_pdf.py` cases.
Proposed by 9scorp4 in Discussion #385.
- **Real Excel workbook integration tests for the salary converter**
(`tests/test_convert_salary_excel_integration.py`, `.github/workflows/ci.yml`) -
generate temporary `.xlsx` files and invoke the documented converter CLI,
checking multiple worksheets, metadata options, Unicode text, localized
numbers, and compatibility with salary lookup. A workbook without salary
headers must fail without creating an output file. CI installs `openpyxl`
across the Python matrix; local runs without this optional dependency skip
the two integration cases while retaining the existing dependency-free tests.
- **`documents/projects/` portfolio ingestion in `/setup` (Path A)** (`documents/README.md`,
`.claude/commands/setup.md`, `.claude/commands/reset.md`, `tests/test_setup_command.py`) -
onboards project writeups, case studies, and documentation (`.md`, `.txt`, `.pdf`)
from `documents/projects/`, extracting structured summaries (problem domain, tech stack,
technical challenges, and measurable outcomes) to populate `## Independent Projects`
in `01-candidate-profile.md`.
- **Source host verification in `/apply` Step 1** (#431, `.claude/commands/apply.md`,
`tests/test_apply_host_check.py`) - before proceeding to draft CV and cover letters,
Step 1 verifies the posting URL's provenance against installed portal boards and the
six standard ATS apex domains (`greenhouse.io`, `lever.co`, `myworkdayjobs.com`/`workday.com`,
`ashbyhq.com`, `smartrecruiters.com`, `workable.com`). Look-alike prefix/suffix spoofing
fails closed, and unrecognized hosts are plainly flagged as unverified in the evaluation
output (`⚠ Unverified source host: <hostname>`) before drafting tokens are spent.
- **`/expand` project and portfolio expansion** (`.claude/commands/expand.md`,
`tests/test_expand_command.py`) - expands candidate discovery
to technical projects from public GitHub repositories, extracting structured summaries
(problem domain, tech stack, key technical challenges, and verifiable outcomes) to
populate the `## Independent Projects` section of `01-candidate-profile.md`.
- **Stale sweep branch in `/outcome`** (`.claude/commands/outcome.md`,
`tests/test_outcome_stale.py`) - introduces `/outcome stale [N]` (and `/outcome sweep [N]`)
to batch-resolve open applications quiet for 60+ (or N) days. Displays a numbered summary
of qualifying applications, requires explicit user confirmation (`all`, `select`, or `skip`),
resolves confirmed rows to `no_response`, logs dated entries to `notes`, updates archive
`outcome.md` files, and hands off to calibration when 3+ applications are resolved.
- **Mechanical layout verification for compiled PDFs** - `tools/verify_layout.py` measures
what `/apply` Step 5b previously only eyeballed: per-page text extent, bottom whitespace,
the largest internal vertical gap, footer collisions, and entry headers or section
headings stranded at a page break. It exists for a failure that survives every existing
check - a moderncv `\cventry` is an unbreakable `tabular`, so an entry that does not fit
jumps to the next page and leaves a hole behind (observed at 273pt, roughly 19 blank
lines) while the document still compiles, still reports the correct page count, and still
passes `tools/verify_pdf.py`. Geometry comes from Poppler `pdftotext -bbox`; Poppler is
optional repo-wide (since #369 `verify_pdf.py` prefers pypdf), and word bounding boxes
have no pypdf equivalent, so this is the one step that still wants it. A missing Poppler
- or the xpdf-based `pdftotext` Git for Windows puts ahead of it in PATH, which rejects
`-bbox` - degrades to a `skipped:` exit 2 rather than reporting a phantom layout failure.
Page count is deliberately left to `verify_pdf.py --pages` so that one rule keeps one
implementation. Thresholds are calibrated for the stock moderncv and `cover.cls`
geometry. Tests use synthetic page geometry, so they need neither Poppler nor a
LaTeX toolchain.
### Changed
- **`/add-template` keeps a registered template's intermediates in `build/`**
(#473, `.claude/commands/add-template.md`, `.gitignore`,
`tests/test_add_template_build_dir.py`) - the elicited compile command
now redirects intermediates (`.aux`, `.log`, ...) to a `build/` folder beside the source
and moves the PDF back, so Step 4's test-compile cleanup deletes one folder instead of
enumerating LaTeX extensions. The LaTeX command deletes the previous PDF first, so a
failed compile leaves no stale PDF for `/apply` to inspect. Toolchains with nothing to
redirect (`typst compile`) keep their command unchanged. The `ACTIVE-TEMPLATE` block now
tells `/apply` to run the command from the output directory and to delete `build/` in
its Step 5e cleanup. Stock templates are unchanged.
### Fixed
- **`freehire-search` titles no longer carry HTML entities or stray whitespace**
(`.agents/skills/freehire-search/cli/src/helpers.ts`) - titles are indexed as the source ATS
wrote them and `toResult` passed them through untouched, so `detail
intern-fullstack-amp-ai-innovation-noise-edaawqq2` printed `Intern - Fullstack &amp; AI
Innovation ` (trailing space included), and the same title reached `search` output. In a
93-result `/scrape` batch (`--country IN --seniority intern`), 1 title carried `&amp;` and 3
ended in whitespace. `/scrape` Step 4.75 reads undecoded entities in titles as a
half-working parser, so by that rule a single such posting marks a healthy portal
degraded. The title now goes through the CLI's existing `decodeHtmlEntities` (already used
for `detail` descriptions) and is trimmed; an empty or whitespace-only title still falls
back to `(untitled)`. Pinned by three new `parsing.test.ts` cases, which fail on the old
pass-through.
- **`freehire-search detail` keeps the salary period, so a yearly figure no longer reads as
monthly** (`.agents/skills/freehire-search/cli/src/helpers.ts`, `SKILL.md`) - `formatSalary`
printed only the currency and amounts, although freehire's enrichment records the period:
`detail software-developer-trainee-codifi-fra635ux` printed `INR 300000–300000` while the
API returns `salary_period: "year"` for that posting, so anyone screening pay against a
monthly figure read a yearly salary as a monthly one, twelve times its real monthly value.
The period is now appended when freehire records one (`INR 300000–300000/year`,
`INR 12000/month`); a record without a period prints exactly as before. Pinned by two new
`parsing.test.ts` cases, both failing on the old formatter.
- **`robots_check` decodes curl output as UTF-8, so a non-ASCII response no longer reads as
UNCONFIRMED on Windows** (`tools/robots_check.py`, `tests/test_robots_check.py`) -
`_fetch()` ran curl with `text=True` and no encoding, so Python decoded the response with
the ANSI code page. On a byte that code page leaves undefined (0x81 in cp1252 and cp1254,
0x9e in cp1254) subprocess's pipe reader thread died, `stdout` came back `None`, and the
gate printed `UNCONFIRMED (AttributeError)` - failing closed, but blocking the
browser-header retry `09-web-research.md` permits. Reproduced through the CLI on real
hosts: tr.indeed.com's robots.txt carries `Disallow: /職涯貼士/` (職 is `e8 81 b7`), and
kap.org.tr answers `/robots.txt` with a UTF-8 HTML 404 whose "A.Ş." carries `c5 9e`. The
call now pins `encoding='utf-8', errors='replace'` (RFC 9309: robots.txt is UTF-8), as
`verify_pdf.py` and `verify_layout.py` already do. That also closes a silent fail-open: a
UTF-8 rule whose bytes the code page does define (`Disallow: /şirket/` under cp1254)
decoded as `/ÅŸirket/`, never matched, and read as allowed. Decision rules are unchanged.
Two new tests swap curl for a child process that prints those bytes and pin cp1254 when no
encoding is passed, so the failure does not depend on the host's locale; both fail without
the fix.
- **The compile-warning check covers `tests/*.py` too** (`tests/test_verify_pdf.py`,
`tests/test_verify_layout.py`) - `FindNonAsciiDateRangesTests`' docstring quotes the regex
`\s*` unescaped, so Python 3.12+ prints a `SyntaxWarning` for the invalid escape `\s` when
the test module is compiled. It shows up in this repo's own CI log on 3.12, 3.13 and 3.14 while
the run stays green, and under `-W error::SyntaxWarning` the module fails to import. Escaped
in the docstring; `ToolsCompileWithoutWarnings` now compiles `tests/*.py` as well as
`tools/*.py` and fails on the unescaped version.
- **`linkedin-search detail` no longer fetches an unrelated posting for a URL on another
host** - `normalizeId` took the first 6+-digit path segment from *any* URL, so a
Greenhouse or Lever apply link (the kind a posting's own page hands out, and what a user
pastes back into `detail`) was reduced to that number and the handler fetched
`jobPosting/<number>` from LinkedIn: whatever job carried that id came back, printed with
exit 0, or `NOT_FOUND` if none did - never an error about the input. Demonstrated by driving
the real handler with a stubbed fetch: `https://boards.greenhouse.io/acme/jobs/4567890`
requested `.../jobPosting/4567890`. Every other portal CLI rejects an off-host detail URL
with `BAD_ID` (the #447 shape); linkedin was the one still trusting the digits. URLs are
now parsed for real: a `linkedin.com` host (apex or any subdomain, scheme optional) plus a
`/jobs/view/<slug-><id>` path yields the id, and anything else - other hosts, look-alike and
userinfo hosts, a linkedin.com profile or search URL - exits 1 with the stderr-JSON
`BAD_ID` contract before any request. Bare ids, URNs, and slash-free title slugs are
unchanged. Pinned by four new `normalizeId` cases and a new `detail-input.test.ts` that
drives `runDetail` and the CLI with fetch stubbed; the off-host cases fail on the old
pattern.
- **`/rank` tracker matching preserves Unicode company and role names**
(`tools/rank_state.py`, `tests/test_rank_state.py`) - ASCII-only normalization
collapsed distinct non-Latin roles to the same empty value and dropped
entirely non-Latin companies from tracker exclusions. Match using Unicode
case folding and NFC normalization, retaining letters, numbers, and combining
marks while continuing to ignore punctuation and spacing. CLI regressions
use the standard tracker header and cover distinct names, tracked matches,
equivalent accent encodings, and the existing ASCII matching behavior.
- **The Python tools no longer crash on Windows when a posting, company, CV line or file
name falls outside the ANSI code page** (`tools/rank_state.py`, `tools/job_key.py`,
`tools/verify_pdf.py`, `tools/verify_layout.py`, `tools/convert_salary_excel.py`,
`salary_lookup.py`, `tests/test_tools_utf8_output.py`) - a piped stdout on Windows
defaults to the ANSI code page (cp1252 on most Western installs), and that is how Claude
Code runs every tool. `/rank`'s candidate listing printed titles and companies with
`ensure_ascii=False`, so a single Cyrillic, CJK, Devanagari, Polish or Turkish posting
ended the run with `UnicodeEncodeError` before any output reached the workflow; the key
audit, the salary lookup, the salary converter and the layout report failed the same way.
Each tool now switches stdout and stderr to UTF-8 at entry, which also stops Danish and
other Western accents from arriving as cp1252 bytes. The regression tests run every
tool in a child process with a cp1252 stdout forced through `PYTHONIOENCODING`, so the
Linux CI job reproduces the Windows failure; all seven fail without the fix. The
subprocess helpers in `tests/test_rank_state.py` and `tests/test_job_key.py` now decode
child output as UTF-8 to match.
- **`/rank` rejects invalid score dimensions before updating an entry**
(`tools/rank_state.py`, `tests/test_rank_state.py`) - enforce the rubric's
inclusive 0-100 range and reject booleans, NaN, and infinities. Invalid results
now use the existing per-job error report, leaving the rejected entry intact
while valid results in the same batch are saved. CLI tests cover every score
dimension, oversized integers, boundary values, and fractional-score rounding.
- **Distinct non-Latin company names no longer share the unknown-company job key**
(`tools/job_key.py`, `tests/test_job_key.py`) - when a non-empty company name
has no ASCII slug, derive its fallback from a hash of the normalized name.
Missing names retain `unknown-company`, and existing ASCII keys are unchanged.
CLI tests cover distinct companies, case and canonical Unicode equivalence,
and stable keys across posting URLs. Existing state is not rewritten; `/scrape`
already recognizes stored postings by URL regardless of their previous key.
- **`/rank` tracker exclusion handles UTF-8 BOMs on reordered CSV headers**
(`tools/rank_state.py`, `tests/test_rank_state.py`) - when `company` or `role`
is the first column, a leading BOM becomes part of the header name and an
already-tracked application is selected for ranking again. Read with
`utf-8-sig` so both BOM-prefixed and plain UTF-8 trackers match correctly.
The standard `date`-first header already worked; regression coverage checks
all three column orders with and without a BOM.
- **`verify_layout.py` no longer emits a `SyntaxWarning` on every run** (`tools/verify_layout.py`,
`tests/test_verify_layout.py`) - the module docstring names the macro whose absence triggers the
Poppler `-bbox` crash, `\hypersetup{pdftitle=...}`, and a bare `\h` in a non-raw docstring is an
invalid escape sequence. Python 3.12+ prints `SyntaxWarning: "\h" is an invalid escape sequence`
the first time the module is compiled (CPython gh-98401) - it shows up in this repo's own CI log,
and in the middle of `/apply` Step 5b's layout report. The 3.15 language reference still
documents the sequence as a `SyntaxWarning`, with a `SyntaxError` only in a future Python
version. Escaped in the docstring; the new `ToolsCompileWithoutWarnings` case compiles the
source of every `tools/*.py` with warnings captured and fails on the unescaped version, so the
next docstring that quotes a LaTeX macro is caught too.
- **`/apply` Step 5b now actually runs the page-count check it claimed Step 5d ran**
(`.claude/commands/apply.md`, `tests/test_apply_page_count.py`) - the 5b prose said
"Page count is not checked here - that is `verify_pdf.py --pages`'s job, and Step 5d already
runs it", and `verify_layout.py`'s docstring declines to measure page count for the same
reason. Step 5d's only `verify_pdf.py` call is `--dump-text`, and no step in the workflow
passed `--pages` at all (only the upstream-only CI assertion on the stock examples does),
so the hard 2-page CV and 1-page cover letter limits were enforced by nothing but the
visual PDF read - the "measure first, then look" failure 5b was written to stop. 5b now
runs `verify_pdf.py --pages 2` on the CV and `--pages 1` on the cover letter ahead of
`verify_layout.py`, names the `ACTIVE-TEMPLATE` page limit as the substitute for a custom
template, and the deferral sentence points at those lines. Four spec tests pin the
invocations, their counts, their order relative to the layout measurement, and that no
prose defers the check to a step that does not run it; all four fail on master.
- **`salary_lookup.py` prints the privacy footnote only when a row actually carries `N/A*`**
- the `* N/A = Too few employees to publish (privacy)` line was appended under every
category table, including one where every row has an index, so the output asserted a
suppression that never happened (the residual noted on #470). The footnote now follows a
flag set by the `N/A*` branch; a table with a suppressed row renders exactly as before.
Two `FormatEntryTests` cases pin both directions; the "omitted" one fails on master.
- **`convert_salary_excel.py` pairs a bare `Count`/`Index` column pair instead of
splitting it, so `salary_lookup.py` no longer labels a published headcount as
privacy-suppressed** - the pairing loop required a non-empty derived category name on
both sides, but a header with no category word (`Count` + `Index`, Danish `Antal` +
`Lønindeks`) strips to an empty name, so the simplest layout the README advertises
("auto-pairs count/index columns") came out as two unrelated standalone categories:
`{"count": {"count": 500}, "index": {"index": 108.5}}`. `salary_lookup` then rendered
a `Count 500 N/A*` row above an `Index - 108.5` row, and the footnote read the
`N/A*` as "too few employees to publish (privacy)" - a false statement about a company
whose headcount is in the file, shown during `/apply`'s salary step. Demonstrated
through the documented Excel -> JSON -> lookup path with `openpyxl`; adding any suffix
(`Antal alle`) made pairing work, which is why the shipped tests, all suffixed, never
saw it. Bare pairs now pair under the README's top-level category name
(`all_employees`); a bare `Antal` with no bare index column still stays a standalone
count, and named pairs alongside are untouched. Four new cases in
`test_convert_salary_excel.py`, including one that renders the converter's output
through `salary_lookup.format_entry`; all four fail on the old pairing rule.
- **`tools/verify_layout.py`'s `skipped:` message named only one cause of a broken
extractor when there are two** (#451) - it blamed the xpdf-based `pdftotext` Git for
Windows puts ahead of Poppler in PATH (no `-bbox` flag, exits 99), but a real Poppler
can abort `-bbox`/`-bbox-layout`/`-htmlmeta` too: Poppler 26.0x before 26.05 crashes on
any PDF whose Info dictionary carries an empty string in any field, and `hyperref`
writes exactly that for every field it does not set. A `lualatex`/`pdflatex` document
built with `hyperref` and no `\hypersetup{pdftitle=...}` - an ordinary `/add-template`
CV template, not a malformed one - hits this with a working Poppler installed, and the
old message sent the reader to check their PATH when nothing was wrong with it. The
message now names both causes; behavior is unchanged, degrading to `skipped:` exit 2
either way, since a broken extractor is still not a broken document.
- **The template-placeholder guard in `test_setup_command.py` now skips on forks** (#463) -
`TemplatesStillCarryThePlaceholders` asserts that `05-cv-templates.md` and
`06-cover-letter-templates.md` still contain `[FIRST_NAME]`, `[LAST_NAME]`, `[YOUR_EMAIL]`,
`[YOUR_PHONE]`, `[YOUR_NAME]`, and `[YOUR_LINKEDIN_URL]`. Running `/setup` - the documented
path, and what Step 3.5/3.6 of that command exist to do - replaces exactly those tokens, so on
a personalized fork `python3 -m unittest discover -s tests` fails both checks permanently and
marks every push red. The class now uses the same `@unittest.skipIf` on `GITHUB_REPOSITORY`
(defaulting to upstream when unset, so local pristine-template runs still execute the guards)
that `test_placeholder_integrity.py` received in #407. The guard landed three days after that
fix and did not pick up the pattern; the `placeholder-integrity` CI job is upstream-gated and
does not cover the `05`/`06` tokens, so `python-tests` was their only check.
- **`verify_pdf.py --contains` now sees through LaTeX's typographic substitutions and
the pdflatex text layer keeps accents precomposed** (Discussions #385, #384) - the
comparison folded whitespace only, but LaTeX ligatures `'` into U+2019 and `--` into
U+2013, so on the stock CV compiled with the documented `lualatex` command
`--contains "Master's degree"` and `--contains "2016-2024"` both reported the keyword
missing from a document that plainly contains it (measured through both extractors;
`Six Sigma` and `Statistics` on the same page passed). The documented remedy for a
missing keyword is to add it, so the false negative nudged toward the one thing the ATS
section forbids. `normalize_text()` now folds both sides - NFC, then curly
apostrophes/quotes to ASCII, en/em dashes to `-`, no-break space to space - at
comparison time only; `--dump-text` still writes the raw layer, because that is what an
ATS parses and the date-range rule in `05-cv-templates.md` needs the raw en-dash visible
there. Separately, pdflatex without T1 font encoding stores accents decomposed
(`e` + U+0300; pypdf reads it as a stray spacing accent), which NFC cannot fully
repair - moderncv 2.5 loads T1 itself under pdflatex but the apt-packaged 2.3.1 does
not, so `cv/main_example.tex` and the guide's preamble gain
`\ifpdftex\usepackage[T1]{fontenc}\fi`, a no-op on the lualatex path. Pinned by
ten new `test_verify_pdf.py` cases (the fold-through ones fail on the whitespace-only
code) and a `test_latex_guidance.py` guard that the line exists and stays
pdflatex-only. Reported and diagnosed by 9scorp4. Fork users: your
personalized `cv/main_example.tex` gains the one guarded preamble line on rebase (a clean
3-way merge unless you edited the preamble); tailored CVs compiled with lualatex need nothing.
- **`jobdanmark-search detail` now backs off on 429/5xx like every other portal's detail
command** - the handler called `fetch()` directly instead of going through the CLI's own
request wrappers, so it carried none of the three things `apiFetch`/`apiPost` guarantee:
no 429/5xx retry loop (a rate-limited detail page wrote `API_ERROR` and exited after one
attempt, where jobnet, jobbank, jobindex, linkedin, and freehire all retry up to six
times), a hand-inlined User-Agent string that would drift from the exported `USER_AGENT`,
and a timeout the wrappers' tests never saw. `/scrape` calls `detail` once per
shortlisted posting, so a burst that tripped jobdanmark's rate limiter dropped those
postings outright - no description, no deadline - while the same burst on any other
portal rode it out. Demonstrated by driving the real command handler with a stubbed 429:
1 fetch attempt and exit 1 before, 7 attempts after (the contract's initial try plus six
retries). Fixed by adding `htmlFetch` to `helpers.ts` with the same backoff schedule,
timeout, and shared User-Agent as the JSON wrappers (404 returns `null` so `detail` keeps
its `NOT_FOUND` contract) and routing `detail` through it. Pinned in the existing
`retry-backoff`, `user-agent`, and `request-timeout` suites, which now cover all three
wrappers, plus a new `detail-backoff.test.ts` that exercises the handler path itself -
its retry cases fail against the bare `fetch()`.
- **Free-form tracker notes no longer break the CSV row** (#454) (`.claude/commands/gmail-sync.md`,
`.claude/commands/outcome.md`, `tests/test_tracker_notes_csv_safe.py`) - two writers put
free-form text into the `notes` column of `job_search_tracker.csv`: `/gmail-sync` Step 7a
copied the raw subject of a received email, and `/outcome` Step 4 appended "a short dated
note" with no constraint on its content. No writer in the framework emits a quoted tracker
field, so an unescaped comma splits the row for a naive split and for `csv.DictReader` alike -
the latter being what the repo's only machine reader of the tracker uses
(`tools/rank_state.py`). `notes` is column 10 of 14, so a subject as ordinary as
`Re: Your application, Data Scientist`, or a note as natural as `rejected, no feedback given`,
shifted `cv_file`, `cover_letter_file` and `source` a column left. A line break is worse: it
ends the row and starts a second one. Nothing validated the row afterwards, and the
`/gmail-sync` half was written unattended, so the corruption was silent. Both append
instructions now carry the rule themselves - `/gmail-sync` deletes commas, double quotes and
line breaks from the subject, `/outcome` writes its note without them - rather than a general
note a writer can miss. Nothing is lost on the `/gmail-sync` side: Step 7a item 2 still
records the subject verbatim in the archive's `outcome.md`, which is Markdown and carries no
such constraint. The fixed-format writers (`followed up YYYY-MM-DD`,
`stale resolved no_response (YYYY-MM-DD)`, `redrafted`) could never contain these characters
and are unchanged.
- **`jobindex-search detail` no longer fetches arbitrary URLs or invents posting-shaped
output** (#447) - the command fetched any `http(s)` input verbatim (no host check) and,
when the path didn't match its one pattern, silently used the whole input URL as the job
id; the only net was "the fetched page has a title", so a non-posting page came back as
a well-formed fake posting with exit 0 (demonstrated with jobindex's own homepage:
`id` = the URL, `title` = the site's tagline, `description` = navigation chrome). Every
other portal CLI rejects unparseable detail input with `BAD_ID` and constructs its fetch
URL from the extracted id; jobindex was the one CLI trusting the raw string - and
`/scrape`/`/rank` agents feed it stored URLs, so a ghost or redirected URL (the #331
class) yielded plausible garbage instead of an error. `buildUrl` now requires a
jobindex.dk host (apex or subdomain - look-alike and userinfo tricks rejected via real
URL parsing) plus a `/jobannonce/<id>` path, rebuilds the fetch URL from the extracted
id (the canonical short form the bare-id path always used), and exits 1 with the
stderr-JSON `BAD_ID` contract otherwise; bare ids stay permissive scheme- and
slash-free tokens (the jobnet precedent - the server 404s unknowns loudly). Pinned by
eight cases in the new `detail-input.test.ts`; the five rejection/canonicalization
cases fail against the verbatim unguarded extraction. Complementary to the `/apply`
host-check rule proposed in #431, which stays with its proposer.
- **`/outcome` and `/interview` no longer confuse two roles at the same company** (#443)
(`.claude/commands/outcome.md`, `.claude/commands/interview.md`,
`tests/test_apply_records_application.py`) - when a tracker row's `cv_file` /
`cover_letter_file` columns are empty, both commands fell back to a company-prefix glob
(`cv/main_<company>*.tex`). Two roles at one company both match it, so `/outcome` copied
whichever the filesystem returned first into the archive as `cv_draft.tex` - the file whose
purpose is to record what was actually submitted - and its own "leave an existing archived
file" rule then made the wrong copy permanent. Both fallbacks now glob the full
`<company>_<role>` stem, derived by the **Subfolder naming** rule in `documents/README.md`
rather than restated, and skip with a note instead of widening the search. Dropping the
hardcoded `.tex` also makes a template registered by `/add-template` findable.
- **`jobnet-search detail` no longer reports an externally hosted ad as not found** (#432) -
Jobnet's `/FindJob/JobAdDetails/<id>` returns 404 for ads with `isExternal: true`, so `detail`
on an ad `search` had just listed exited 1 with `NOT_FOUND`, and `/scrape` read the posting as
gone rather than hosted elsewhere (2 of 3 ads in a fresh sample). On that 404 the command now
falls back to the search endpoint, which does carry the ad's description and the external
application URL, and returns the record marked `isExternal: true` with a stderr note; fields the
search payload does not carry (`views`, `approvalStatus`, the boolean flags) are `null`, never
guessed. Verified live on two external ads.
- **`09-web-research.md`'s curl snippets no longer write into the repo when `$SCRATCHPAD`
is unset** - both runnable blocks in the 403-escalation path start with `cd "$SCRATCHPAD"`,
and nothing in the repository ever sets that variable (`git grep 'SCRATCHPAD='` returns
nothing). Unset, it expands to `cd ""`, which succeeds and leaves the shell where it
started, so the `&&` chain proceeds and `curl -o page.html` writes to the working
directory - in practice the checkout, which is exactly what the paragraph directly beneath
the curl block forbids ("Write to the session scratchpad directory, never into the repo").
The file's instruction and its own snippet disagreed, and the snippet won silently.
Both expansions are now guarded with `${SCRATCHPAD:?...}`, turning a silent repo write into
an immediate failure whose message names where the value comes from. Behaviour is unchanged
wherever the variable is set. The same undefined reference in `.claude/commands/rank.md`
was removed by #425 as a side effect of rewriting Step 2/4; this is the remaining instance.
- **`seen_jobs.json` keys are now a pure function of the posting** - `/scrape` Step 4 described
the key as prose (`"<url_or_company_title_key>"`) and nothing said how to derive it, so each
run slugified in its own way. Two failures followed, both observed in a live state file. Keys
carried characters that break the path they later become: `/apply` and `/outcome` derive an
archive folder from the same company+role pair, which is why `documents/README.md` has a
subfolder rule, and keys like `deloitte_junior-cybersecurity-analyst-(ot/iot)` and
`neverhack-estonia_penetration-tester-/-red-teamer` violate it. And the same posting was
stored twice when two runs truncated one title at different points
(`deloitte_cyber-intelligence-center-security-analy` and
`...-security-analyst-at` are one job, one URL, two entries) - which defeats the dedup the
file exists for. `tools/job_key.py` now owns the derivation: the slug is normalised, and
truncation is length-capped *and* disambiguated by a hash of the full slug, so a long title
always produces the same key and two long titles sharing a prefix cannot collide. Step 4
calls the helper instead of describing it. `--audit` reports non-conforming entries in an
existing state file and deliberately never rewrites them: stored keys are matched against
`job_search_tracker.csv` by company+role elsewhere, so a silent rewrite would break the link
between a stored job and its application record.
Existing state files need no migration: Step 2's candidate filter matches a posting to a stored
entry by URL regardless of that entry's key, so a workspace whose entries predate the helper does
not see its still-live postings re-presented as new.
## [1.7.1] - 2026-09-06
### Added
@@ -1214,7 +1628,8 @@ At this baseline the framework provides:
- **Cross-runtime support** - a root `AGENTS.md` pointer so Codex and Antigravity can
discover the portable portal skills, with Claude Code as the reference runtime.
[Unreleased]: https://github.com/MadsLorentzen/ai-job-search/compare/v1.7.1...HEAD
[Unreleased]: https://github.com/MadsLorentzen/ai-job-search/compare/v1.7.2...HEAD
[1.7.2]: https://github.com/MadsLorentzen/ai-job-search/compare/v1.7.1...v1.7.2
[1.7.1]: https://github.com/MadsLorentzen/ai-job-search/compare/v1.7.0...v1.7.1
[1.7.0]: https://github.com/MadsLorentzen/ai-job-search/compare/v1.6.0...v1.7.0
[1.6.0]: https://github.com/MadsLorentzen/ai-job-search/compare/v1.5.0...v1.6.0
+2 -1
View File
@@ -61,7 +61,7 @@ The framework encodes career guidance best practices, including structured evalu
## Prerequisites
- [Claude Code](https://claude.com/claude-code) (CLI). Using a different agent tool (Codex, Antigravity, Gemini CLI)? Start at [`AGENTS.md`](AGENTS.md) - the portal search skills work there out of the box, and [community forks](https://github.com/MadsLorentzen/ai-job-search/discussions/78) adapt the full workflow.
- [Claude Code](https://claude.com/claude-code) (CLI). Claude Code has no free tier: you need a Claude Pro/Max/Team subscription or Anthropic API credits (pay-per-token, usually cheaper for occasional use). Using a different agent tool (Codex, Antigravity, Gemini CLI)? Start at [`AGENTS.md`](AGENTS.md) - the portal search skills work there out of the box, and [community forks](https://github.com/MadsLorentzen/ai-job-search/discussions/78) adapt the full workflow.
- Python 3.10+
- [Bun](https://bun.sh) (for job search CLI tools)
- LaTeX distribution with `lualatex` and `xelatex`: [TeX Live](https://tug.org/texlive/), [MacTeX](https://tug.org/mactex/), [TinyTeX](https://yihui.org/tinytex/), or [MiKTeX](https://miktex.org/). The CV compiles with `lualatex` (pdflatex often fails on modern MiKTeX installs with `fontawesome5` font-expansion errors); the cover letter compiles with `xelatex` because `cover.cls` requires `fontspec`. If using a minimal TeX install such as TinyTeX or BasicTeX, install the extra packages listed in [SETUP.md](SETUP.md#minimal-tex-install-tinytexbasictex).
@@ -225,6 +225,7 @@ ai-job-search/
│ ├── robots_check.py # Gate the browser-header retry against robots.txt
│ ├── security_guards.py # CI guards: permission allowlist, gitignore rules, manifests
│ ├── upstream_triage.py # Sort upstream commits into worth-reviewing vs probably-skip
│ ├── verify_layout.py # Measure a compiled PDF's page layout (holes, orphans, footer collisions)
│ ├── verify_pdf.py # Verify a compiled PDF's page count and extractable text
│ └── README_SALARY_TOOL.md # Salary tool setup instructions
├── job_scraper/ # Scraper state (seen jobs, results)
+7
View File
@@ -23,6 +23,13 @@
\renewcommand*{\sectionstyle}[1]{{\sectionfont\color{color1}#1}}
\usepackage[utf8]{inputenc}
% pdflatex fallback only (the documented engine is lualatex, which skips this
% branch). Without T1 font encoding pdflatex builds accented letters with
% \accent, and the PDF text layer stores them decomposed - `e` + U+0300 rather
% than U+00E8 - so an ATS keyword match on "Genève" fails while the page looks
% right. moderncv 2.5 loads T1 itself under pdflatex; 2.3.1 (Debian/Ubuntu apt)
% does not. \ifpdftex comes from iftex, which every moderncv version loads.
\ifpdftex\usepackage[T1]{fontenc}\fi
% moderncv loads hyperref itself in an \AtEndPreamble hook, so \hypersetup
% must go in an \AtEndPreamble of our own: on moderncv < 2.4 a top-level
% \usepackage{hyperref} clashes with the class's own
+20
View File
@@ -12,6 +12,7 @@ documents/
├── linkedin/ # LinkedIn profile export (PDF)
├── diplomas/ # Degree certificates and transcripts
├── references/ # Reference letters
├── projects/ # Independent project summaries, case studies, or portfolio docs
├── postings/ # Raw job posting text, pasted manually for pages Claude can't fetch
│ └── <Company> - <Job Title>.txt # Filename = company + job title, content = full posting text
├── applications/ # Past job applications
@@ -97,6 +98,25 @@ Reference letters from former managers, supervisors, or collaborators.
---
## projects/
Summaries, case studies, READMEs, writeups, or documentation for independent, open-source, freelance, or personal portfolio projects.
**Supported formats:** `.md`, `.txt`, `.pdf`
**What `/setup` extracts:**
- Project name and description
- Problem domain and target audience
- Tech stack, tools, and libraries used
- Key technical challenges and architectural decisions
- Measurable outcomes, metrics, or performance improvements (added to `01-candidate-profile.md` under `## Independent Projects`)
**Naming:** Use descriptive project names, e.g. `project_realtime_chat.md`, `portfolio_compiler.txt`, `open_source_etl.pdf`.
**Tip:** These feed into the `## Independent Projects` section of `01-candidate-profile.md` and provide concrete technical evidence that `/apply` can weave into tailored CVs and cover letters.
---
## postings/
A drop folder for raw job posting text when Claude can't fetch a page directly (bot-blocked ATS platforms like Lever, Greenhouse behind Cloudflare, JS-heavy SPAs that return empty content, etc.). You open the posting yourself and paste the full text into a `.txt` file here.
View File
Whitespace-only changes.
+21 -1
View File
@@ -319,6 +319,7 @@ def format_entry(entry, metadata):
lines.append(f" {'Category':<22} {'Count':>6} {index_label:>8} {'vs Baseline':>10}")
lines.append(f" {'-'*50}")
suppressed = False # did any row render its index as N/A*?
for label, data in categories.items():
display_label = label.replace("_", " ").title()
count = data.get("count")
@@ -339,9 +340,14 @@ def format_entry(entry, metadata):
else:
index_str = "N/A*"
diff_str = ""
suppressed = True
lines.append(f" {display_label:<22} {count_str:>6} {index_str:>8} {diff_str:>10}")
lines.append(f"\n * N/A = Too few employees to publish (privacy)")
# The footnote explains the N/A* marker; printing it under a table with
# no such row asserts a privacy suppression that did not happen.
lines.append("")
if suppressed:
lines.append(" * N/A = Too few employees to publish (privacy)")
if metadata.get("baseline_description"):
lines.append(f" {metadata['baseline_description']}")
else:
@@ -379,7 +385,21 @@ def print_validation_report(errors, warnings):
return 0
def _force_utf8_output() -> None:
"""Write UTF-8 whatever the host's default encoding is.
A piped stdout on Windows defaults to the ANSI code page (cp1252 on most
Western installs), so printing a company, title or file name outside it
raised UnicodeEncodeError before the workflow saw any output.
"""
for stream in (sys.stdout, sys.stderr):
reconfigure = getattr(stream, "reconfigure", None) # absent on a StringIO under test
if reconfigure:
reconfigure(encoding="utf-8")
def main():
_force_utf8_output()
parser = argparse.ArgumentParser(description="Salary Benchmark Lookup")
parser.add_argument("company", nargs="?", help="Company name to search for")
parser.add_argument("--city", help="Filter by city name")
+26
View File
@@ -0,0 +1,26 @@
"""Guard for /add-template's build directory rule.
The elicited LaTeX compile command redirects intermediates to `build/` and
moves the PDF back beside the source, where `/apply` reads it. Dropping the
leading `rm -f` lets a failed compile leave a stale PDF for `/apply` to
inspect; dropping the `mv` leaves no PDF beside the source at all.
"""
import unittest
from pathlib import Path
REPO = Path(__file__).resolve().parent.parent
ADD_TEMPLATE = REPO / ".claude" / "commands" / "add-template.md"
LATEX_COMMAND = (
"rm -f <file>.pdf && mkdir -p build && lualatex -interaction=nonstopmode "
"-output-directory=build <file>.tex && mv build/<file>.pdf ./"
)
class AddTemplateKeepsIntermediatesInBuild(unittest.TestCase):
def test_latex_command_redirects_to_build_and_returns_a_fresh_pdf(self):
self.assertIn(LATEX_COMMAND, ADD_TEMPLATE.read_text(encoding="utf-8"))
if __name__ == "__main__":
unittest.main()
+131
View File
@@ -0,0 +1,131 @@
"""Guards for /apply's source host verification rule in Step 1 (#431).
Pins the invariants from the maintainer design in issue #431:
- URLs must be verified before drafting against installed portal boards or known ATS apexes.
- The 6 standard ATS apex domains must be checked: greenhouse.io, lever.co,
myworkdayjobs.com (or workday.com), ashbyhq.com, smartrecruiters.com, workable.com.
- Look-alike attacks (prefixes, suffixes, userinfo tricks) must fail closed.
- Any other host must be named plainly in the output as unverified.
"""
import re
import unittest
from pathlib import Path
from urllib.parse import urlparse
REPO = Path(__file__).resolve().parent.parent
APPLY_COMMAND_FILE = REPO / ".claude" / "commands" / "apply.md"
KNOWN_ATS_APEXES = {
"greenhouse.io",
"lever.co",
"myworkdayjobs.com",
"workday.com",
"ashbyhq.com",
"smartrecruiters.com",
"workable.com",
}
SHIPPED_PORTAL_HOSTS = {
"jobindex.dk",
"linkedin.com",
"jobnet.dk",
"jobbank.dk",
"jobdanmark.dk",
"freehire.me",
}
def classify_posting_host(url_str: str, installed_portals: set[str] = SHIPPED_PORTAL_HOSTS) -> tuple[str, str]:
"""Reference implementation of the host provenance rule in /apply Step 1.
Returns (tier, host), where tier is one of:
- 'installed_portal'
- 'official_ats'
- 'unverified'
"""
try:
parsed = urlparse(url_str)
host = (parsed.hostname or "").lower().strip()
except Exception:
return "unverified", ""
if not host:
return "unverified", ""
# Check installed portal boards (exact match or subdomain match)
for portal in installed_portals:
if host == portal or host.endswith(f".{portal}"):
return "installed_portal", host
# Check known official ATS apexes (exact match or subdomain match)
for apex in KNOWN_ATS_APEXES:
if host == apex or host.endswith(f".{apex}"):
return "official_ats", host
return "unverified", host
class ApplyHostVerificationSpecTests(unittest.TestCase):
def setUp(self):
self.text = APPLY_COMMAND_FILE.read_text(encoding="utf-8")
step1_match = re.search(r"## Step 1: DRAFTER - Evaluate Fit(.*?)(?=## Step 2:)", self.text, re.DOTALL)
self.assertTrue(step1_match, "Step 1 must exist in apply.md")
self.step1_text = step1_match.group(1)
def test_step1_contains_source_host_verification_heading(self):
self.assertIn("Source Host Verification", self.step1_text)
def test_step1_documents_all_six_ats_apexes(self):
for apex in ["greenhouse.io", "lever.co", "myworkdayjobs.com", "ashbyhq.com", "smartrecruiters.com", "workable.com"]:
self.assertIn(apex, self.step1_text, f"Step 1 must specify ATS apex: {apex}")
def test_step1_documents_look_alike_fail_closed_rules(self):
self.assertIn("evil-greenhouse.io", self.step1_text)
self.assertIn("fail closed", self.step1_text)
def test_step1_requires_unverified_hosts_to_be_named_plainly(self):
self.assertIn("Unverified source host", self.step1_text)
def test_classifier_identifies_official_ats_subdomains(self):
urls = [
"https://boards.greenhouse.io/acme/jobs/12345",
"https://job-boards.greenhouse.io/acme/jobs/12345",
"https://jobs.lever.co/corp/67890",
"https://acme.myworkdayjobs.com/en-US/Careers/job/1",
"https://jobs.ashbyhq.com/startup/abc-123",
"https://jobs.smartrecruiters.com/Enterprise/456",
"https://apply.workable.com/tech-corp/j/789/",
]
for url in urls:
tier, host = classify_posting_host(url)
self.assertEqual(tier, "official_ats", f"{url} should classify as official_ats, got {tier}")
def test_classifier_identifies_installed_portal_hosts(self):
urls = [
"https://www.jobindex.dk/jobannonce/12345",
"https://www.linkedin.com/jobs/view/999999",
"https://jobnet.dk/find-job/8888",
"https://jobbank.dk/job/7777",
"https://freehire.me/job/6666",
]
for url in urls:
tier, host = classify_posting_host(url)
self.assertEqual(tier, "installed_portal", f"{url} should classify as installed_portal, got {tier}")
def test_classifier_fails_closed_on_look_alikes_and_unverified_hosts(self):
suspicious = [
"https://evil-greenhouse.io/job/1",
"https://boards.greenhouse.io.evil.com/job/1",
"https://boards.greenhouse.io@evil-domain.com/job/1",
"https://myworkdayjobs.com.phishing.net/login",
"https://lever.co.attacker.org/apply",
"https://unknown-board.example.com/posting/123",
]
for url in suspicious:
tier, host = classify_posting_host(url)
self.assertEqual(tier, "unverified", f"{url} must fail closed as unverified, got {tier}")
if __name__ == "__main__":
unittest.main()
+85
View File
@@ -0,0 +1,85 @@
"""Guard for /apply Step 5b's page-count check.
The 2-page CV and 1-page cover letter limits are the hard rules of
05-cv-templates.md and 06-cover-letter-templates.md, and two places defer
their enforcement to `tools/verify_pdf.py --pages`: `verify_layout.py`'s
docstring ("page count is verify_pdf.py's job, and CI runs it") and Step 5b's
own prose, which used to say "Step 5d already runs it". Step 5d's only
invocation is `--dump-text`, and no other step passed `--pages` at all, so
the one rule with a mechanical check had zero runnable implementations in
the workflow. These tests pin that the invocations exist where the prose says
they do, with the counts the guides require, and that no step defers the
check to another step that does not run it.
"""
import re
import unittest
from pathlib import Path
REPO = Path(__file__).resolve().parent.parent
APPLY = REPO / ".claude" / "commands" / "apply.md"
VERIFY_LAYOUT = REPO / "tools" / "verify_layout.py"
def section(path, heading):
"""The body of one markdown section, up to the next heading of any depth."""
text = path.read_text(encoding="utf-8")
start = text.index(heading) + len(heading)
rest = text[start:]
end = re.search(r"^#{1,4} ", rest, re.MULTILINE)
return rest[: end.start()] if end else rest
def page_count_invocations(text):
"""(document path, page count) for every runnable verify_pdf --pages line."""
return re.findall(
r"^python tools/verify_pdf\.py (\S+) --pages (\d+)\s*$", text, re.MULTILINE
)
class ApplyRunsThePageCountCheck(unittest.TestCase):
def setUp(self):
self.step_5b = section(APPLY, "### 5b. Inspect layout")
def test_step_5b_checks_both_documents_with_the_guides_page_limits(self):
invocations = dict(page_count_invocations(self.step_5b))
self.assertEqual(
invocations.get("cv/main_<company>_<role>.pdf"),
"2",
"Step 5b must run verify_pdf.py --pages 2 on the CV - the hard "
"2-page limit has no other mechanical check",
)
self.assertEqual(
invocations.get("cover_letters/cover_<company>_<role>.pdf"),
"1",
"Step 5b must run verify_pdf.py --pages 1 on the cover letter",
)
def test_page_count_runs_before_the_layout_measurement(self):
# verify_layout.py's own docstring declines to check page count because
# verify_pdf.py --pages does; the deferral only holds if that runs first.
first_pages = self.step_5b.index("--pages")
first_layout = self.step_5b.index("verify_layout.py")
self.assertLess(first_pages, first_layout)
def test_no_step_defers_the_check_to_a_step_that_does_not_run_it(self):
text = APPLY.read_text(encoding="utf-8")
self.assertNotIn(
"Step 5d already runs it",
text,
"Step 5d's only verify_pdf call is --dump-text; the page-count "
"invocation lives in 5b and the prose must point there",
)
def test_layout_tools_deferral_is_backed_by_a_runnable_invocation(self):
docstring = VERIFY_LAYOUT.read_text(encoding="utf-8")
self.assertIn("verify_pdf.py --pages", docstring)
self.assertGreaterEqual(
len(page_count_invocations(APPLY.read_text(encoding="utf-8"))),
2,
"verify_layout.py defers page count to verify_pdf.py --pages, so "
"/apply must actually invoke it",
)
if __name__ == "__main__":
unittest.main()
+93
View File
@@ -11,6 +11,7 @@ byte-identical to /outcome's, which is the entire reason for reusing it.
How each reader treats `drafted` is pinned per reader below, because the
right answer differs between them.
"""
import fnmatch
import re
import subprocess
import sys
@@ -385,6 +386,98 @@ class DeadlineSurvivesEveryWrite(unittest.TestCase):
self.assertIn(needle, haystack, why)
class FallbackGlobFindsOneRolesDocuments(unittest.TestCase):
"""The `cv_file` fallback must select one role's documents, not one company's.
`/apply` names drafts `cv/main_<company>_<role><CV_EXT>`, so two roles
at one company differ only in the role half. When the tracker row's
`cv_file`/`cover_letter_file` columns are empty - a row written before
#291, added by hand, or by /outcome's own outside-the-workflow path -
both readers fall back to a glob. A company-prefix glob matches both
roles and the first hit wins silently: /outcome copies it to
`cv_draft.tex`, and its own "leave an existing archived file" rule then
makes the wrong answer permanent (#443).
The globs are extracted from the specs rather than restated here, so
these tests pin what the specs actually say.
"""
COMPANY = "Acme"
ROLES = ("Data Scientist", "ML Engineer", "ML Engineer II")
CASES = [
(OUTCOME, "## Step 3: Archive the Application Materials",
"by the **Subfolder naming** rule in `documents/README.md`",
"the archive locator must derive the stem by the one documented rule, "
"not invent a second derivation that drifts from it"),
(OUTCOME, "## Step 3: Archive the Application Materials",
"Never widen those globs to the company alone",
"without the prohibition the next edit relaxes the glob when it finds "
"no match, which is exactly the wrong-file-recorded-as-submitted case"),
(INTERVIEW, "## Step 1: Load the Application Context",
"by the **Subfolder naming** rule in `documents/README.md`",
"interview's fallback must resolve the same stem /apply wrote"),
(INTERVIEW, "## Step 1: Load the Application Context",
"Never widen those globs to the company alone",
"prep built from the sibling role's CV is a live-conversation failure"),
]
def test_both_readers_glob_the_full_stem(self):
for path, heading, needle, why in self.CASES:
with self.subTest(file=path.name, rule=needle):
self.assertIn(needle, section(path, heading), why)
@staticmethod
def globs(path, heading):
"""The two fallback globs exactly as the spec writes them."""
body = section(path, heading)
found = re.findall(r"`(cv/main_[^`]+|cover_letters/cover_[^`]+)`", body)
return [g for g in found if "*" in g]
def resolve(self, glob, role):
"""Substitute the spec's placeholders the way the reader would."""
stem = ArchiveNameIsOnePathComponent.derive(self.COMPANY, role)
company = ArchiveNameIsOnePathComponent.derive(self.COMPANY, "").rstrip("_")
return glob.replace("<company>_<role>", stem).replace("<company>", company)
def drafted_files(self, ext=".tex"):
"""Exactly what /apply Step 5 leaves in cv/ for two roles at one company."""
return [
"cv/main_%s%s" % (ArchiveNameIsOnePathComponent.derive(self.COMPANY, r), ext)
for r in self.ROLES
]
def test_the_cv_glob_selects_the_row_s_own_role(self):
on_disk = self.drafted_files()
for path, heading in ((OUTCOME, "## Step 3: Archive the Application Materials"),
(INTERVIEW, "## Step 1: Load the Application Context")):
cv_glob = next(g for g in self.globs(path, heading) if g.startswith("cv/"))
for role, expected in zip(self.ROLES, on_disk):
with self.subTest(file=path.name, role=role):
hits = fnmatch.filter(on_disk, self.resolve(cv_glob, role))
self.assertEqual(
hits, [expected],
"%s's fallback glob %r matched %r for role %r. A glob that "
"matches both roles hands /outcome whichever the filesystem "
"returns first, and it archives that as what was submitted."
% (path.name, cv_glob, hits, role),
)
def test_the_glob_finds_a_non_tex_template(self):
"""`/add-template` makes `.typ` a real output; a hardcoded `.tex` misses it."""
on_disk = self.drafted_files(ext=".typ")
cv_glob = next(
g for g in self.globs(OUTCOME, "## Step 3: Archive the Application Materials")
if g.startswith("cv/")
)
hits = fnmatch.filter(on_disk, self.resolve(cv_glob, self.ROLES[0]))
self.assertEqual(
hits, [on_disk[0]],
"the fallback hardcodes an extension, so a template registered by "
"/add-template is invisible to it and /outcome archives nothing",
)
class ArchiveNameIsOnePathComponent(unittest.TestCase):
"""`<company>_<role>` must derive a single path component.
+79
View File
@@ -3,6 +3,7 @@ import unittest
from contextlib import redirect_stderr
from types import SimpleNamespace
from salary_lookup import format_entry
from tools.convert_salary_excel import (
INDEX_PATTERNS,
detect_column_type,
@@ -391,6 +392,84 @@ class ParseNumericCellLocaleTests(unittest.TestCase):
self.assertEqual(parse_numeric_cell("1.234.567,89"), 1234567.89)
class BareCountIndexPairingTests(unittest.TestCase):
"""A count/index pair whose headers carry no category word is still a pair.
"Count" + "Index" (Danish "Antal" + "Lønindeks") both strip to an empty
category name, and the pairing loop used to require a non-empty name on
both sides, so the single-category layout the README describes as
"auto-pairs count/index columns" came out as two unrelated standalone
columns. salary_lookup then rendered the count row with "N/A*" for the
index - "too few employees to publish (privacy)" - about a company whose
headcount was right there in the file. The literal name is asserted (not
the module constant) so the cases run, and fail, against the old converter.
"""
DEFAULT_CATEGORY = "all_employees"
def test_bare_english_pair_is_paired_under_the_default_category(self):
ws = FakeWorksheet([
("Company", "City", "Count", "Index"),
("Acme Corp", "Copenhagen", 500, 108.5),
])
companies = parse_sheet(ws)
self.assertEqual(
companies[0]["categories"],
{self.DEFAULT_CATEGORY: {"count": 500, "index": 108.5}},
)
def test_bare_danish_pair_is_paired_under_the_default_category(self):
ws = FakeWorksheet([
("Firma", "By", "Antal", "Lønindeks"),
("Acme Corp", "Aarhus", 500, 108.5),
])
companies = parse_sheet(ws)
self.assertEqual(
companies[0]["categories"],
{self.DEFAULT_CATEGORY: {"count": 500, "index": 108.5}},
)
def test_bare_pair_does_not_cross_pair_with_a_named_category(self):
# The bare pair and the named pair coexist; neither steals the other's
# column, and a lone "Antal" with no bare index column stays standalone
# (pinned separately by test_standalone_count_column_is_stored_as_count_not_index).
ws = FakeWorksheet([
("Company", "Antal", "IT Count", "IT Index", "Lønindeks"),
("Acme Corp", 500, 30, 112.0, 108.5),
])
companies = parse_sheet(ws)
self.assertEqual(
companies[0]["categories"],
{
self.DEFAULT_CATEGORY: {"count": 500, "index": 108.5},
"it": {"count": 30, "index": 112.0},
},
)
def test_lookup_renders_the_bare_pair_as_one_row_without_the_privacy_footnote_firing(self):
# End to end through the documented path: converter output is what
# salary_lookup.format_entry displays during /apply. Before the fix the
# same sheet produced a "Count 500 N/A*" row plus an "Index - 108.5"
# row - the N/A* asserting a privacy suppression that never happened.
ws = FakeWorksheet([
("Company", "City", "Count", "Index"),
("Acme Corp", "Copenhagen", 500, 108.5),
])
entry = parse_sheet(ws)[0]
rendered = format_entry(entry, {"index_baseline": 100, "index_label": "Index"})
self.assertNotIn("N/A*", rendered.split("* N/A =")[0])
self.assertRegex(rendered, r"All Employees\s+500\s+108\.5\s+\+8\.5%")
self.assertNotRegex(rendered, r"^\s*Count\s+500", )
class CompoundCategoryPairingTests(unittest.TestCase):
def test_parse_sheet_pairs_danish_compound_index_with_count(self):
# "Lønindeks alle" is *detected* as an index column via
@@ -0,0 +1,90 @@
"""Exercise the documented Excel -> converter CLI -> salary JSON path."""
import json
import subprocess
import sys
import unittest
from pathlib import Path
from tempfile import TemporaryDirectory
try:
import openpyxl
except ImportError:
openpyxl = None
from salary_lookup import collect_validation_issues, format_entry
TOOL = Path(__file__).resolve().parent.parent / "tools" / "convert_salary_excel.py"
@unittest.skipUnless(openpyxl, "requires optional openpyxl (installed in CI)")
class ConverterCLI(unittest.TestCase):
def setUp(self):
directory = TemporaryDirectory()
self.addCleanup(directory.cleanup)
self.root = Path(directory.name)
self.workbook = self.root / "salary input.xlsx"
self.output = self.root / "salary output.json"
def run_converter(self, *args):
return subprocess.run(
[sys.executable, str(TOOL), str(self.workbook),
"--output", str(self.output), *args],
cwd=self.root, capture_output=True, text=True,
)
def test_real_workbook_round_trips_multiple_sheets_and_metadata(self):
book = openpyxl.Workbook()
try:
first = book.active
first.title = "All employees"
first.append(["Company", "City", "Count", "Index"])
first.append(["Acme Corp", "Copenhagen", 500, 108.5])
first.append(["Café Étoile", "Paris", 12, "102,5"])
second = book.create_sheet("Engineering")
second.append(["Company", "City", "Engineering Count", "Engineering Index"])
second.append(["Other Corp", "Berlin", 8, 112.5])
book.save(self.workbook)
finally:
book.close()
result = self.run_converter(
"--source", "Integration fixture", "--baseline", "100",
"--baseline-desc", "Index 100 = median salary",
)
self.assertEqual(result.returncode, 0, result.stderr)
data = json.loads(self.output.read_text(encoding="utf-8"))
self.assertEqual(data["metadata"], {
"source": "Integration fixture", "index_baseline": 100,
"index_label": "Index", "baseline_description": "Index 100 = median salary",
})
self.assertEqual(data["companies"], [
{"company": "Acme Corp", "city": "Copenhagen", "categories": {
"all_employees": {"count": 500, "index": 108.5}}},
{"company": "Café Étoile", "city": "Paris", "categories": {
"all_employees": {"count": 12, "index": 102.5}}},
{"company": "Other Corp", "city": "Berlin", "categories": {
"engineering": {"count": 8, "index": 112.5}}},
])
self.assertEqual(collect_validation_issues(data), ([], []))
rendered = format_entry(data["companies"][0], data["metadata"])
self.assertIn("All Employees", rendered)
self.assertIn("+8.5%", rendered)
self.assertNotIn("N/A*", rendered)
def test_workbook_without_salary_headers_fails_without_writing_output(self):
book = openpyxl.Workbook()
try:
book.active.append(["Notes", "Comments"])
book.active.append(["No salary data", "Just a worksheet"])
book.save(self.workbook)
finally:
book.close()
result = self.run_converter()
self.assertEqual(result.returncode, 1, result.stderr)
self.assertIn("No data could be parsed", result.stderr)
self.assertFalse(self.output.exists())
if __name__ == "__main__":
unittest.main()
+53
View File
@@ -0,0 +1,53 @@
"""Tests for the /expand command specification."""
import unittest
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
EXPAND_COMMAND_FILE = REPO_ROOT / ".claude" / "commands" / "expand.md"
class ExpandCommandTests(unittest.TestCase):
def test_expand_command_file_exists(self):
self.assertTrue(EXPAND_COMMAND_FILE.exists(), "expand.md must exist under .claude/commands/")
def test_expand_command_file_starts_with_correct_header(self):
text = EXPAND_COMMAND_FILE.read_text(encoding="utf-8")
first_line = text.lstrip().splitlines()[0]
self.assertTrue(
first_line.startswith("# /expand"),
f"Command file must start with '# /expand', got: {first_line!r}",
)
def test_expand_covers_all_discovery_sources(self):
text = EXPAND_COMMAND_FILE.read_text(encoding="utf-8")
sources = [
"documents/cv/",
"documents/linkedin/",
"documents/diplomas/",
"documents/references/",
"GitHub Profile",
]
for src in sources:
self.assertIn(src, text, f"expand.md must include discovery source: {src}")
def test_expand_maps_github_projects_to_independent_projects_section(self):
text = EXPAND_COMMAND_FILE.read_text(encoding="utf-8")
self.assertIn("## Independent Projects", text)
self.assertIn("Independent Projects & Portfolio", text)
self.assertIn("GitHub — repo-name", text)
self.assertIn("Portfolio & projects grounded in code", text)
self.assertNotIn("documents/projects/", text)
def test_expand_enforces_additive_and_confirmation_principles(self):
text = EXPAND_COMMAND_FILE.read_text(encoding="utf-8")
self.assertIn("Additive only", text)
self.assertIn("User confirms before writing", text)
self.assertIn("`all`", text)
self.assertIn("`review`", text)
self.assertIn("`skip`", text)
if __name__ == "__main__":
unittest.main()
+165
View File
@@ -0,0 +1,165 @@
"""Tests for tools/job_key.py - the canonical seen_jobs.json key function.
/scrape's key rule was prose only, so runs slugified inconsistently and the
state file accumulated two failures: keys carrying "/", "," and "&" that break
the archive-folder path `/apply`/`/outcome` derive from company+role, and the
same job stored twice under two different truncations of a long title. These
pin the fix - a pure, deterministic function of company+title(+url) - and the
audit that finds both failure classes in an existing file.
"""
import json
import subprocess
import sys
import unittest
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "tools"))
from job_key import is_canonical, is_legacy_shape, make_key, slugify # noqa: E402
REPO = Path(__file__).resolve().parent.parent
TOOL = REPO / "tools" / "job_key.py"
class Slugify(unittest.TestCase):
def test_basic(self):
self.assertEqual(slugify("Acme Corp"), "acme-corp")
def test_strips_punctuation_that_breaks_paths(self):
self.assertEqual(slugify("Ops Consulting, LLC"), "ops-consulting-llc")
self.assertEqual(slugify("Penetration Tester / Red Teamer"), "penetration-tester-red-teamer")
self.assertEqual(slugify("Junior Cybersecurity Analyst (OT/IoT)"), "junior-cybersecurity-analyst-ot-iot")
def test_non_latin_script_reduces_to_empty(self):
self.assertEqual(slugify("시큐리온"), "")
self.assertEqual(slugify("Код Безопасности"), "")
class MakeKey(unittest.TestCase):
def test_shape(self):
key = make_key("Acme Corp", "SOC Analyst (L2)")
self.assertEqual(key, "acme-corp_soc-analyst-l2")
self.assertTrue(is_canonical(key))
def test_deterministic_across_calls(self):
title = "Cyber Intelligence Center Security Analyst with an unusually long title"
self.assertEqual(make_key("Deloitte", title), make_key("Deloitte", title))
def test_long_titles_never_collide_after_truncation(self):
"""The bug that produced two Deloitte entries for one posting: two
runs truncated the same long title at different points. A hash of the
full slug makes truncation deterministic instead of lossy."""
a = make_key("Deloitte", "Cyber Intelligence Center Security Analyst with trailing text A")
b = make_key("Deloitte", "Cyber Intelligence Center Security Analyst with trailing text B")
self.assertNotEqual(a, b)
def test_non_latin_title_falls_back_to_the_portal_job_id(self):
key = make_key(
"SecuriON",
"안드로이드 앱(악성코드) 분석가 채용",
url="https://kr.linkedin.com/jobs/view/x-4461771225",
)
self.assertEqual(key, "securion_4461771225")
def test_non_latin_title_with_no_url_id_still_produces_a_canonical_key(self):
key = make_key("SecuriON", "안드로이드 앱 분석가", url="")
self.assertTrue(is_canonical(key))
self.assertNotEqual(key, "securion_")
def test_non_latin_company_falls_back_without_producing_a_bare_prefix(self):
key = make_key("Код Безопасности", "Malware Analytic", url="")
self.assertTrue(is_canonical(key))
self.assertFalse(key.startswith("_"))
class CompanyFallbackCLI(unittest.TestCase):
def key_for(self, company, url="https://example.com/jobs/123456"):
proc = subprocess.run(
[sys.executable, str(TOOL), "--company", company,
"--title", "Software Engineer", "--url", url],
capture_output=True, text=True, encoding="utf-8", check=True,
)
return proc.stdout.strip()
def test_distinct_non_latin_companies_have_distinct_keys(self):
keys = [self.key_for(company) for company in ("腾讯", "阿里巴巴", "")]
self.assertEqual(len(set(keys)), 3)
self.assertTrue(all(is_canonical(key) for key in keys))
def test_company_fallback_is_stable_across_case_normalization_and_urls(self):
for first, second in (("КОМПАНИЯ", "компания"), ("ガンホー", "カ\u3099ンホー")):
with self.subTest(first=first, second=second):
self.assertEqual(
self.key_for(first),
self.key_for(second, "https://example.com/jobs/654321"),
)
def test_missing_company_and_existing_ascii_keys_are_unchanged(self):
self.assertEqual(self.key_for(""), "unknown-company_software-engineer")
self.assertEqual(self.key_for(" "), "unknown-company_software-engineer")
self.assertEqual(self.key_for("Acme Corp"), "acme-corp_software-engineer")
class CanonicalAndLegacyShape(unittest.TestCase):
def test_canonical_accepts_company_underscore_title(self):
self.assertTrue(is_canonical("acme-corp_soc-analyst"))
def test_canonical_rejects_path_breaking_characters(self):
for bad in ("deloitte_junior-cybersecurity-analyst-(ot/iot)",
"neverhack-estonia_penetration-tester-/-red-teamer",
"ops-consulting,-llc_malware-analyst",
"",
"securion_"):
self.assertFalse(is_canonical(bad), f"{bad!r} should not be canonical")
def test_legacy_three_part_shape_is_flagged_separately_from_malformed(self):
self.assertTrue(is_legacy_shape("nviso-security_soc-analyst_athens"))
self.assertFalse(is_canonical("nviso-security_soc-analyst_athens"))
# A malformed key (bad characters) is never also reported as legacy shape.
self.assertFalse(is_legacy_shape("deloitte_junior-cybersecurity-analyst-(ot/iot)"))
class AuditCLI(unittest.TestCase):
def run_audit(self, seen: dict) -> tuple[dict, int]:
import tempfile
with tempfile.NamedTemporaryFile("w", suffix=".json", delete=False) as fh:
json.dump({"seen": seen}, fh)
path = fh.name
proc = subprocess.run(
[sys.executable, str(TOOL), "--audit", path], capture_output=True, text=True, encoding="utf-8"
)
return json.loads(proc.stdout), proc.returncode
def test_clean_state_exits_zero(self):
report, code = self.run_audit({"acme_soc-analyst": {"company": "Acme", "title": "SOC Analyst"}})
self.assertEqual(code, 0)
self.assertEqual(report["malformed_keys"], [])
self.assertEqual(report["duplicate_urls"], {})
def test_malformed_key_exits_nonzero(self):
report, code = self.run_audit(
{"deloitte_junior-cybersecurity-analyst-(ot/iot)": {"company": "Deloitte", "title": "x"}}
)
self.assertEqual(code, 1)
self.assertIn("deloitte_junior-cybersecurity-analyst-(ot/iot)", report["malformed_keys"])
def test_duplicate_url_exits_nonzero(self):
report, code = self.run_audit(
{
"a": {"company": "Acme", "title": "x", "url": "https://x/1"},
"b": {"company": "Acme", "title": "y", "url": "https://x/1"},
}
)
self.assertEqual(code, 1)
self.assertIn("https://x/1", report["duplicate_urls"])
def test_legacy_shape_alone_does_not_fail_the_audit(self):
"""Harmless drift, not damage - the sweep-worthy rewrite is a decision
the maintainer makes, not something the audit enforces."""
report, code = self.run_audit({"acme_soc-analyst_athens": {"company": "Acme", "title": "x"}})
self.assertEqual(code, 0)
self.assertIn("acme_soc-analyst_athens", report["legacy_three_part_keys"])
if __name__ == "__main__":
unittest.main()
+40
View File
@@ -136,5 +136,45 @@ class TestAtsExtractionEncoding(unittest.TestCase):
self.assert_pdftotext_commands_pin_utf8(CV_TEMPLATES)
class TestPdflatexFontEncodingGuard(unittest.TestCase):
"""#384: the pdflatex fallback must load T1 fontenc, and only under pdflatex.
Without T1, pdflatex stores accented letters decomposed in the text layer
(`e` + U+0300), so an ATS keyword match on `Genève` fails while the PDF
looks right. moderncv 2.5 loads T1 itself; the apt-packaged 2.3.1 does not.
The line must be guarded so the documented lualatex path is untouched.
"""
GUARDED_FONTENC = re.compile(r"\\ifpdftex\s*\\usepackage\[T1\]\{fontenc\}\s*\\fi")
def assert_has_guarded_fontenc(self, path):
text = path.read_text(encoding="utf-8")
self.assertRegex(
text,
self.GUARDED_FONTENC,
f"{path.name} must carry `\\ifpdftex\\usepackage[T1]{{fontenc}}\\fi` so a "
"pdflatex fallback keeps accents precomposed in the text layer",
)
unguarded = [
f"{path.name}:{lineno}: {line.strip()}"
for lineno, line in enumerate(text.splitlines(), 1)
if "fontenc" in line
and not line.lstrip().startswith("%")
and not self.GUARDED_FONTENC.search(line)
]
self.assertEqual(
unguarded,
[],
"fontenc must stay inside the \\ifpdftex guard - lualatex output "
"must not change:\n" + "\n".join(unguarded),
)
def test_example_cv_guards_fontenc_for_pdflatex(self):
self.assert_has_guarded_fontenc(EXAMPLE_CV)
def test_cv_guide_preamble_guards_fontenc_for_pdflatex(self):
self.assert_has_guarded_fontenc(CV_TEMPLATES)
if __name__ == "__main__":
unittest.main()
+65
View File
@@ -0,0 +1,65 @@
"""Guards for /outcome's stale sweep branch (Step 2c).
Pins the invariants for batch-resolving quiet applications:
- Step 0 documents `stale` / `sweep` and `stale <N>` / `sweep <N>`.
- Step 1.3 offers stale sweep when open rows exceed 60 days quiet.
- Step 2c defines the Stale Sweep Branch.
- Drafted applications are strictly excluded (never submitted).
- The default threshold is 60 days quiet.
- User confirmation (all, select, skip) is strictly required before writing.
- Status is resolved to canonical 'no_response' spelling.
"""
import re
import unittest
from pathlib import Path
REPO = Path(__file__).resolve().parent.parent
COMMAND = REPO / ".claude" / "commands" / "outcome.md"
class OutcomeStaleBranchSpecTests(unittest.TestCase):
def setUp(self):
self.text = COMMAND.read_text(encoding="utf-8")
def test_stale_argument_documented_in_step0(self):
self.assertIn("`stale` or `sweep`", self.text)
self.assertIn("`stale <N>` or `sweep <N>`", self.text)
def test_step1_suggests_stale_sweep(self):
self.assertIn("/outcome stale", self.text)
self.assertIn("60+ days", self.text)
def test_step2c_section_exists(self):
self.assertIn("## Step 2c: Stale Sweep Branch", self.text)
def test_drafted_rows_excluded_from_stale_candidates(self):
match = re.search(r"## Step 2c: Stale Sweep Branch(.*?)(?=## Step 3:)", self.text, re.DOTALL)
self.assertTrue(match, "Step 2c must exist")
step2c = match.group(1)
self.assertIn("neither final nor `drafted`", step2c)
self.assertIn("never submitted and cannot receive a response", step2c)
def test_default_60_day_threshold(self):
match = re.search(r"## Step 2c: Stale Sweep Branch(.*?)(?=## Step 3:)", self.text, re.DOTALL)
self.assertTrue(match)
step2c = match.group(1)
self.assertIn("60 days", step2c)
def test_user_confirmation_options_required(self):
match = re.search(r"## Step 2c: Stale Sweep Branch(.*?)(?=## Step 3:)", self.text, re.DOTALL)
self.assertTrue(match)
step2c = match.group(1)
self.assertIn("`all`", step2c)
self.assertIn("`select`", step2c)
self.assertIn("`skip`", step2c)
def test_resolves_to_canonical_no_response(self):
match = re.search(r"## Step 2c: Stale Sweep Branch(.*?)(?=## Step 3:)", self.text, re.DOTALL)
self.assertTrue(match)
step2c = match.group(1)
self.assertIn("no_response", step2c)
if __name__ == "__main__":
unittest.main()
+89 -1
View File
@@ -10,6 +10,7 @@ matches Step 4's existing rules exactly - the location_verdict legacy
migration, the deadline null-is-not-a-correction rule, and verbatim
strengths/gaps persistence.
"""
import csv
import json
import subprocess
import sys
@@ -52,6 +53,7 @@ class RankStateCase(unittest.TestCase):
[sys.executable, str(TOOL), *args, "--state", str(self.state), "--today", TODAY],
capture_output=True,
text=True,
encoding="utf-8",
)
self.assertEqual(proc.returncode, expect, proc.stderr)
return json.loads(proc.stdout)
@@ -105,6 +107,58 @@ class Candidates(RankStateCase):
self.assertEqual([row["key"] for row in out["selected"]], ["b"])
self.assertEqual(out["excluded_by_tracker"], 1)
def test_tracker_exclusion_handles_utf8_bom_and_reordered_columns(self):
self.write_state({"a": entry(company="Acme", title="SOC Analyst"), "b": entry(company="Other")})
tracker = self.tmp / "tracker.csv"
# A spreadsheet export can prepend a BOM to either matching column.
for encoding in ("utf-8", "utf-8-sig"):
for csv_text in (
"date,company,role\r\n2026-08-01,ACME,soc analyst\r\n",
"company,role,date\r\nACME,soc analyst,2026-08-01\r\n",
"role,company,date\r\nsoc analyst,ACME,2026-08-01\r\n",
):
with self.subTest(encoding=encoding, header=csv_text.splitlines()[0]):
tracker.write_bytes(csv_text.encode(encoding))
out = self.run_tool("candidates", "--tracker", str(tracker))
self.assertEqual([row["key"] for row in out["selected"]], ["b"])
self.assertEqual(out["excluded_by_tracker"], 1)
def test_tracker_exclusion_preserves_unicode_identity(self):
# Use the standard /outcome header and the real candidates CLI.
header = (
"date,company,sector,role,role_type,channel,status,contact_person,"
"fit_rating,notes,cv_file,cover_letter_file,source,deadline"
).split(",")
cases = (
# Tracked company/role, candidate company/title, expected exclusion.
("Acme", "设计师", "Acme", "工程师", False),
("Acme", "工程师", "Acme", "工程师", True),
("腾讯", "Engineer", "腾讯", "Engineer", True),
("腾讯", "Engineer", "阿里巴巴", "Engineer", False),
("Компания", "Инженер", "КОМПАНИЯ", "ИНЖЕНЕР", True),
("Café", "Engineer", "Cafe\u0301", "Engineer", True),
("Straße", "Engineer", "STRASSE", "Engineer", True),
("कला Labs", "Engineer", "कल Labs", "Engineer", False),
("Acme, Inc.", "SOC Analyst", "ACME_INC", "soc-analyst", True),
)
tracker = self.tmp / "tracker.csv"
for company, role, candidate_company, title, excluded in cases:
with self.subTest(tracked=(company, role), candidate=(candidate_company, title)):
self.write_state({
"candidate": entry(company=candidate_company, title=title),
"other": entry(company="Other", title="Untracked role"),
})
with tracker.open("w", encoding="utf-8", newline="") as fh:
writer = csv.DictWriter(fh, fieldnames=header)
writer.writeheader()
writer.writerow({"date": TODAY, "company": company, "role": role, "status": "applied"})
out = self.run_tool("candidates", "--tracker", str(tracker))
self.assertEqual(
[row["key"] for row in out["selected"]],
["other"] if excluded else ["candidate", "other"],
)
self.assertEqual(out["excluded_by_tracker"], int(excluded))
def test_focus_filters_on_title_company_and_stored_fit_notes(self):
self.write_state(
{
@@ -132,7 +186,7 @@ class Candidates(RankStateCase):
proc = subprocess.run(
[sys.executable, str(TOOL), "candidates", "--state", str(self.tmp / "nope.json"),
"--tracker", str(self.tmp / "n.csv")],
capture_output=True, text=True,
capture_output=True, text=True, encoding="utf-8",
)
self.assertNotEqual(proc.returncode, 0)
self.assertIn("not found", proc.stderr + proc.stdout)
@@ -415,6 +469,40 @@ class Apply(RankStateCase):
)
self.assertEqual(self.read_state()["a"]["status"], "new")
def test_invalid_scores_report_errors_without_changing_the_entry(self):
dimensions = ("technical", "experience", "behavioral", "career")
invalid = (-1, 101, True, False, float("nan"), float("inf"), -float("inf"), 10**400)
for dimension in dimensions:
for value in invalid:
with self.subTest(dimension=dimension, value=value):
original = entry(status="ranked", rank_score=70, strengths=["keep"])
self.write_state({"a": original, "b": entry()})
scores = dict.fromkeys(dimensions, 50)
scores[dimension] = value
out = self.run_tool(
"apply", "--results", self.results([
{"key": "a", "status": "scored", "scores": scores},
{"key": "b", "status": "scored", "scores": dict.fromkeys(dimensions, 80)},
]), expect=1,
)
self.assertEqual(len(out["errors"]), 1)
self.assertEqual(out["errors"][0]["key"], "a")
self.assertIn(dimension, out["errors"][0]["error"])
self.assertEqual(self.read_state()["a"], original)
self.assertEqual([row["key"] for row in out["ranked"]], ["b"])
self.assertEqual(self.read_state()["b"]["rank_score"], 80)
def test_score_boundaries_and_fractional_scores_remain_valid(self):
for value, expected in ((0, 0), (100, 100), (72.5, 73)):
with self.subTest(value=value):
self.write_state({"a": entry()})
out = self.run_tool("apply", "--results", self.results([
{"key": "a", "status": "scored", "scores": dict.fromkeys(
("technical", "experience", "behavioral", "career"), value)},
]))
self.assertEqual(out["errors"], [])
self.assertEqual(self.read_state()["a"]["rank_score"], expected)
def test_re_scoring_an_already_ranked_job_is_idempotent(self):
"""Re-running /rank never re-scores an already-ranked job unless --all
says so (Step 4), but if it does score one again, apply must produce
+54
View File
@@ -259,6 +259,60 @@ class TestArgumentHardening(unittest.TestCase):
self.assertEqual(seen[0], "https://x.example/robots.txt")
class TestNonAsciiCurlOutput(unittest.TestCase):
"""curl's output is decoded as UTF-8, never the locale's code page.
With text=True and no encoding, Windows decoded with the locale's code page.
Under cp1254 (Turkish) the bytes 0x81 and 0x9e are undefined, so the pipe
reader thread died, stdout came back None, and the gate printed
"UNCONFIRMED (AttributeError)" for policies it never read - blocking a
retry those policies allow (reproduced 2026-09-29).
"""
def _gate_with_curl_output(self, url, body, status):
"""gate(url) with curl swapped for a process that prints `body` and the
status the way curl's -w does, decoded however _fetch asks."""
import robots_check
# A fixture cp1254 can decode would pass without the fix.
self.assertRaises(UnicodeDecodeError, body.decode, "cp1254")
raw = body + b"\n" + str(status).encode()
real_run = subprocess.run
def fake_curl(argv, **kwargs):
# Unpinned text mode takes the host locale's code page. Pin the
# reporter's, so a UTF-8 host (CI) cannot pass the bug by luck.
if kwargs.get("text") and not kwargs.get("encoding"):
kwargs["encoding"] = "cp1254"
write = "import sys; sys.stdout.buffer.write(bytes.fromhex(sys.argv[1]))"
return real_run([sys.executable, "-c", write, raw.hex()], **kwargs)
robots_check.subprocess.run = fake_curl
try:
return robots_check.gate(url)
finally:
robots_check.subprocess.run = real_run
def test_rule_holding_a_cp1254_undefined_byte_is_read_and_obeyed(self):
"""The rule is verbatim from tr.indeed.com/robots.txt: 職 is e8 81 b7.
DISALLOWED on its path proves it was decoded, not merely survived."""
body = "User-agent: *\nDisallow: /職涯貼士/\n".encode("utf-8")
rc, msg = self._gate_with_curl_output("https://tr.indeed.example/cmp/x/reviews", body, 200)
self.assertEqual(rc, 0, msg)
self.assertIn("robots.txt permits this path", msg)
rc, msg = self._gate_with_curl_output("https://tr.indeed.example/職涯貼士/x", body, 200)
self.assertEqual(rc, 1, msg)
self.assertIn("DISALLOWED", msg)
def test_404_page_holding_a_cp1254_undefined_byte_still_means_no_policy(self):
"""kap.org.tr answers /robots.txt with a UTF-8 HTML 404 that names
"Merkezi Kayıt Kuruluşu A.Ş." - Ş is c5 9e."""
body = '<html lang="tr"><body>Merkezi Kayıt Kuruluşu A.Ş.</body></html>'.encode("utf-8")
rc, msg = self._gate_with_curl_output("https://kap.example/tr/sirket-bilgileri", body, 404)
self.assertEqual(rc, 0, msg)
self.assertIn("no robots.txt published", msg)
if __name__ == "__main__":
unittest.main()
+33
View File
@@ -25,6 +25,39 @@ from salary_lookup import (
# ---------------------------------------------------------------------------
class FormatEntryTests(unittest.TestCase):
PRIVACY_FOOTNOTE = "* N/A = Too few employees to publish (privacy)"
def test_privacy_footnote_is_omitted_when_no_row_is_suppressed(self):
# The footnote explains the N/A* marker. Printed under a table where
# every row has an index, it asserts a privacy suppression that never
# happened (residual noted on #470).
entry = {
"company": "Example Corp",
"city": "",
"categories": {"all_employees": {"count": 500, "index": 108.5}},
}
rendered = format_entry(entry, {"index_baseline": 100, "index_label": "Index"})
self.assertNotIn("N/A", rendered)
self.assertNotIn(self.PRIVACY_FOOTNOTE, rendered)
self.assertRegex(rendered, r"All Employees\s+500\s+108\.5\s+\+8\.5%")
def test_privacy_footnote_is_printed_when_a_row_is_suppressed(self):
entry = {
"company": "Example Corp",
"city": "",
"categories": {
"all_employees": {"count": 500, "index": 108.5},
"small_team": {"count": 3, "index": None},
},
}
rendered = format_entry(entry, {"index_baseline": 100, "index_label": "Index"})
self.assertRegex(rendered, r"Small Team\s+3\s+N/A\*")
self.assertIn(self.PRIVACY_FOOTNOTE, rendered)
def test_zero_count_is_displayed_as_zero(self):
entry = {
"company": "Example Corp",
+16
View File
@@ -136,5 +136,21 @@ class SeenJobsPostingDateTests(unittest.TestCase):
)
class SeenJobsDedupContinuityTests(unittest.TestCase):
"""The new key rule must not replay jobs stored under legacy keys."""
def test_existing_urls_are_seen_regardless_of_key(self):
text = SCRAPER_SKILL.read_text(encoding="utf-8")
self.assertRegex(
text,
r"URL matches any existing `seen_jobs\.json` entry, regardless of\s+that entry's key",
"legacy seen_jobs entries must be matched by URL during the key-rule transition",
)
def test_step4_presentation_mentions_url_deduplication(self):
text = SCRAPER_SKILL.read_text(encoding="utf-8")
self.assertRegex(text, r"matched by URL or\s+company\+title")
if __name__ == "__main__":
unittest.main()
+30
View File
@@ -9,9 +9,12 @@ were not, so a full Path B/C run left `[YOUR_NAME]`, `[YOUR_EMAIL]` and
on the drafter noticing. A real user (#420) ran `/setup` and then hand-edited both
files to close the gap.
"""
import os
import unittest
from pathlib import Path
UPSTREAM = "MadsLorentzen/ai-job-search"
REPO = Path(__file__).resolve().parent.parent
COMMAND = REPO / ".claude" / "commands" / "setup.md"
SKILL_DIR = REPO / ".claude" / "skills" / "job-application-assistant"
@@ -67,6 +70,10 @@ class SetupStep3ContactBlocks(unittest.TestCase):
self.assertIn("06-cover-letter-templates.md", summary)
@unittest.skipIf(
os.environ.get("GITHUB_REPOSITORY", UPSTREAM) != UPSTREAM,
"template-placeholder guard targets the pristine upstream template; forks personalize 05-cv-templates.md and 06-cover-letter-templates.md via /setup",
)
class TemplatesStillCarryThePlaceholders(unittest.TestCase):
"""The instructions above target real tokens; if a template renames them,
the instruction and this test must move together."""
@@ -83,5 +90,28 @@ class TemplatesStillCarryThePlaceholders(unittest.TestCase):
self.assertIn("\\signature{[YOUR_NAME]}", text)
class SetupPathAProjectsIngestion(unittest.TestCase):
"""Guards for /setup Path A document ingestion of documents/projects/."""
def setUp(self):
self.text = COMMAND.read_text(encoding="utf-8")
self.sections = _sections(self.text)
def test_step0_scan_includes_projects(self):
step0 = self.sections["Step 0: Welcome & Choose Path"]
self.assertIn("projects/", step0)
def test_step_a1_inventory_includes_projects(self):
self.assertIn("**projects/**:", self.text)
def test_step_a3_parsing_includes_projects_spec(self):
self.assertIn("`projects/` documents:", self.text)
self.assertIn("measurable outcomes", self.text)
def test_step_a5_and_a6_map_to_independent_projects(self):
self.assertIn("## Independent Projects", self.text)
self.assertIn("New independent project:", self.text)
if __name__ == "__main__":
unittest.main()
+154
View File
@@ -0,0 +1,154 @@
"""The Python tools must write UTF-8 whatever the host's default encoding is.
On Windows a piped stdout (which is how Claude Code runs every tool) defaults
to the ANSI code page, cp1252 on most Western installs. A tool that prints a
posting title, company, CV line or file name outside that code page then dies
with UnicodeEncodeError before the workflow sees any output: `/rank` could not
list a single Cyrillic, CJK, Devanagari, Polish or Turkish posting.
Each case runs the real CLI in a child process with a legacy stdout forced via
PYTHONIOENCODING, so the Linux CI job reproduces what Windows users hit. The
child must exit cleanly and its bytes must decode as UTF-8.
"""
import json
import os
import shutil
import subprocess
import sys
import unittest
from pathlib import Path
from tempfile import TemporaryDirectory
try:
import openpyxl
except ImportError:
openpyxl = None
REPO = Path(__file__).resolve().parent.parent
TOOLS = REPO / "tools"
# One name per script family that cp1252 cannot encode.
CYRILLIC = "Яндекс"
CJK = "腾讯"
POLISH = "Żabka Łódź"
def run_legacy_stdout(argv, cwd=None):
"""Run a tool the way a Western-locale Windows host pipes it."""
env = dict(os.environ, PYTHONIOENCODING="cp1252", PYTHONUTF8="0")
return subprocess.run([sys.executable, *map(str, argv)], cwd=cwd, env=env, capture_output=True)
class ToolsWriteUtf8(unittest.TestCase):
def setUp(self):
directory = TemporaryDirectory()
self.addCleanup(directory.cleanup)
self.tmp = Path(directory.name)
def assert_clean_utf8(self, proc, expect_code, *needles, stream="stdout"):
err = proc.stderr.decode("utf-8", "replace")
self.assertNotIn("UnicodeEncodeError", err)
self.assertEqual(proc.returncode, expect_code, err)
text = getattr(proc, stream).decode("utf-8")
for needle in needles:
self.assertIn(needle, text)
return text
def write_state(self, seen):
state = self.tmp / "seen_jobs.json"
state.write_text(json.dumps({"version": 1, "seen": seen}, ensure_ascii=False), encoding="utf-8")
return state
@staticmethod
def entry(company, title):
return {
"title": title, "company": company, "url": "https://example.com/jobs/123456",
"portal": "example", "status": "new", "first_seen": "2026-09-20",
}
def test_rank_candidates_lists_non_latin_postings(self):
state = self.write_state({"a": self.entry(CYRILLIC, "Инженер"), "b": self.entry(CJK, "工程师")})
proc = run_legacy_stdout([TOOLS / "rank_state.py", "candidates", "--state", state, "--today", "2026-09-21"])
out = json.loads(self.assert_clean_utf8(proc, 0, CYRILLIC, CJK))
self.assertEqual(out["eligible"], 2)
def test_rank_apply_prints_the_ranking_for_non_latin_postings(self):
state = self.write_state({"a": self.entry(CJK, "工程师")})
results = self.tmp / "results.json"
results.write_text(json.dumps([{
"key": "a", "status": "scored",
"scores": {"technical": 80, "experience": 80, "behavioral": 80, "career": 80},
}]), encoding="utf-8")
proc = run_legacy_stdout([
TOOLS / "rank_state.py", "apply", "--results", results, "--state", state, "--today", "2026-09-21",
])
out = json.loads(self.assert_clean_utf8(proc, 0, CJK))
self.assertEqual([row["key"] for row in out["ranked"]], ["a"])
def test_job_key_audit_reports_non_latin_entries(self):
state = self.write_state({f"{CYRILLIC}_Инженер": self.entry(CYRILLIC, "Инженер")})
proc = run_legacy_stdout([TOOLS / "job_key.py", "--audit", state])
self.assertNotIn(b"UnicodeEncodeError", proc.stderr)
self.assertIn(CYRILLIC, proc.stdout.decode("utf-8"))
def test_salary_lookup_prints_a_company_outside_cp1252(self):
shutil.copy(REPO / "salary_lookup.py", self.tmp / "salary_lookup.py")
(self.tmp / "salary_data.json").write_text(json.dumps({
"metadata": {"source": "fixture", "index_baseline": 100, "index_label": "Index",
"baseline_description": "Index 100 = baseline"},
"companies": [{"company": POLISH, "city": "Łódź",
"categories": {"all_employees": {"count": 50, "index": 104.0}}}],
}, ensure_ascii=False), encoding="utf-8")
proc = run_legacy_stdout([self.tmp / "salary_lookup.py", "Żabka"], cwd=self.tmp)
self.assert_clean_utf8(proc, 0, POLISH)
def test_verify_pdf_names_a_non_latin_file_in_its_error(self):
missing = self.tmp / f"main_{CJK}_engineer.pdf"
proc = run_legacy_stdout([TOOLS / "verify_pdf.py", missing])
self.assert_clean_utf8(proc, 1, CJK, stream="stderr")
def test_verify_layout_reports_a_non_latin_cv_line(self):
# Poppler is not installed in the Python CI job, so feed main() synthetic
# page geometry (as test_verify_layout.py does) but still in a child
# process, because the encoding under test belongs to the process.
driver = self.tmp / "drive_verify_layout.py"
driver.write_text(
"import sys\n"
"from unittest.mock import patch\n"
f"sys.path.insert(0, {str(REPO)!r})\n"
"from tools import verify_layout as v\n"
"A4 = 842.0\n"
"def line(text, top, left=70.0, height=10.0):\n"
" return v.Line(top=top, bottom=top + height, left=left, height=height, text=text)\n"
"pages = [\n"
f" v.Page(height=A4, lines=[line('Profil', 100.0), line({POLISH!r}, 700.0, left=60.0)]),\n"
" v.Page(height=A4, lines=[line('kontynuacja punktu', 80.0, left=90.0), line('dalej', 700.0, left=90.0)]),\n"
"]\n"
"with patch.object(v, 'parse_pdf', return_value=pages):\n"
" sys.argv = ['verify_layout.py', __file__]\n"
" sys.exit(v.main())\n",
encoding="utf-8",
)
proc = run_legacy_stdout([driver])
self.assertNotIn(b"UnicodeEncodeError", proc.stderr)
self.assertIn(POLISH, proc.stdout.decode("utf-8"))
@unittest.skipUnless(openpyxl, "requires optional openpyxl (installed in CI)")
def test_salary_converter_names_a_non_latin_worksheet(self):
workbook = self.tmp / "salary.xlsx"
book = openpyxl.Workbook()
try:
book.active.title = CYRILLIC
book.active.append(["Company", "City", "Count", "Index"])
book.active.append([CYRILLIC, "Москва", 40, 101.5])
book.save(workbook)
finally:
book.close()
proc = run_legacy_stdout([
TOOLS / "convert_salary_excel.py", workbook, "--output", self.tmp / "out.json",
])
self.assert_clean_utf8(proc, 0, CYRILLIC)
if __name__ == "__main__":
unittest.main()
+135
View File
@@ -0,0 +1,135 @@
"""Guards for the rule that keeps free-form `notes` from breaking a tracker row.
No writer in this framework emits a quoted tracker field, so a comma inside
`notes` splits the row - for the `csv.DictReader` in `tools/rank_state.py` as
much as for a naive split - and shifts `cv_file`, `cover_letter_file` and
`source` a column left. A line break is worse: it ends the row. Two writers put
free-form text into `notes`: `/gmail-sync` Step 7a copies an email subject, and
`/outcome` Step 4 writes a short note of its own. The fixed-format writers
(`followed up YYYY-MM-DD`, `stale resolved no_response (YYYY-MM-DD)`,
`redrafted`) cannot contain the characters and are not listed.
The spec IS the implementation, so the guard is `CASES`: each rule must sit on
the line that instructs the append, not merely somewhere in the section. The
shape tests below it parse with `csv.DictReader` and document why the rule
exists; they pass on master too.
"""
import csv
import io
import re
import unittest
from pathlib import Path
REPO = Path(__file__).resolve().parent.parent
COMMANDS = REPO / ".claude" / "commands"
GMAIL_SYNC = COMMANDS / "gmail-sync.md"
OUTCOME = COMMANDS / "outcome.md"
TRACKER_HEADER = (
"date,company,sector,role,role_type,channel,status,contact_person,"
"fit_rating,notes,cv_file,cover_letter_file,source,deadline"
)
def section(path, heading):
"""The body of one markdown section, up to the next heading of any depth."""
text = path.read_text(encoding="utf-8")
start = text.index(heading) + len(heading)
rest = text[start:]
end = re.search(r"^#{1,4} ", rest, re.MULTILINE)
return rest[: end.start()] if end else rest
class FreeFormNotesWritersStateTheRule(unittest.TestCase):
"""Format: (path, heading, line_anchor, rule, why)"""
CASES = [
(
GMAIL_SYNC,
"### Step 7a: Write Approved Updates",
"append to `notes`",
"with every comma, double quote and line break deleted from the subject first",
"Step 7a item 2 deliberately keeps the subject verbatim in `outcome.md`, "
"so the rule must sit on the tracker append, not anywhere in the step",
),
(
OUTCOME,
"## Step 4: Update the Tracker",
"append a short dated note",
"containing no commas, double quotes or line breaks",
"Step 4 is the primary status-update path and its note is written "
"free-form - `rejected, no feedback given` is the natural sentence",
),
]
def test_rule_is_stated_where_the_append_happens(self):
for path, heading, anchor, rule, why in self.CASES:
with self.subTest(path=path.name, heading=heading):
lines = [l for l in section(path, heading).splitlines() if anchor in l]
self.assertEqual(len(lines), 1, f"expected one append instruction: {why}")
self.assertIn(rule, lines[0], why)
class NotesShapeUnderTheShippedReader(unittest.TestCase):
"""Why the rule exists, demonstrated with the reader the repo ships."""
SUBJECT = 'Re: Your application, Data Scientist - "next steps"'
def test_sanitised_note_keeps_the_row_parseable(self):
safe = self.SUBJECT.replace(",", "").replace('"', "")
rows = self._parse(f'2026-09-12 gmail-sync: acknowledged ("{safe}")')
self.assertEqual(len(rows), 1, "the note must not end the row early")
row = rows[0]
self.assertIsNone(row.get(None), "the row must be no wider than the header")
self.assertEqual(row["cv_file"], "cv/main_acme_data_scientist.tex")
self.assertEqual(
row["cover_letter_file"], "cover_letters/cover_acme_data_scientist.tex"
)
self.assertEqual(row["source"], "linkedin")
def test_a_comma_in_the_note_shifts_the_columns(self):
for note in (
f'2026-09-12 gmail-sync: acknowledged ("{self.SUBJECT}")',
"2026-09-12 rejected, no feedback given",
):
with self.subTest(note=note):
row = self._parse(note)[0]
self.assertIsNotNone(row.get(None), "the comma must widen the row")
self.assertNotEqual(row["cv_file"], "cv/main_acme_data_scientist.tex")
self.assertNotEqual(row["source"], "linkedin")
def test_a_line_break_in_the_note_splits_the_row_in_two(self):
rows = self._parse('2026-09-12 gmail-sync: acknowledged ("Re: update\nlater")')
self.assertEqual(len(rows), 2)
self.assertIsNone(rows[0]["cv_file"], "the first row ends mid-note")
@classmethod
def _parse(cls, notes):
stream = io.StringIO(TRACKER_HEADER + "\n" + cls._row(notes) + "\n")
return list(csv.DictReader(stream))
@staticmethod
def _row(notes):
return ",".join(
[
"2026-09-01",
"Acme",
"tech",
"Data Scientist",
"full_time",
"portal",
"applied",
"",
"8",
notes,
"cv/main_acme_data_scientist.tex",
"cover_letters/cover_acme_data_scientist.tex",
"linkedin",
"2026-09-30",
]
)
if __name__ == "__main__":
unittest.main()
+209
View File
@@ -0,0 +1,209 @@
"""Offline tests for tools/verify_layout.py.
Every case is built from synthetic Page/Line geometry rather than a compiled
PDF, so the suite needs neither Poppler nor a LaTeX toolchain - matching the
repo's CI policy of keeping the Python tool tests self-contained.
The cases marked SILENT FAILURE are the ones that motivated the tool: each
describes a document that compiles cleanly, reports the expected page count,
and passes tools/verify_pdf.py, while the rendered page is visibly broken.
"""
import io
import subprocess
import unittest
from contextlib import redirect_stdout
from pathlib import Path
from unittest.mock import patch
from tools import verify_layout
from tools.verify_layout import Line, Page, find_orphans, main, parse_pdf, report
A4_HEIGHT = 842.0
def line(top: float, left: float = 50.0, height: float = 10.0, text: str = "x") -> Line:
return Line(top=top, bottom=top + height, left=left, height=height, text=text)
class TestGapAndBottomSpace(unittest.TestCase):
def setUp(self):
# Three body lines, then a 322pt jump, then the page-number footer.
self.holed = Page(
A4_HEIGHT,
[line(50), line(64), line(78), line(400), line(770, text="1/2")],
)
def test_largest_gap_reports_size_and_position(self):
"""SILENT FAILURE: the hole an ejected \\cventry leaves behind."""
gap, y = self.holed.largest_gap()
self.assertEqual((round(gap), round(y)), (322, 78))
def test_bottom_space_ignores_the_footer_band(self):
"""Measured to the last body line (y410), not to the page number at y770."""
self.assertEqual(round(self.holed.bottom_space), 432)
def test_page_with_no_body_lines_is_empty(self):
self.assertTrue(Page(A4_HEIGHT, []).empty)
def test_report_flags_the_hole(self):
with redirect_stdout(io.StringIO()): # report() prints its per-page measurements
problems = report(Path("synthetic"), [self.holed])
self.assertTrue(any("hole" in m for m in problems), problems)
class TestFooterBand(unittest.TestCase):
def test_single_line_in_band_is_just_the_page_number(self):
self.assertFalse(Page(A4_HEIGHT, [line(50), line(800, text="2/2")]).footer_crowded)
def test_two_lines_in_band_means_body_text_spilled_in(self):
"""SILENT FAILURE: \\enlargethispage pushing body text over the footer."""
self.assertTrue(Page(A4_HEIGHT, [line(50), line(780), line(800)]).footer_crowded)
class TestHeadingAndIndentDetection(unittest.TestCase):
def setUp(self):
self.page = Page(
A4_HEIGHT,
[
line(50, height=16.0, text="Professional Experience"),
line(80, left=50.0),
line(94, left=70.0),
],
)
def test_taller_line_is_a_heading(self):
self.assertTrue(self.page.is_heading(self.page.body[0]))
self.assertFalse(self.page.is_heading(self.page.body[1]))
def test_left_edge_separates_bullets_from_headers(self):
self.assertTrue(self.page.is_indented(self.page.body[2]))
self.assertFalse(self.page.is_indented(self.page.body[1]))
class TestOrphans(unittest.TestCase):
def test_page_ending_on_a_section_heading(self):
"""SILENT FAILURE: a heading stranded at the bottom, content overleaf."""
p1 = Page(A4_HEIGHT, [line(50), line(64), line(700, height=16.0, text="Education")])
p2 = Page(A4_HEIGHT, [line(60, text="Example University"), line(74, left=70.0)])
self.assertTrue(any("ends on the section heading" in m for m in find_orphans([p1, p2])))
def test_entry_header_orphaned_from_its_bullets(self):
"""SILENT FAILURE: the \\cventry title on one page, its bullets on the next."""
q1 = Page(A4_HEIGHT, [line(50), line(700, left=50.0, text="Software Engineer")])
q2 = Page(A4_HEIGHT, [line(60, left=70.0, text="- built the thing")])
self.assertTrue(any("orphaned from its bullets" in m for m in find_orphans([q1, q2])))
def test_lone_list_marker_is_a_split_bullet_not_an_orphaned_header(self):
"""moderncv gives the itemize marker its own bbox line: different defect, different fix."""
m1 = Page(A4_HEIGHT, [line(50), line(700, left=50.0, text="●")])
m2 = Page(A4_HEIGHT, [line(60, left=70.0, text="continued item text here")])
self.assertTrue(any("lone list marker" in m for m in find_orphans([m1, m2])))
def test_clean_break_reports_nothing(self):
r1 = Page(A4_HEIGHT, [line(50), line(700, left=50.0)])
r2 = Page(A4_HEIGHT, [line(60, left=50.0)])
self.assertEqual(find_orphans([r1, r2]), [])
def test_indent_is_judged_against_the_document_margin(self):
"""A page that OPENS with bullets must not mistake their indent for its margin."""
s1 = Page(A4_HEIGHT, [line(50, left=50.0), line(700, left=50.0, text="Data Analyst")])
s2 = Page(A4_HEIGHT, [line(60, left=70.0, text="- first bullet")])
self.assertTrue(any("orphaned from its bullets" in m for m in find_orphans([s1, s2])))
class TestExtractorFailure(unittest.TestCase):
"""A broken extractor must not masquerade as a broken document.
Git for Windows ships an xpdf-based pdftotext with no -bbox flag; it shadows
Poppler in a default PATH and exits 99. Reported as a layout problem it would
send /apply chasing a phantom hole, so it has to land on the skip path.
"""
def test_pdftotext_without_bbox_raises_a_skippable_error(self):
failure = subprocess.CalledProcessError(99, "pdftotext", stderr="Error: unknown flag")
with patch("tools.verify_layout.shutil.which", return_value="/usr/bin/pdftotext"), patch(
"tools.verify_layout.subprocess.run", side_effect=failure
):
with self.assertRaisesRegex(RuntimeError, "bounding boxes"):
parse_pdf(Path("cv/main_example.pdf"))
def test_poppler_abort_on_empty_info_string_names_that_cause_too(self):
"""Poppler 26.0x before 26.05 aborts -bbox on an empty Info-dict string, e.g.
the empty /Title hyperref writes when pdftitle is unset (#451). That crash is
not the xpdf-shadowing case - it has no -bbox flag and exits 99 - so the
message must name both, not just the one the exit code happens to match.
"""
failure = subprocess.CalledProcessError(
1,
"pdftotext",
stderr="libc++abi: terminating due to uncaught exception of type "
"std::out_of_range: basic_string",
)
with patch("tools.verify_layout.shutil.which", return_value="/usr/bin/pdftotext"), patch(
"tools.verify_layout.subprocess.run", side_effect=failure
):
with self.assertRaisesRegex(RuntimeError, "bounding boxes") as ctx:
parse_pdf(Path("cv/main_example.pdf"))
message = str(ctx.exception)
self.assertIn("xpdf", message)
self.assertIn("Poppler aborted", message)
self.assertIn("hyperref", message)
def test_missing_poppler_raises_a_skippable_error(self):
with patch("tools.verify_layout.shutil.which", return_value=None):
with self.assertRaisesRegex(RuntimeError, "not found"):
parse_pdf(Path("cv/main_example.pdf"))
def test_extractor_failure_exits_2_not_1(self):
"""Exit 1 means "your document is broken"; a dead extractor must never claim that."""
with patch("tools.verify_layout.parse_pdf", side_effect=RuntimeError("no -bbox")), patch(
"sys.argv", ["verify_layout.py", __file__]
):
err = io.StringIO()
with redirect_stdout(io.StringIO()), patch("sys.stderr", err):
self.assertEqual(main(), 2)
self.assertIn("skipped:", err.getvalue())
class ToolsCompileWithoutWarnings(unittest.TestCase):
"""Every tool's source must compile clean.
`verify_layout.py`'s docstring documents LaTeX macros, and a bare `\\h` in a
non-raw docstring is an invalid escape sequence: Python 3.12+ emits a
SyntaxWarning when the module is compiled (CPython gh-98401), 3.10 and 3.11
a DeprecationWarning, so the test records both; the language
reference still documents it as a warning in 3.15, with a SyntaxError only
in a future Python version. The tool is run per-document from `/apply`, so
the warning lands in the middle of a verification report. Compiling is a
pure `compile()` over the source text - no cache file, no temp file - so the
check costs nothing and covers every `tools/*.py`, guarding the next
docstring that quotes a macro too. It covers `tests/*.py` as well: a test
docstring that quotes a regex (`\\s*` in `test_verify_pdf.py`) warned in CI
on every matrix Python while the run stayed green.
"""
def test_sources_have_no_invalid_escape_sequences(self):
import warnings
tools_dir = Path(verify_layout.__file__).resolve().parent
tests_dir = Path(__file__).resolve().parent
sources = sorted(tools_dir.glob("*.py")) + sorted(tests_dir.glob("*.py"))
self.assertIn(Path(verify_layout.__file__).resolve(), sources)
self.assertIn(Path(__file__).resolve(), sources)
offenders: dict[str, list[str]] = {}
for source in sources:
with warnings.catch_warnings(record=True) as caught:
warnings.simplefilter("always")
compile(source.read_text(encoding="utf-8"), str(source), "exec")
syntax = [
str(w.message)
for w in caught
if issubclass(w.category, (SyntaxWarning, DeprecationWarning))
]
if syntax:
offenders[f"{source.parent.name}/{source.name}"] = syntax
self.assertEqual({}, offenders, "tools/*.py or tests/*.py warn about invalid escapes when compiled")
if __name__ == "__main__":
unittest.main()
+166
View File
@@ -6,7 +6,10 @@ from unittest.mock import patch
from tools.verify_pdf import (
VerificationError,
build_parser,
extract_text_layer,
find_non_ascii_date_ranges,
normalize_text,
parse_page_count,
run_tool,
verify_pdf,
@@ -22,6 +25,99 @@ class ParsePageCountTests(unittest.TestCase):
parse_page_count("Title: Example\n")
class NormalizeTextTests(unittest.TestCase):
"""`--contains` must see through what LaTeX does to plain source text.
Measured on the stock CV compiled with the documented `lualatex` command
(#385): the apostrophe in `Master's` reaches the text layer as U+2019 and
the `--` in `2016--2024` as U+2013, so a whitespace-only fold reports both
keywords missing from a CV that plainly contains them. Under pdflatex
without T1 font encoding, accents arrive decomposed (`e` + U+0300) instead
of precomposed (#384).
"""
def test_folds_curly_apostrophe_to_ascii(self):
self.assertEqual(normalize_text("Master\u2019s degree"), "Master's degree")
self.assertEqual(normalize_text("\u2018quoted\u2019"), "'quoted'")
def test_folds_curly_double_quotes_to_ascii(self):
self.assertEqual(normalize_text("\u201cSix Sigma\u201d"), '"Six Sigma"')
def test_folds_en_and_em_dashes_to_hyphen(self):
self.assertEqual(normalize_text("2016\u20132024"), "2016-2024")
self.assertEqual(normalize_text("role\u2014title"), "role-title")
def test_folds_no_break_space_to_space(self):
self.assertEqual(normalize_text("EUR\u00a0600k"), "EUR 600k")
def test_folds_decomposed_accents_to_nfc(self):
decomposed = "Gene\u0300ve Universite\u0301"
precomposed = "Gen\u00e8ve Universit\u00e9"
self.assertEqual(normalize_text(decomposed), precomposed)
def test_still_collapses_whitespace(self):
self.assertEqual(normalize_text("Professional\n Experience "), "Professional Experience")
def test_fold_is_symmetric(self):
# A user who pastes the curly form from a posting must match an ASCII layer too.
self.assertEqual(normalize_text("Master\u2019s"), normalize_text("Master's"))
class FindNonAsciiDateRangesTests(unittest.TestCase):
"""The date-range rule reads the raw layer; `--contains` folds and cannot.
`05-cv-templates.md` documents a Workday import that dropped a role's end
date because `2016--2024` reaches the text layer as `2016<U+2013>2024`, and
asks Step 5d to confirm every entry's years are joined by an ASCII hyphen.
The fold that makes `--contains "2016-2024"` pass on that layer (#458) is
what makes `--contains` unable to detect it - so this check never folds.
The newline cases are yang2632's finding on the PR: `\\s*` between year and
dash also matched a line break, so a heading ending in an ASCII date joined
a dash that merely opened the next line.
"""
def test_en_dash_between_years_is_reported_with_its_code_point(self):
hits = find_non_ascii_date_ranges("Six Sigma Green Belt, 2016\u20132024.\n")
self.assertEqual(hits, [("Six Sigma Green Belt, 2016\u20132024.", "\u2013")])
def test_ascii_hyphen_range_is_clean(self):
self.assertEqual(find_non_ascii_date_ranges("2016-2024\nMar 2016 - Jul 2016\n"), [])
def test_a_year_on_either_side_of_the_dash_is_enough(self):
# Month-qualified and open-ended ranges: the year is only on one side.
for text in ("Mar 2016 \u2013 Jul 2016", "2016 \u2013 Present", "\u2013 2024"):
with self.subTest(text=text):
self.assertEqual(len(find_non_ascii_date_ranges(text)), 1)
def test_other_unicode_dashes_and_the_minus_sign_are_caught(self):
for dash in ("\u2010", "\u2011", "\u2012", "\u2014", "\u2015", "\u2212"):
with self.subTest(dash=dash):
self.assertEqual(find_non_ascii_date_ranges(f"2016{dash}2024")[0][1], dash)
def test_numeric_range_without_a_year_is_not_a_date(self):
# 05-cv-templates.md keeps `--` in prose ranges like EUR 600k--1M.
self.assertEqual(find_non_ascii_date_ranges("EUR 600k\u20131M, 12\u201315 people"), [])
def test_year_ending_a_line_is_not_joined_to_the_next_lines_dash(self):
text = "Heading 1988-1994\n\u2013 note\n"
self.assertEqual(find_non_ascii_date_ranges(text), [])
def test_dash_ending_a_line_is_not_joined_to_the_next_lines_year(self):
text = "Six Sigma \u2013\n2016 onwards\n"
self.assertEqual(find_non_ascii_date_ranges(text), [])
def test_same_line_spaces_around_the_dash_are_still_caught(self):
self.assertEqual(len(find_non_ascii_date_ranges("1988 \u2013 1994")), 1)
def test_hits_are_reported_in_document_order_one_per_range(self):
text = "2016\u20132024 role\nmore text\nJan 2010 \u2013 Dec 2012 degree\n"
self.assertEqual(
[line for line, _ in find_non_ascii_date_ranges(text)],
["2016\u20132024 role", "Jan 2010 \u2013 Dec 2012 degree"],
)
class VerifyPdfTests(unittest.TestCase):
def setUp(self):
self.temp_dir = tempfile.TemporaryDirectory()
@@ -73,6 +169,76 @@ class VerifyPdfTests(unittest.TestCase):
with self.assertRaisesRegex(VerificationError, "Professional Experience"):
verify_pdf(self.pdf, required_text=("Professional Experience",))
@patch("tools.verify_pdf._extract_pypdf", return_value=None)
@patch("tools.verify_pdf.run_tool")
def test_required_text_matches_latex_typographic_substitutions(self, mock_run_tool, _pypdf):
# What the stock template's lualatex text layer actually contains for the
# source `Master's degree ... 2016--2024` (code points measured, see class
# docstring of NormalizeTextTests).
mock_run_tool.side_effect = [
"Master\u2019s degree in Statistics. Six Sigma Green Belt, 2016\u20132024.\n",
"Pages: 1\n",
]
verify_pdf(self.pdf, required_text=("Master's degree", "2016-2024"))
@patch("tools.verify_pdf._extract_pypdf", return_value=None)
@patch("tools.verify_pdf.run_tool")
def test_required_text_matches_decomposed_pdflatex_accents(self, mock_run_tool, _pypdf):
mock_run_tool.side_effect = [
"Universite\u0301 de Gene\u0300ve\n", # pdflatex without T1 fontenc
"Pages: 1\n",
]
verify_pdf(self.pdf, required_text=("Universit\u00e9 de Gen\u00e8ve",))
@patch("tools.verify_pdf._extract_pypdf", return_value=None)
@patch("tools.verify_pdf.run_tool")
def test_dump_text_keeps_the_raw_layer_unfolded(self, mock_run_tool, _pypdf):
# The fold is comparison-time only: the ATS parser sees the raw layer, and
# the date-range rule in 05-cv-templates.md needs the en-dash visible here.
mock_run_tool.side_effect = ["2016\u20132024\n", "Pages: 1\n"]
dump = Path(self.temp_dir.name) / "dump.txt"
verify_pdf(self.pdf, required_text=("2016-2024",), dump_text=dump)
self.assertEqual(dump.read_text(encoding="utf-8"), "2016\u20132024\n")
@patch("tools.verify_pdf._extract_pypdf", return_value=None)
@patch("tools.verify_pdf.run_tool")
def test_ascii_dates_rejects_the_en_dash_range_that_contains_accepts(self, mock_run_tool, _pypdf):
# The same layer, two verdicts: --contains folds U+2013 to "-" and passes;
# --ascii-dates reads the raw layer and fails, naming the code point.
layer = "Role Title 2016\u20132024\n"
mock_run_tool.side_effect = [layer, "Pages: 1\n", layer, "Pages: 1\n"]
verify_pdf(self.pdf, required_text=("2016-2024",))
with self.assertRaisesRegex(VerificationError, r"U\+2013.*2016\u20132024"):
verify_pdf(self.pdf, required_text=("2016-2024",), ascii_dates=True)
@patch("tools.verify_pdf._extract_pypdf", return_value=None)
@patch("tools.verify_pdf.run_tool")
def test_ascii_dates_accepts_hyphen_ranges(self, mock_run_tool, _pypdf):
mock_run_tool.side_effect = ["Role Title 2016-2024\nMar 2016 - Jul 2016\n", "Pages: 1\n"]
verify_pdf(self.pdf, ascii_dates=True)
@patch("tools.verify_pdf._extract_pypdf", return_value=None)
@patch("tools.verify_pdf.run_tool")
def test_ascii_dates_still_writes_the_dump_before_failing(self, mock_run_tool, _pypdf):
mock_run_tool.side_effect = ["2016\u20132024\n", "Pages: 1\n"]
dump = Path(self.temp_dir.name) / "dump.txt"
with self.assertRaises(VerificationError):
verify_pdf(self.pdf, dump_text=dump, ascii_dates=True)
self.assertEqual(dump.read_text(encoding="utf-8"), "2016\u20132024\n")
def test_ascii_dates_flag_is_off_by_default_and_parses(self):
self.assertFalse(build_parser().parse_args(["x.pdf"]).ascii_dates)
self.assertTrue(build_parser().parse_args(["x.pdf", "--ascii-dates"]).ascii_dates)
def test_rejects_missing_pdf(self):
with self.assertRaisesRegex(VerificationError, "PDF does not exist"):
verify_pdf(Path(self.temp_dir.name) / "missing.pdf")
+27 -3
View File
@@ -52,6 +52,9 @@ COMPOUND_PATTERNS = {"antal", "indeks", "løn", "gennemsnit", "medarbejdere"}
# data. They are dropped at classification so they are not mistaken for a salary
# category. Matched as whole tokens only, like other pattern sets.
ID_PATTERNS = {"id", "personnummer"}
# Category name for a count/index pair whose headers carry no category word at
# all ("Count" + "Index"). Matches the top-level category in README_SALARY_TOOL.md.
DEFAULT_CATEGORY = "all_employees"
def parse_numeric_cell(value):
@@ -216,7 +219,14 @@ def parse_sheet(ws, sheet_label=None):
else:
untyped_cols.append((col_idx, col_header))
# Pair count/index columns by matching category name
# Pair count/index columns by matching category name. A bare "Count" /
# "Index" pair (Danish "Antal" / "Lønindeks") strips to an empty name on
# both sides - the single-category layout the README's "auto-pairs
# count/index columns" line describes. It is still one pair, so it gets
# the README's default category name instead of being emitted as two
# unrelated standalone columns: salary_lookup renders that split as a
# count row whose index reads "N/A*", i.e. "too few employees to publish
# (privacy)", about a company with a published headcount.
categories = []
used_counts = set()
used_indexes = set()
@@ -225,8 +235,8 @@ def parse_sheet(ws, sheet_label=None):
for ii, (i_idx, i_header, i_cat) in enumerate(index_cols):
if ii in used_indexes:
continue
if c_cat and i_cat and c_cat == i_cat:
cat_name = c_cat.replace(" ", "_").replace("-", "_")
if c_cat == i_cat:
cat_name = (c_cat or DEFAULT_CATEGORY).replace(" ", "_").replace("-", "_")
categories.append({
"name": cat_name,
"count_col": c_idx,
@@ -317,7 +327,21 @@ def parse_sheet(ws, sheet_label=None):
return companies
def _force_utf8_output() -> None:
"""Write UTF-8 whatever the host's default encoding is.
A piped stdout on Windows defaults to the ANSI code page (cp1252 on most
Western installs), so printing a company, title or file name outside it
raised UnicodeEncodeError before the workflow saw any output.
"""
for stream in (sys.stdout, sys.stderr):
reconfigure = getattr(stream, "reconfigure", None) # absent on a StringIO under test
if reconfigure:
reconfigure(encoding="utf-8")
def main():
_force_utf8_output()
parser = argparse.ArgumentParser(
description="Convert salary Excel data to JSON"
)
+198
View File
@@ -0,0 +1,198 @@
#!/usr/bin/env python3
"""Canonical dedup key for a job posting, and an audit for existing state.
`/scrape` Step 4 keys every seen_jobs.json entry by company+title. The rule was
prose only ("<url_or_company_title_key>"), so each run slugified in its own way
and the state file accumulated two distinct failures:
* Keys carrying characters that break things downstream. `/apply` and
`/outcome` derive an archive folder name from the same company+role pair,
and documents/README.md's subfolder rule exists because a "/" splits that
path across directories. Real examples found in a live workspace:
"deloitte_junior-cybersecurity-analyst-(ot/iot)",
"neverhack-estonia_penetration-tester-/-red-teamer",
"ops-consulting,-llc_malware-analyst".
* The same job stored twice under different keys, because one run truncated
the title at a different point than the next. "deloitte_cyber-intelligence-
center-security-analy" and "deloitte_cyber-intelligence-center-security-
analyst-at" are one posting, one URL, two entries - and dedup is the whole
point of the file.
Both are fixed by making the key a pure, deterministic function of the posting.
Truncation is length-capped *and* disambiguated by a hash of the full slug, so a
long title always produces the same key and two different long titles never
collide.
A title that slugifies to nothing (a posting written in a non-Latin script) has
no usable key half at all - "securion_" was a real entry, and it would have
collided with every future non-Latin posting from that company. Those fall back
to the portal's numeric id from the URL.
Usage:
python3 tools/job_key.py --company "Acme Corp" --title "SOC Analyst (L2)"
python3 tools/job_key.py --audit job_scraper/seen_jobs.json
Exit 0 when a key is produced, or when an audit finds nothing. Exit 1 when an
audit finds violations.
"""
import argparse
import hashlib
import json
import re
import sys
import unicodedata
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
STATE = ROOT / "job_scraper" / "seen_jobs.json"
COMPANY_MAX = 40
TITLE_MAX = 60
HASH_LEN = 6
# Anything outside this set becomes a separator. Deliberately strict: "/" and
# "," are the characters that actually caused damage, and an allowlist cannot
# be surprised by the next punctuation mark a job board invents.
_NON_SLUG = re.compile(r"[^a-z0-9]+")
_JOB_ID = re.compile(r"(\d{6,})")
def slugify(text: str) -> str:
"""Lowercase ASCII slug. Non-Latin scripts legitimately reduce to ''."""
if not text:
return ""
decomposed = unicodedata.normalize("NFKD", str(text))
ascii_only = decomposed.encode("ascii", "ignore").decode("ascii")
return _NON_SLUG.sub("-", ascii_only.lower()).strip("-")
def _cap(slug: str, limit: int) -> str:
"""Cap length without making truncation lossy across runs.
A bare truncation is what produced the duplicate Deloitte entries: two runs
cut the same title at different points and the file gained a second key for
one job. Appending a hash of the *full* slug makes the result deterministic
for a given title and distinct for any other.
"""
if len(slug) <= limit:
return slug
digest = hashlib.sha1(slug.encode("utf-8")).hexdigest()[:HASH_LEN]
return f"{slug[:limit].rstrip('-')}-{digest}"
def make_key(company: str, title: str, url: str = "") -> str:
"""The canonical seen_jobs.json key for one posting."""
company_slug = _cap(slugify(company), COMPANY_MAX)
if not company_slug:
name = unicodedata.normalize("NFC", str(company or "").strip().casefold())
# An absent name stays unknown; a non-Latin name still has an identity.
digest = hashlib.sha1(name.encode("utf-8")).hexdigest()[:HASH_LEN]
company_slug = f"company-{digest}" if name else "unknown-company"
title_slug = _cap(slugify(title), TITLE_MAX)
if not title_slug:
# No Latin characters in the title. The portal's own numeric id is the
# only stable handle left; never emit a bare "company_" prefix.
match = _JOB_ID.search(url or "")
if match:
title_slug = match.group(1)
else:
basis = slugify(unicodedata.normalize("NFKD", str(title or url or "")))
digest = hashlib.sha1((str(title) + str(url)).encode("utf-8")).hexdigest()[:HASH_LEN]
title_slug = basis or f"untitled-{digest}"
return f"{company_slug}_{title_slug}"
# A canonical key is "<company-slug>_<title-slug>": lowercase alphanumerics and
# hyphens on either side of exactly one underscore. The underscore is the
# separator, so it is the one character outside the slug alphabet that belongs.
_CANONICAL = re.compile(r"^[a-z0-9][a-z0-9-]*_[a-z0-9][a-z0-9-]*$")
def is_canonical(key: str) -> bool:
"""Structurally safe as a dedup key and as an archive folder name."""
return bool(key) and bool(_CANONICAL.match(key))
def is_legacy_shape(key: str) -> bool:
"""Old three-part "company_title_location" keys.
Harmless - they carry no path-breaking character - but they are not what
make_key produces, so a later run would store the same job under a new key
and reintroduce a duplicate. Reported apart from real damage so the fix
stays a decision rather than an automatic rename.
"""
return bool(key) and key.count("_") > 1 and all(
re.fullmatch(r"[a-z0-9][a-z0-9-]*", part) for part in key.split("_") if part
)
def audit(path: Path) -> int:
try:
doc = json.loads(path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as exc:
print(f"cannot read {path}: {exc}", file=sys.stderr)
return 1
seen = doc.get("seen", doc)
if not isinstance(seen, dict):
print(f"{path}: expected an object of job entries", file=sys.stderr)
return 1
malformed = [k for k in seen if not is_canonical(k) and not is_legacy_shape(k)]
legacy = [k for k in seen if is_legacy_shape(k)]
by_url: dict[str, list[str]] = {}
for key, entry in seen.items():
url = (entry.get("url") or "").rstrip("/")
if url:
by_url.setdefault(url, []).append(key)
duplicates = {u: ks for u, ks in by_url.items() if len(ks) > 1}
# A key that does not match what make_key would produce today is drift, not
# damage: reported separately so a rename is a choice, never automatic.
drift = [
k for k, v in seen.items()
if is_canonical(k) and k != make_key(v.get("company", ""), v.get("title", ""), v.get("url", ""))
]
print(json.dumps({
"entries": len(seen),
"malformed_keys": malformed,
"legacy_three_part_keys": legacy,
"duplicate_urls": duplicates,
"keys_not_matching_current_rule": len(drift),
}, indent=2, ensure_ascii=False))
return 1 if (malformed or duplicates) else 0
def _force_utf8_output() -> None:
"""Write UTF-8 whatever the host's default encoding is.
A piped stdout on Windows defaults to the ANSI code page (cp1252 on most
Western installs), so printing a company, title or file name outside it
raised UnicodeEncodeError before the workflow saw any output.
"""
for stream in (sys.stdout, sys.stderr):
reconfigure = getattr(stream, "reconfigure", None) # absent on a StringIO under test
if reconfigure:
reconfigure(encoding="utf-8")
def main() -> int:
_force_utf8_output()
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument("--company")
ap.add_argument("--title")
ap.add_argument("--url", default="")
ap.add_argument("--audit", nargs="?", const=str(STATE), metavar="STATE_JSON")
args = ap.parse_args()
if args.audit:
return audit(Path(args.audit))
if args.company is None or args.title is None:
ap.error("give --company and --title, or --audit")
print(make_key(args.company, args.title, args.url))
return 0
if __name__ == "__main__":
sys.exit(main())
+26 -3
View File
@@ -42,6 +42,7 @@ import os
import re
import sys
import tempfile
import unicodedata
from datetime import date, timedelta
from pathlib import Path
@@ -97,7 +98,13 @@ def parse_iso(value) -> date | None:
def norm(text) -> str:
return re.sub(r"[^a-z0-9]", "", str(text or "").lower())
"""Ignore case and separators without discarding non-Latin identity."""
text = unicodedata.normalize("NFC", str(text or "").casefold())
# Combining marks can distinguish names even after NFC (e.g. Indic vowels).
return "".join(
char for char in text
if char.isalnum() or unicodedata.category(char).startswith("M")
)
def tracker_pairs(path: Path) -> set[tuple[str, str]]:
@@ -107,7 +114,7 @@ def tracker_pairs(path: Path) -> set[tuple[str, str]]:
import csv
pairs = set()
with path.open(encoding="utf-8", newline="") as fh:
with path.open(encoding="utf-8-sig", newline="") as fh:
for row in csv.DictReader(fh):
company, role = norm(row.get("company")), norm(row.get("role"))
if company:
@@ -234,8 +241,10 @@ def overall_score(scores: dict) -> int:
total = 0.0
for dim, weight in WEIGHTS.items():
value = scores.get(dim)
if not isinstance(value, (int, float)):
if isinstance(value, bool) or not isinstance(value, (int, float)):
raise ValueError(f"missing or non-numeric score '{dim}'")
if not 0 <= value <= 100:
raise ValueError(f"score '{dim}' must be finite and between 0 and 100")
total += float(value) * weight
return int(total + 0.5)
@@ -347,7 +356,21 @@ def cmd_apply(args) -> int:
return 1 if errors else 0
def _force_utf8_output() -> None:
"""Write UTF-8 whatever the host's default encoding is.
A piped stdout on Windows defaults to the ANSI code page (cp1252 on most
Western installs), so printing a company, title or file name outside it
raised UnicodeEncodeError before the workflow saw any output.
"""
for stream in (sys.stdout, sys.stderr):
reconfigure = getattr(stream, "reconfigure", None) # absent on a StringIO under test
if reconfigure:
reconfigure(encoding="utf-8")
def main() -> int:
_force_utf8_output()
common = argparse.ArgumentParser(add_help=False)
common.add_argument("--state", type=Path, default=STATE)
common.add_argument("--today", type=date.fromisoformat, default=date.today())
+4 -1
View File
@@ -37,7 +37,10 @@ def _fetch(url, ua):
r = subprocess.run(
['curl', '-sS', '-L', '--max-redirs', '5', '--max-time', '12', '-A', ua,
'-H', 'Accept: text/plain,*/*', '-w', '\n%{http_code}', '--', url],
capture_output=True, text=True, timeout=20)
# robots.txt is UTF-8 (RFC 9309). Unpinned, Turkish Windows decoded it as
# cp1254, where bytes like 0x81 are undefined: the pipe reader thread died,
# stdout came back None, and tr.indeed.com read as UNCONFIRMED.
capture_output=True, text=True, encoding='utf-8', errors='replace', timeout=20)
if r.returncode != 0:
raise RuntimeError('curl exit %d' % r.returncode)
body, _, code = r.stdout.rpartition('\n')
+5
View File
@@ -53,8 +53,12 @@ ALLOWED_PERMISSIONS = {
"Bash(python3 salary_lookup.py:*)",
"Bash(python tools/rank_state.py:*)",
"Bash(python3 tools/rank_state.py:*)",
"Bash(python tools/job_key.py:*)",
"Bash(python3 tools/job_key.py:*)",
"Bash(python tools/verify_pdf.py:*)",
"Bash(python3 tools/verify_pdf.py:*)",
"Bash(python tools/verify_layout.py:*)",
"Bash(python3 tools/verify_layout.py:*)",
"Bash(pdftotext:*)",
}
@@ -80,6 +84,7 @@ REQUIRED_IGNORE_RULES = [
"documents/linkedin/**",
"documents/diplomas/**",
"documents/references/**",
"documents/projects/**",
"documents/applications/**",
"documents/postings/**",
# Belt-and-braces, not the primary guard: nothing writes here.
+349
View File
@@ -0,0 +1,349 @@
#!/usr/bin/env python3
"""Measure a compiled CV or cover letter's page layout, instead of eyeballing it.
The compile-and-inspect loop in `05-cv-templates.md` and the verification checklist in
CLAUDE.md already require the layout properties below. Nothing executes them: they are
checked by looking at the rendered page, which is exactly how they get missed. Each
failure below produces a clean compile, a correct page count, and a PDF that passes
`tools/verify_pdf.py`:
orphaned entry A moderncv \\cventry renders as a tabular, so a job entry is one
unbreakable block. When it does not fit, the whole entry moves to
the next page - or its header lands at the bottom of one page with
the bullets resuming on the next. CLAUDE.md calls this "the most
common failure".
internal hole The space an ejected entry leaves behind. Observed in the wild at
273pt, roughly 19 blank lines, mid-page, on a document whose page
count was correct and whose visual read looked fine.
page ends early A non-final page that stops well short of the bottom.
final page thin A last page mostly empty, which reads as an unfinished document.
footer collision Body text pushed into the page-number band, the usual result of
rescuing a page with \\enlargethispage or a negative \\vspace.
Page count is deliberately NOT checked here: `tools/verify_pdf.py --pages` already does
that, and CI runs it. Two implementations of one rule drift.
Geometry comes from Poppler word bounding boxes (`pdftotext -bbox`). Poppler is optional
repo-wide - since #369 `verify_pdf.py` prefers pypdf and falls back to Poppler - but word
bounding boxes have no pypdf equivalent, so this is the one step that still wants it.
Without it, or with a broken extractor, the check reports `skipped:` and exits 2 rather
than inventing a layout failure. A broken extractor has two distinct causes: an xpdf-based
`pdftotext` (Git for Windows ships one ahead of Poppler in PATH) has no `-bbox` flag at
all, and Poppler 26.0x before 26.05 aborts `-bbox`/`-bbox-layout`/`-htmlmeta` on a PDF
whose Info dictionary carries an empty string in any field - which `hyperref` writes for
every field it does not set, so any `lualatex`/`pdflatex` document built with `hyperref`
and no `\\hypersetup{pdftitle=...}` triggers a real Poppler crashing on a legal PDF (#451).
Line height serves
as a font-size proxy to spot section headings; left edge (xMin) separates bullet lines
from entry headers.
The thresholds below are calibrated for the stock moderncv (`cv/`) and cover.cls
(`cover_letters/`) geometry. A template registered via `/add-template` may need them
retuned - an article-class page number sitting outside the 90pt footer band, for
instance, is read as body text and turns the space above it into a phantom hole.
Usage:
python tools/verify_layout.py cv/main_acme_ml_engineer.pdf
python tools/verify_layout.py cover_letters/cover_acme_ml_engineer.pdf
Exit codes: 0 clean, 1 layout problem, 2 bad invocation or no usable extractor.
The shipped `cv/main_example.pdf` exits 1 by design: its placeholder page 2 is mostly
empty, which is the thin-final-page failure the checklist asks you to fix before sending.
Tests live in tests/test_verify_layout.py and run against synthetic pages, so the
suite needs neither Poppler nor a compiled PDF.
"""
from __future__ import annotations
import argparse
import re
import shutil
import subprocess
import sys
from dataclasses import dataclass
from pathlib import Path
# A gap larger than this between consecutive lines is a hole, not spacing. Section
# spacing in the stock templates runs to roughly 45pt; 100pt is about seven lines.
GAP_LIMIT_PT = 100.0
# Bottom whitespace on a page that is not the last one. The stock geometry leaves ~85pt.
BOTTOM_LIMIT_FRACTION = 0.25
# A final page emptier than this reads as an unfinished document.
LAST_PAGE_THIN_FRACTION = 0.35
# The page-number footer lives in the bottom margin and is a text line like any other to
# Poppler. Ignore this band when measuring the body, or every page looks like it has a
# hole above its footer.
FOOTER_BAND_PT = 90.0
# A line indented at least this far past the page's left edge is a bullet or a
# continuation, not an entry header or a section heading.
INDENT_PT = 8.0
# A line this much taller than the body median is a section heading.
HEADING_HEIGHT_RATIO = 1.25
PAGE_RE = re.compile(r'<page width="([\d.]+)" height="([\d.]+)">(.*?)</page>', re.S)
WORD_RE = re.compile(
r'<word xMin="([\d.]+)" yMin="([\d.]+)" xMax="[\d.]+" yMax="([\d.]+)">([^<]*)</word>'
)
@dataclass
class Line:
top: float
bottom: float
left: float
height: float
text: str
class Page:
def __init__(self, height: float, lines: list[Line]):
self.height = height
self.lines = sorted(lines, key=lambda l: l.top)
@property
def body(self) -> list[Line]:
cutoff = self.height - FOOTER_BAND_PT
return [l for l in self.lines if l.top < cutoff]
@property
def empty(self) -> bool:
return not self.body
@property
def bottom_space(self) -> float:
return self.height - max(l.bottom for l in self.body) if self.body else self.height
@property
def left_edge(self) -> float:
return min(l.left for l in self.body) if self.body else 0.0
@property
def body_median_height(self) -> float:
heights = sorted(l.height for l in self.body)
return heights[len(heights) // 2] if heights else 0.0
@property
def footer_crowded(self) -> bool:
"""One line in the bottom band is a page number; two means body text spilled in."""
band = self.height - FOOTER_BAND_PT
return len({round(l.top, 1) for l in self.lines if l.top >= band}) > 1
def is_indented(self, line: Line) -> bool:
return line.left > self.left_edge + INDENT_PT
def is_heading(self, line: Line) -> bool:
median = self.body_median_height
return bool(median) and line.height > median * HEADING_HEIGHT_RATIO
def largest_gap(self) -> tuple[float, float]:
"""Largest top-to-top distance between body lines, and where it starts.
Measured top-to-top rather than bottom-to-top, so a tall line inflates the gap
by its own height (a 160pt void under a heading reads as ~174pt). That errs
toward over-detection, which is the right direction for a check whose job is
to stop a hole from shipping.
"""
tops = sorted({round(l.top, 1) for l in self.body})
if len(tops) < 2:
return (0.0, 0.0)
return max((tops[i + 1] - tops[i], tops[i]) for i in range(len(tops) - 1))
def parse_pdf(path: Path) -> list[Page]:
if not shutil.which("pdftotext"):
raise RuntimeError("pdftotext (Poppler) not found; install poppler-utils")
try:
out = subprocess.run(
["pdftotext", "-bbox", "-enc", "UTF-8", str(path), "-"],
capture_output=True,
text=True,
# pdftotext emits UTF-8; without this Windows decodes it as cp1252 and
# a non-ASCII glyph in the CV crashes the run (same fix as verify_pdf.py).
encoding="utf-8",
errors="replace",
check=True,
).stdout
except subprocess.CalledProcessError as exc:
# A broken extractor has two distinct causes, not one: an xpdf-based pdftotext
# has no -bbox at all and exits 99 (Git for Windows puts one ahead of Poppler
# in PATH); a real Poppler before 26.05 aborts -bbox on a PDF whose Info dict
# has an empty string in any field - which hyperref writes for every field it
# does not set, so a lualatex/pdflatex document with hyperref and no pdftitle
# triggers this even with a working Poppler (#451). Either way this is a
# broken extractor, not a broken document, so it degrades to the skip path
# instead of exit 1.
stderr_lines = (exc.stderr or "").strip().splitlines()
detail = stderr_lines[0] if stderr_lines else f"exit {exc.returncode}"
raise RuntimeError(
f"pdftotext could not produce bounding boxes for {path} ({detail}); "
"either the pdftotext first in PATH is an xpdf build with no -bbox flag "
"(Git for Windows ships one ahead of Poppler), or Poppler aborted on this "
"document - Poppler 26.0x before 26.05 aborts on a PDF whose Info "
"dictionary carries an empty string, as hyperref writes when pdftitle is unset"
) from exc
pages = []
for _w, h, body in PAGE_RE.findall(out):
buckets: dict[float, list[tuple[float, float, float, str]]] = {}
for x_min, y_min, y_max, text in WORD_RE.findall(body):
key = round(float(y_min), 0) # words on one line share a rounded yMin
buckets.setdefault(key, []).append((float(x_min), float(y_min), float(y_max), text))
lines = [
Line(
top=min(w[1] for w in words),
bottom=max(w[2] for w in words),
left=min(w[0] for w in words),
height=max(w[2] - w[1] for w in words),
text=" ".join(w[3] for w in sorted(words)),
)
for words in buckets.values()
]
pages.append(Page(float(h), lines))
return pages
def find_orphans(pages: list[Page]) -> list[str]:
"""A page ending on an entry header or section heading whose content resumes overleaf.
Two shapes, both documented failures:
* the last body line of a page is a section heading (stranded heading)
* the last body lines are un-indented (an entry header) while the next page opens
with indented bullet lines, i.e. the entry was split across the break
"""
problems = []
# Indentation must be judged against the document's left margin, not each page's own
# minimum: a page that *opens* with indented bullets would otherwise treat their
# indent as its margin and report nothing.
body_lines = [l for p in pages for l in p.body]
if not body_lines:
return problems
doc_left = min(l.left for l in body_lines)
def indented(line: Line) -> bool:
return line.left > doc_left + INDENT_PT
for i in range(len(pages) - 1):
here, nxt = pages[i], pages[i + 1]
if here.empty or nxt.empty:
continue
last, first = here.body[-1], nxt.body[0]
if here.is_heading(last):
problems.append(
f"p{i + 1} ends on the section heading {last.text.strip()!r} with its content "
f"on p{i + 2}. Shorten the entry that follows it, or let the heading and its "
"first entry move to the next page together"
)
elif not indented(last) and indented(first):
# moderncv puts an itemize marker in its own bbox line at the list's left
# edge, so a list item split across the break looks like an un-indented
# header followed by indented text. Different defect, different fix.
if not re.search(r"\w", last.text):
problems.append(
f"p{i + 1} ends on a lone list marker whose text continues on p{i + 2} "
f"({first.text.strip()[:60]!r}): a bullet is split across the page break. "
"Shorten the preceding content so the whole item fits on one page"
)
else:
problems.append(
f"p{i + 1} ends on the un-indented line {last.text.strip()[:60]!r} while "
f"p{i + 2} opens with the indented line {first.text.strip()[:60]!r}: an entry "
"header is orphaned from its bullets. Add \\needspace before that "
"\\cventry, or shorten it"
)
return problems
def report(path: Path, pages: list[Page]) -> list[str]:
problems: list[str] = []
print(f"{path}: {len(pages)} page(s) (page count is verify_pdf.py's job, not checked here)")
for i, page in enumerate(pages, 1):
if page.empty:
problems.append(f"p{i} contains no text")
print(f" p{i}: EMPTY")
continue
gap, gap_y = page.largest_gap()
share = page.bottom_space / page.height
print(
f" p{i}: text y {page.body[0].top:.0f}..{page.body[-1].bottom:.0f}"
f" of {page.height:.0f}pt | bottom {page.bottom_space:.0f}pt ({share * 100:.0f}%)"
f" | largest gap {gap:.0f}pt at y{gap_y:.0f}"
)
if gap > GAP_LIMIT_PT:
problems.append(
f"p{i} has a {gap:.0f}pt hole at y{gap_y:.0f} (~{gap / 14:.0f} blank lines). "
"A moderncv \\cventry is an unbreakable tabular: shorten the entry that "
"follows the hole so it fits, or move a shorter section above it"
)
if i < len(pages) and share > BOTTOM_LIMIT_FRACTION:
problems.append(
f"p{i} ends {page.bottom_space:.0f}pt ({share * 100:.0f}%) early although "
"more pages follow, which reads as a broken page break"
)
if page.footer_crowded:
problems.append(
f"p{i} has body text inside the bottom margin band, colliding with the "
"footer; stop stretching the page with \\enlargethispage and cut content"
)
if i == len(pages) > 1 and share > LAST_PAGE_THIN_FRACTION:
problems.append(
f"p{i} is the last page and {share * 100:.0f}% empty, which reads as an "
"unfinished document; restore the highest-relevance content previously cut"
)
problems.extend(find_orphans(pages))
return problems
def _force_utf8_output() -> None:
"""Write UTF-8 whatever the host's default encoding is.
A piped stdout on Windows defaults to the ANSI code page (cp1252 on most
Western installs), so printing a company, title or file name outside it
raised UnicodeEncodeError before the workflow saw any output.
"""
for stream in (sys.stdout, sys.stderr):
reconfigure = getattr(stream, "reconfigure", None) # absent on a StringIO under test
if reconfigure:
reconfigure(encoding="utf-8")
def main() -> int:
_force_utf8_output()
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("pdf", nargs="?", type=Path)
args = ap.parse_args()
if not args.pdf:
ap.error("pdf is required")
if not args.pdf.exists():
print(f"error: {args.pdf} not found", file=sys.stderr)
return 2
try:
pages = parse_pdf(args.pdf)
except RuntimeError as exc:
print(f"skipped: {exc}", file=sys.stderr)
return 2
problems = report(args.pdf, pages)
if problems:
print("\nLAYOUT PROBLEMS:")
for m in problems:
print(f" - {m}")
return 1
print("layout: clean")
return 0
if __name__ == "__main__":
sys.exit(main())
+120 -2
View File
@@ -4,12 +4,23 @@
Text-layer extraction tries pypdf (BSD, optional `pip install pypdf`) first,
then Poppler `pdftotext` if pypdf is missing, raises, or returns zero
extractable characters. Poppler remains the fallback.
`--contains` compares after `normalize_text()` has folded both sides: whitespace,
Unicode normalization form (NFC), and the typographic substitutions LaTeX makes to
the source text. The fold is comparison-time only - the `--dump-text` output stays
the raw text layer an ATS parser actually sees.
`--ascii-dates` is the one check that deliberately does NOT fold: it scans the raw
layer for a year joined to a Unicode dash - the en-dash LaTeX makes from `--`, which
a Workday import dropped along with the date (`05-cv-templates.md`, "Date fields
must be ASCII ranges") - and fails naming each hit with its code point.
"""
import argparse
import re
import subprocess
import sys
import unicodedata
from pathlib import Path
@@ -47,10 +58,71 @@ def parse_page_count(pdfinfo_output):
return int(match.group(1))
# Typographic substitutions the moderncv/cover.cls templates produce from plain
# source text, mapped back to what a user types into --contains. LaTeX ligatures
# ' into U+2019 and -- into U+2013, so "Master's degree" and "2016-2024" are
# absent from the text layer of a CV that plainly contains them (#385). Applied
# to both sides of the comparison; the extracted dump is never rewritten.
TYPOGRAPHIC_FOLDS = str.maketrans(
{
"\u2018": "'", # ` -> quoteleft
"\u2019": "'", # ' -> quoteright (the possessive apostrophe)
"\u201c": '"', # `` -> quotedblleft
"\u201d": '"', # '' -> quotedblright
"\u2013": "-", # -- -> endash (the \cventry date-range case)
"\u2014": "-", # --- -> emdash
"\u00a0": " ", # ~ -> no-break space
}
)
def normalize_text(text):
"""Fold a string for comparison: NFC, typographic punctuation, whitespace.
NFC covers the pdflatex text layer, which without T1 font encoding stores
accented letters decomposed (`e` + U+0300) while a user types them
precomposed (U+00E8); both forms fold to the same string (#384). The fold
applies to what is compared, never to what is dumped: the date-range rule in
`05-cv-templates.md` still needs the raw en-dash visible in `--dump-text`.
"""
text = unicodedata.normalize("NFC", text).translate(TYPOGRAPHIC_FOLDS)
return " ".join(text.split())
# Dash-like code points a year must never be joined to in a date range. U+2013 is
# what LaTeX makes from `--` and what the documented Workday import dropped; the
# rest are the other Unicode dashes and the minus sign, which a parser that splits
# a range only on U+002D treats the same way.
NON_ASCII_DASHES = "\u2010\u2011\u2012\u2013\u2014\u2015\u2212"
_YEAR = r"(?:19|20)\d{2}"
# Horizontal whitespace only between the year and the dash: `\s*` also matched
# a newline, so a year ending one line joined a dash opening the next (a bullet,
# a wrapped prose line) and an ASCII date was reported as U+2013.
NON_ASCII_DATE_RANGE = re.compile(
rf"{_YEAR}[^\S\n]*[{NON_ASCII_DASHES}]|[{NON_ASCII_DASHES}][^\S\n]*{_YEAR}"
)
def find_non_ascii_date_ranges(text):
"""Return (line, dash) for every year joined to a non-ASCII dash in raw text.
Works on the raw text layer, never on `normalize_text()` output: the fold
maps U+2013 back to `-` so `--contains "2016-2024"` can match what the
template renders, which is exactly why `--contains` cannot see this defect.
A year on either side of the dash is enough ("Mar 2016 - Jul 2016",
"2016 - Present"), so the check is locale-agnostic; a numeric range with
no year ("EUR 600k-1M") is not a date and is left alone.
"""
hits = []
for match in NON_ASCII_DATE_RANGE.finditer(text):
dash = next(c for c in match.group(0) if c in NON_ASCII_DASHES)
line_start = text.rfind("\n", 0, match.start()) + 1
line_end = text.find("\n", match.end())
line = text[line_start : line_end if line_end != -1 else len(text)]
hits.append((" ".join(line.split()), dash))
return hits
def _extract_pypdf(pdf_path):
"""Return (text, pages) or None if pypdf is unavailable, raises, or yields no text."""
try:
@@ -87,7 +159,14 @@ def extract_text_layer(pdf_path):
return text, pages, "pdftotext"
def verify_pdf(pdf_path, expected_pages=None, min_chars=1, required_text=(), dump_text=None):
def verify_pdf(
pdf_path,
expected_pages=None,
min_chars=1,
required_text=(),
dump_text=None,
ascii_dates=False,
):
pdf_path = Path(pdf_path)
if not pdf_path.is_file():
raise VerificationError(f"PDF does not exist: {pdf_path}")
@@ -125,6 +204,17 @@ def verify_pdf(pdf_path, expected_pages=None, min_chars=1, required_text=(), dum
raise VerificationError(
f"text layer is missing required text: {required!r} (extractor: {extractor})"
)
if ascii_dates:
hits = find_non_ascii_date_ranges(extracted_text)
if hits:
listed = "; ".join(f"U+{ord(dash):04X} in {line[:80]!r}" for line, dash in hits)
raise VerificationError(
f"{len(hits)} date range(s) joined by a non-ASCII dash - an ATS that splits "
f"ranges on U+002D drops the date; write the date argument with a single "
f"ASCII hyphen (05-cv-templates.md, 'Date fields must be ASCII ranges'): "
f"{listed} (extractor: {extractor})"
)
return extractor, extracted_text, actual_pages
@@ -144,17 +234,44 @@ def build_parser():
"--contains",
action="append",
default=[],
help="text that must appear after whitespace normalization; repeatable",
help=(
"text that must appear in the text layer; both sides are folded for "
"whitespace, NFC, and LaTeX's typographic substitutions (curly "
"apostrophes/quotes, en/em dashes, no-break spaces); repeatable"
),
)
parser.add_argument(
"--dump-text",
type=Path,
help="write the extracted text layer to this path (UTF-8)",
)
parser.add_argument(
"--ascii-dates",
action="store_true",
help=(
"fail if the raw text layer has a year joined to a Unicode dash (the "
"en-dash LaTeX makes from --), which ATS date parsers drop; this check "
"never folds, unlike --contains"
),
)
return parser
def _force_utf8_output() -> None:
"""Write UTF-8 whatever the host's default encoding is.
A piped stdout on Windows defaults to the ANSI code page (cp1252 on most
Western installs), so printing a company, title or file name outside it
raised UnicodeEncodeError before the workflow saw any output.
"""
for stream in (sys.stdout, sys.stderr):
reconfigure = getattr(stream, "reconfigure", None) # absent on a StringIO under test
if reconfigure:
reconfigure(encoding="utf-8")
def main(argv=None):
_force_utf8_output()
args = build_parser().parse_args(argv)
try:
extractor, text, pages = verify_pdf(
@@ -163,6 +280,7 @@ def main(argv=None):
args.min_chars,
args.contains,
dump_text=args.dump_text,
ascii_dates=args.ascii_dates,
)
except VerificationError as exc:
print(f"Error: {args.pdf}: {exc}", file=sys.stderr)