mirror of
https://github.com/MadsLorentzen/ai-job-search.git
synced 2026-09-17 08:36:25 +00:00
feat(layout): measure compiled PDF layout instead of eyeballing it (#378)
* feat(layout): measure compiled PDF layout instead of eyeballing it /apply Step 5b asks for layout properties and executes none of them: they are checked by reading the rendered page, which is exactly how they get missed. The failure that motivates this is silent under every existing check. A moderncv \cventry renders as a tabular, so an entry is one unbreakable block; when it does not fit in the space left, the whole entry moves to the next page and leaves a hole behind. The document still compiles, still reports the expected page count, and still passes tools/verify_pdf.py. Observed in the wild at 273pt, roughly 19 blank lines, mid-page, on a CV whose visual read looked fine. tools/verify_layout.py reports per page where the text starts and stops, bottom whitespace as a share of page height, and the largest gap between lines, then exits 1 on a hole over 100pt, a non-final page ending more than 25% early, body text colliding with the page-number footer, a final page more than 35% empty, or an entry header or section heading stranded at a page break. Page count is deliberately not checked here - verify_pdf.py --pages already does that, and two implementations of one rule drift. Geometry comes from Poppler pdftotext -bbox, already a dependency; a missing Poppler exits 2 with "skipped:" rather than failing the run. Tests build synthetic Page/Line geometry, so the suite needs neither Poppler nor a LaTeX toolchain and runs on the existing 3.10-3.14 matrix. * fix(layout): survive Windows encoding and a pdftotext without -bbox Review found two failures on the repo's primary platform: subprocess.run(..., text=True) decoded pdftotext's UTF-8 output with the Windows ANSI codepage and crashed on the stock cv/main_example.pdf. It now passes encoding="utf-8" with errors="replace" at the call site, matching the fix verify_pdf.py already carries from #369. Git for Windows ships an xpdf-based pdftotext that shadows Poppler in a default PATH and has no -bbox flag; it exited 99, the CalledProcessError escaped, and the run ended in exit 1 - indistinguishable from a real layout problem, which would send /apply chasing a phantom hole. That now routes to the existing "skipped:" exit 2 path with a message naming the likely cause, covered by three tests on the extractor-failure path. Also from review: - .claude/settings.json and security_guards.py gain the verify_layout.py permission entries. apply.md Step 5b runs the tool on every /apply, so without them every run prompts. - The docstring and CHANGELOG no longer call Poppler a dependency verify_pdf.py relies on. Since #369 verify_pdf prefers pypdf and Poppler is the fallback; word bboxes have no pypdf equivalent, so this is the one step that still wants it, and that is now what the text says. - Docstring and apply.md state that the thresholds are calibrated for the stock moderncv and cover.cls geometry, and that the shipped example CV fails the thin-final-page rule by design. - largest_gap documents that it measures top-to-top, so a tall line inflates the gap by its own height - over-detection, the safe direction. - The test module imports via tools.verify_layout like test_verify_pdf.py instead of sys.path.insert. * fix(verify_layout): quote only the first stderr line in the skip message (#378) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013fqqLgQSnwgWkv98twQhHi --------- Co-authored-by: Mads Lorentzen <madslorentzen_17@hotmail.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Yan Cheng (程彦)
co-authored by
Claude Fable 5.1
Mads Lorentzen
Claude Fable 5.1
Mads Lorentzen
parent
8c81edc330
commit
cbd8a991ab
@@ -229,7 +229,22 @@ If either compile fails, fix the error and re-compile until clean.
|
||||
|
||||
### 5b. Inspect layout
|
||||
|
||||
Read both PDFs via the Read tool and verify:
|
||||
**Measure first, then look.** A visual read catches gross breakage but cannot tell you that a page is 40% empty, and the failure below survives both a clean compile and a correct page count:
|
||||
|
||||
```bash
|
||||
python tools/verify_layout.py cv/main_<company>_<role>.pdf
|
||||
python tools/verify_layout.py cover_letters/cover_<company>_<role>.pdf
|
||||
```
|
||||
|
||||
The script reports, per page, where the text starts and stops, bottom whitespace as a share of page height, and the largest vertical gap between lines. It exits 1 on: a hole over 100pt (~7 blank lines), a non-final page ending more than 25% early, body text colliding with the page-number footer, a final page more than 35% empty, and an entry header or section heading stranded at a page break. Page count is **not** checked here — that is `verify_pdf.py --pages`'s job, and Step 5d already runs it.
|
||||
|
||||
The hole check is the one a visual read misses. A moderncv `\cventry` renders as a `tabular`, so it is an **unbreakable block**: when it does not fit in the space left, the whole entry jumps to the next page and leaves a hole behind, while the document still compiles and still reports the right page count. Fix it by shortening the entry that follows the hole, not by stretching the page.
|
||||
|
||||
If Poppler is missing, or the `pdftotext` first in PATH is the xpdf build Git for Windows ships (no `-bbox`), the script exits 2 with `skipped:` — note the degraded mode in the Step 6 report and rely on the visual inspection alone. Exit 2 is never a layout verdict.
|
||||
|
||||
The thresholds are calibrated for the stock moderncv and `cover.cls` geometry; a template registered via `/add-template` may report a phantom hole above a footer the 90pt band does not cover.
|
||||
|
||||
Then read both PDFs via the Read tool and verify:
|
||||
|
||||
**CV (`cv/main_<company>_<role>.pdf`):**
|
||||
- [ ] Exactly 2 pages (not 1, not 3)
|
||||
|
||||
@@ -16,6 +16,8 @@
|
||||
"Bash(python3 tools/job_key.py:*)",
|
||||
"Bash(python tools/verify_pdf.py:*)",
|
||||
"Bash(python3 tools/verify_pdf.py:*)",
|
||||
"Bash(python tools/verify_layout.py:*)",
|
||||
"Bash(python3 tools/verify_layout.py:*)",
|
||||
"Bash(pdftotext:*)"
|
||||
]
|
||||
}
|
||||
|
||||
@@ -28,6 +28,23 @@ per-file diff commands.
|
||||
resolves confirmed rows to `no_response`, logs dated entries to `notes`, updates archive
|
||||
`outcome.md` files, and hands off to calibration when 3+ applications are resolved.
|
||||
|
||||
- **Mechanical layout verification for compiled PDFs** - `tools/verify_layout.py` measures
|
||||
what `/apply` Step 5b previously only eyeballed: per-page text extent, bottom whitespace,
|
||||
the largest internal vertical gap, footer collisions, and entry headers or section
|
||||
headings stranded at a page break. It exists for a failure that survives every existing
|
||||
check - a moderncv `\cventry` is an unbreakable `tabular`, so an entry that does not fit
|
||||
jumps to the next page and leaves a hole behind (observed at 273pt, roughly 19 blank
|
||||
lines) while the document still compiles, still reports the correct page count, and still
|
||||
passes `tools/verify_pdf.py`. Geometry comes from Poppler `pdftotext -bbox`; Poppler is
|
||||
optional repo-wide (since #369 `verify_pdf.py` prefers pypdf), and word bounding boxes
|
||||
have no pypdf equivalent, so this is the one step that still wants it. A missing Poppler
|
||||
- or the xpdf-based `pdftotext` Git for Windows puts ahead of it in PATH, which rejects
|
||||
`-bbox` - degrades to a `skipped:` exit 2 rather than reporting a phantom layout failure.
|
||||
Page count is deliberately left to `verify_pdf.py --pages` so that one rule keeps one
|
||||
implementation. Thresholds are calibrated for the stock moderncv and `cover.cls`
|
||||
geometry. Tests use synthetic page geometry, so they need neither Poppler nor a
|
||||
LaTeX toolchain.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **`jobnet-search detail` no longer reports an externally hosted ad as not found** (#432) -
|
||||
|
||||
@@ -225,6 +225,7 @@ ai-job-search/
|
||||
│ ├── robots_check.py # Gate the browser-header retry against robots.txt
|
||||
│ ├── security_guards.py # CI guards: permission allowlist, gitignore rules, manifests
|
||||
│ ├── upstream_triage.py # Sort upstream commits into worth-reviewing vs probably-skip
|
||||
│ ├── verify_layout.py # Measure a compiled PDF's page layout (holes, orphans, footer collisions)
|
||||
│ ├── verify_pdf.py # Verify a compiled PDF's page count and extractable text
|
||||
│ └── README_SALARY_TOOL.md # Salary tool setup instructions
|
||||
├── job_scraper/ # Scraper state (seen jobs, results)
|
||||
|
||||
@@ -0,0 +1,146 @@
|
||||
"""Offline tests for tools/verify_layout.py.
|
||||
|
||||
Every case is built from synthetic Page/Line geometry rather than a compiled
|
||||
PDF, so the suite needs neither Poppler nor a LaTeX toolchain - matching the
|
||||
repo's CI policy of keeping the Python tool tests self-contained.
|
||||
|
||||
The cases marked SILENT FAILURE are the ones that motivated the tool: each
|
||||
describes a document that compiles cleanly, reports the expected page count,
|
||||
and passes tools/verify_pdf.py, while the rendered page is visibly broken.
|
||||
"""
|
||||
|
||||
import io
|
||||
import subprocess
|
||||
import unittest
|
||||
from contextlib import redirect_stdout
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
from tools.verify_layout import Line, Page, find_orphans, main, parse_pdf, report
|
||||
|
||||
A4_HEIGHT = 842.0
|
||||
|
||||
|
||||
def line(top: float, left: float = 50.0, height: float = 10.0, text: str = "x") -> Line:
|
||||
return Line(top=top, bottom=top + height, left=left, height=height, text=text)
|
||||
|
||||
|
||||
class TestGapAndBottomSpace(unittest.TestCase):
|
||||
def setUp(self):
|
||||
# Three body lines, then a 322pt jump, then the page-number footer.
|
||||
self.holed = Page(
|
||||
A4_HEIGHT,
|
||||
[line(50), line(64), line(78), line(400), line(770, text="1/2")],
|
||||
)
|
||||
|
||||
def test_largest_gap_reports_size_and_position(self):
|
||||
"""SILENT FAILURE: the hole an ejected \\cventry leaves behind."""
|
||||
gap, y = self.holed.largest_gap()
|
||||
self.assertEqual((round(gap), round(y)), (322, 78))
|
||||
|
||||
def test_bottom_space_ignores_the_footer_band(self):
|
||||
"""Measured to the last body line (y410), not to the page number at y770."""
|
||||
self.assertEqual(round(self.holed.bottom_space), 432)
|
||||
|
||||
def test_page_with_no_body_lines_is_empty(self):
|
||||
self.assertTrue(Page(A4_HEIGHT, []).empty)
|
||||
|
||||
def test_report_flags_the_hole(self):
|
||||
with redirect_stdout(io.StringIO()): # report() prints its per-page measurements
|
||||
problems = report(Path("synthetic"), [self.holed])
|
||||
self.assertTrue(any("hole" in m for m in problems), problems)
|
||||
|
||||
|
||||
class TestFooterBand(unittest.TestCase):
|
||||
def test_single_line_in_band_is_just_the_page_number(self):
|
||||
self.assertFalse(Page(A4_HEIGHT, [line(50), line(800, text="2/2")]).footer_crowded)
|
||||
|
||||
def test_two_lines_in_band_means_body_text_spilled_in(self):
|
||||
"""SILENT FAILURE: \\enlargethispage pushing body text over the footer."""
|
||||
self.assertTrue(Page(A4_HEIGHT, [line(50), line(780), line(800)]).footer_crowded)
|
||||
|
||||
|
||||
class TestHeadingAndIndentDetection(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.page = Page(
|
||||
A4_HEIGHT,
|
||||
[
|
||||
line(50, height=16.0, text="Professional Experience"),
|
||||
line(80, left=50.0),
|
||||
line(94, left=70.0),
|
||||
],
|
||||
)
|
||||
|
||||
def test_taller_line_is_a_heading(self):
|
||||
self.assertTrue(self.page.is_heading(self.page.body[0]))
|
||||
self.assertFalse(self.page.is_heading(self.page.body[1]))
|
||||
|
||||
def test_left_edge_separates_bullets_from_headers(self):
|
||||
self.assertTrue(self.page.is_indented(self.page.body[2]))
|
||||
self.assertFalse(self.page.is_indented(self.page.body[1]))
|
||||
|
||||
|
||||
class TestOrphans(unittest.TestCase):
|
||||
def test_page_ending_on_a_section_heading(self):
|
||||
"""SILENT FAILURE: a heading stranded at the bottom, content overleaf."""
|
||||
p1 = Page(A4_HEIGHT, [line(50), line(64), line(700, height=16.0, text="Education")])
|
||||
p2 = Page(A4_HEIGHT, [line(60, text="Example University"), line(74, left=70.0)])
|
||||
self.assertTrue(any("ends on the section heading" in m for m in find_orphans([p1, p2])))
|
||||
|
||||
def test_entry_header_orphaned_from_its_bullets(self):
|
||||
"""SILENT FAILURE: the \\cventry title on one page, its bullets on the next."""
|
||||
q1 = Page(A4_HEIGHT, [line(50), line(700, left=50.0, text="Software Engineer")])
|
||||
q2 = Page(A4_HEIGHT, [line(60, left=70.0, text="- built the thing")])
|
||||
self.assertTrue(any("orphaned from its bullets" in m for m in find_orphans([q1, q2])))
|
||||
|
||||
def test_lone_list_marker_is_a_split_bullet_not_an_orphaned_header(self):
|
||||
"""moderncv gives the itemize marker its own bbox line: different defect, different fix."""
|
||||
m1 = Page(A4_HEIGHT, [line(50), line(700, left=50.0, text="●")])
|
||||
m2 = Page(A4_HEIGHT, [line(60, left=70.0, text="continued item text here")])
|
||||
self.assertTrue(any("lone list marker" in m for m in find_orphans([m1, m2])))
|
||||
|
||||
def test_clean_break_reports_nothing(self):
|
||||
r1 = Page(A4_HEIGHT, [line(50), line(700, left=50.0)])
|
||||
r2 = Page(A4_HEIGHT, [line(60, left=50.0)])
|
||||
self.assertEqual(find_orphans([r1, r2]), [])
|
||||
|
||||
def test_indent_is_judged_against_the_document_margin(self):
|
||||
"""A page that OPENS with bullets must not mistake their indent for its margin."""
|
||||
s1 = Page(A4_HEIGHT, [line(50, left=50.0), line(700, left=50.0, text="Data Analyst")])
|
||||
s2 = Page(A4_HEIGHT, [line(60, left=70.0, text="- first bullet")])
|
||||
self.assertTrue(any("orphaned from its bullets" in m for m in find_orphans([s1, s2])))
|
||||
|
||||
class TestExtractorFailure(unittest.TestCase):
|
||||
"""A broken extractor must not masquerade as a broken document.
|
||||
|
||||
Git for Windows ships an xpdf-based pdftotext with no -bbox flag; it shadows
|
||||
Poppler in a default PATH and exits 99. Reported as a layout problem it would
|
||||
send /apply chasing a phantom hole, so it has to land on the skip path.
|
||||
"""
|
||||
|
||||
def test_pdftotext_without_bbox_raises_a_skippable_error(self):
|
||||
failure = subprocess.CalledProcessError(99, "pdftotext", stderr="Error: unknown flag")
|
||||
with patch("tools.verify_layout.shutil.which", return_value="/usr/bin/pdftotext"), patch(
|
||||
"tools.verify_layout.subprocess.run", side_effect=failure
|
||||
):
|
||||
with self.assertRaisesRegex(RuntimeError, "bounding boxes"):
|
||||
parse_pdf(Path("cv/main_example.pdf"))
|
||||
|
||||
def test_missing_poppler_raises_a_skippable_error(self):
|
||||
with patch("tools.verify_layout.shutil.which", return_value=None):
|
||||
with self.assertRaisesRegex(RuntimeError, "not found"):
|
||||
parse_pdf(Path("cv/main_example.pdf"))
|
||||
|
||||
def test_extractor_failure_exits_2_not_1(self):
|
||||
"""Exit 1 means "your document is broken"; a dead extractor must never claim that."""
|
||||
with patch("tools.verify_layout.parse_pdf", side_effect=RuntimeError("no -bbox")), patch(
|
||||
"sys.argv", ["verify_layout.py", __file__]
|
||||
):
|
||||
err = io.StringIO()
|
||||
with redirect_stdout(io.StringIO()), patch("sys.stderr", err):
|
||||
self.assertEqual(main(), 2)
|
||||
self.assertIn("skipped:", err.getvalue())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -57,6 +57,8 @@ ALLOWED_PERMISSIONS = {
|
||||
"Bash(python3 tools/job_key.py:*)",
|
||||
"Bash(python tools/verify_pdf.py:*)",
|
||||
"Bash(python3 tools/verify_pdf.py:*)",
|
||||
"Bash(python tools/verify_layout.py:*)",
|
||||
"Bash(python3 tools/verify_layout.py:*)",
|
||||
"Bash(pdftotext:*)",
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,322 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Measure a compiled CV or cover letter's page layout, instead of eyeballing it.
|
||||
|
||||
The compile-and-inspect loop in `05-cv-templates.md` and the verification checklist in
|
||||
CLAUDE.md already require the layout properties below. Nothing executes them: they are
|
||||
checked by looking at the rendered page, which is exactly how they get missed. Each
|
||||
failure below produces a clean compile, a correct page count, and a PDF that passes
|
||||
`tools/verify_pdf.py`:
|
||||
|
||||
orphaned entry A moderncv \\cventry renders as a tabular, so a job entry is one
|
||||
unbreakable block. When it does not fit, the whole entry moves to
|
||||
the next page - or its header lands at the bottom of one page with
|
||||
the bullets resuming on the next. CLAUDE.md calls this "the most
|
||||
common failure".
|
||||
internal hole The space an ejected entry leaves behind. Observed in the wild at
|
||||
273pt, roughly 19 blank lines, mid-page, on a document whose page
|
||||
count was correct and whose visual read looked fine.
|
||||
page ends early A non-final page that stops well short of the bottom.
|
||||
final page thin A last page mostly empty, which reads as an unfinished document.
|
||||
footer collision Body text pushed into the page-number band, the usual result of
|
||||
rescuing a page with \\enlargethispage or a negative \\vspace.
|
||||
|
||||
Page count is deliberately NOT checked here: `tools/verify_pdf.py --pages` already does
|
||||
that, and CI runs it. Two implementations of one rule drift.
|
||||
|
||||
Geometry comes from Poppler word bounding boxes (`pdftotext -bbox`). Poppler is optional
|
||||
repo-wide - since #369 `verify_pdf.py` prefers pypdf and falls back to Poppler - but word
|
||||
bounding boxes have no pypdf equivalent, so this is the one step that still wants it.
|
||||
Without it, or with an extractor that cannot do `-bbox` (Git-for-Windows ships an
|
||||
xpdf-based `pdftotext` that shadows Poppler in PATH and rejects the flag), the check
|
||||
reports `skipped:` and exits 2 rather than inventing a layout failure. Line height serves
|
||||
as a font-size proxy to spot section headings; left edge (xMin) separates bullet lines
|
||||
from entry headers.
|
||||
|
||||
The thresholds below are calibrated for the stock moderncv (`cv/`) and cover.cls
|
||||
(`cover_letters/`) geometry. A template registered via `/add-template` may need them
|
||||
retuned - an article-class page number sitting outside the 90pt footer band, for
|
||||
instance, is read as body text and turns the space above it into a phantom hole.
|
||||
|
||||
Usage:
|
||||
python tools/verify_layout.py cv/main_acme_ml_engineer.pdf
|
||||
python tools/verify_layout.py cover_letters/cover_acme_ml_engineer.pdf
|
||||
|
||||
Exit codes: 0 clean, 1 layout problem, 2 bad invocation or no usable extractor.
|
||||
|
||||
The shipped `cv/main_example.pdf` exits 1 by design: its placeholder page 2 is mostly
|
||||
empty, which is the thin-final-page failure the checklist asks you to fix before sending.
|
||||
|
||||
Tests live in tests/test_verify_layout.py and run against synthetic pages, so the
|
||||
suite needs neither Poppler nor a compiled PDF.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
# A gap larger than this between consecutive lines is a hole, not spacing. Section
|
||||
# spacing in the stock templates runs to roughly 45pt; 100pt is about seven lines.
|
||||
GAP_LIMIT_PT = 100.0
|
||||
|
||||
# Bottom whitespace on a page that is not the last one. The stock geometry leaves ~85pt.
|
||||
BOTTOM_LIMIT_FRACTION = 0.25
|
||||
|
||||
# A final page emptier than this reads as an unfinished document.
|
||||
LAST_PAGE_THIN_FRACTION = 0.35
|
||||
|
||||
# The page-number footer lives in the bottom margin and is a text line like any other to
|
||||
# Poppler. Ignore this band when measuring the body, or every page looks like it has a
|
||||
# hole above its footer.
|
||||
FOOTER_BAND_PT = 90.0
|
||||
|
||||
# A line indented at least this far past the page's left edge is a bullet or a
|
||||
# continuation, not an entry header or a section heading.
|
||||
INDENT_PT = 8.0
|
||||
|
||||
# A line this much taller than the body median is a section heading.
|
||||
HEADING_HEIGHT_RATIO = 1.25
|
||||
|
||||
PAGE_RE = re.compile(r'<page width="([\d.]+)" height="([\d.]+)">(.*?)</page>', re.S)
|
||||
WORD_RE = re.compile(
|
||||
r'<word xMin="([\d.]+)" yMin="([\d.]+)" xMax="[\d.]+" yMax="([\d.]+)">([^<]*)</word>'
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Line:
|
||||
top: float
|
||||
bottom: float
|
||||
left: float
|
||||
height: float
|
||||
text: str
|
||||
|
||||
|
||||
class Page:
|
||||
def __init__(self, height: float, lines: list[Line]):
|
||||
self.height = height
|
||||
self.lines = sorted(lines, key=lambda l: l.top)
|
||||
|
||||
@property
|
||||
def body(self) -> list[Line]:
|
||||
cutoff = self.height - FOOTER_BAND_PT
|
||||
return [l for l in self.lines if l.top < cutoff]
|
||||
|
||||
@property
|
||||
def empty(self) -> bool:
|
||||
return not self.body
|
||||
|
||||
@property
|
||||
def bottom_space(self) -> float:
|
||||
return self.height - max(l.bottom for l in self.body) if self.body else self.height
|
||||
|
||||
@property
|
||||
def left_edge(self) -> float:
|
||||
return min(l.left for l in self.body) if self.body else 0.0
|
||||
|
||||
@property
|
||||
def body_median_height(self) -> float:
|
||||
heights = sorted(l.height for l in self.body)
|
||||
return heights[len(heights) // 2] if heights else 0.0
|
||||
|
||||
@property
|
||||
def footer_crowded(self) -> bool:
|
||||
"""One line in the bottom band is a page number; two means body text spilled in."""
|
||||
band = self.height - FOOTER_BAND_PT
|
||||
return len({round(l.top, 1) for l in self.lines if l.top >= band}) > 1
|
||||
|
||||
def is_indented(self, line: Line) -> bool:
|
||||
return line.left > self.left_edge + INDENT_PT
|
||||
|
||||
def is_heading(self, line: Line) -> bool:
|
||||
median = self.body_median_height
|
||||
return bool(median) and line.height > median * HEADING_HEIGHT_RATIO
|
||||
|
||||
def largest_gap(self) -> tuple[float, float]:
|
||||
"""Largest top-to-top distance between body lines, and where it starts.
|
||||
|
||||
Measured top-to-top rather than bottom-to-top, so a tall line inflates the gap
|
||||
by its own height (a 160pt void under a heading reads as ~174pt). That errs
|
||||
toward over-detection, which is the right direction for a check whose job is
|
||||
to stop a hole from shipping.
|
||||
"""
|
||||
tops = sorted({round(l.top, 1) for l in self.body})
|
||||
if len(tops) < 2:
|
||||
return (0.0, 0.0)
|
||||
return max((tops[i + 1] - tops[i], tops[i]) for i in range(len(tops) - 1))
|
||||
|
||||
|
||||
def parse_pdf(path: Path) -> list[Page]:
|
||||
if not shutil.which("pdftotext"):
|
||||
raise RuntimeError("pdftotext (Poppler) not found; install poppler-utils")
|
||||
try:
|
||||
out = subprocess.run(
|
||||
["pdftotext", "-bbox", "-enc", "UTF-8", str(path), "-"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
# pdftotext emits UTF-8; without this Windows decodes it as cp1252 and
|
||||
# a non-ASCII glyph in the CV crashes the run (same fix as verify_pdf.py).
|
||||
encoding="utf-8",
|
||||
errors="replace",
|
||||
check=True,
|
||||
).stdout
|
||||
except subprocess.CalledProcessError as exc:
|
||||
# An xpdf-based pdftotext has no -bbox and exits 99. That is a broken extractor,
|
||||
# not a broken document, so it degrades to the skip path instead of exit 1.
|
||||
stderr_lines = (exc.stderr or "").strip().splitlines()
|
||||
detail = stderr_lines[0] if stderr_lines else f"exit {exc.returncode}"
|
||||
raise RuntimeError(
|
||||
f"pdftotext could not produce bounding boxes for {path} ({detail}); "
|
||||
"a pdftotext without -bbox is usually the xpdf build that Git for Windows "
|
||||
"puts ahead of Poppler in PATH"
|
||||
) from exc
|
||||
pages = []
|
||||
for _w, h, body in PAGE_RE.findall(out):
|
||||
buckets: dict[float, list[tuple[float, float, float, str]]] = {}
|
||||
for x_min, y_min, y_max, text in WORD_RE.findall(body):
|
||||
key = round(float(y_min), 0) # words on one line share a rounded yMin
|
||||
buckets.setdefault(key, []).append((float(x_min), float(y_min), float(y_max), text))
|
||||
lines = [
|
||||
Line(
|
||||
top=min(w[1] for w in words),
|
||||
bottom=max(w[2] for w in words),
|
||||
left=min(w[0] for w in words),
|
||||
height=max(w[2] - w[1] for w in words),
|
||||
text=" ".join(w[3] for w in sorted(words)),
|
||||
)
|
||||
for words in buckets.values()
|
||||
]
|
||||
pages.append(Page(float(h), lines))
|
||||
return pages
|
||||
|
||||
|
||||
def find_orphans(pages: list[Page]) -> list[str]:
|
||||
"""A page ending on an entry header or section heading whose content resumes overleaf.
|
||||
|
||||
Two shapes, both documented failures:
|
||||
* the last body line of a page is a section heading (stranded heading)
|
||||
* the last body lines are un-indented (an entry header) while the next page opens
|
||||
with indented bullet lines, i.e. the entry was split across the break
|
||||
"""
|
||||
problems = []
|
||||
# Indentation must be judged against the document's left margin, not each page's own
|
||||
# minimum: a page that *opens* with indented bullets would otherwise treat their
|
||||
# indent as its margin and report nothing.
|
||||
body_lines = [l for p in pages for l in p.body]
|
||||
if not body_lines:
|
||||
return problems
|
||||
doc_left = min(l.left for l in body_lines)
|
||||
|
||||
def indented(line: Line) -> bool:
|
||||
return line.left > doc_left + INDENT_PT
|
||||
|
||||
for i in range(len(pages) - 1):
|
||||
here, nxt = pages[i], pages[i + 1]
|
||||
if here.empty or nxt.empty:
|
||||
continue
|
||||
last, first = here.body[-1], nxt.body[0]
|
||||
|
||||
if here.is_heading(last):
|
||||
problems.append(
|
||||
f"p{i + 1} ends on the section heading {last.text.strip()!r} with its content "
|
||||
f"on p{i + 2}. Shorten the entry that follows it, or let the heading and its "
|
||||
"first entry move to the next page together"
|
||||
)
|
||||
elif not indented(last) and indented(first):
|
||||
# moderncv puts an itemize marker in its own bbox line at the list's left
|
||||
# edge, so a list item split across the break looks like an un-indented
|
||||
# header followed by indented text. Different defect, different fix.
|
||||
if not re.search(r"\w", last.text):
|
||||
problems.append(
|
||||
f"p{i + 1} ends on a lone list marker whose text continues on p{i + 2} "
|
||||
f"({first.text.strip()[:60]!r}): a bullet is split across the page break. "
|
||||
"Shorten the preceding content so the whole item fits on one page"
|
||||
)
|
||||
else:
|
||||
problems.append(
|
||||
f"p{i + 1} ends on the un-indented line {last.text.strip()[:60]!r} while "
|
||||
f"p{i + 2} opens with the indented line {first.text.strip()[:60]!r}: an entry "
|
||||
"header is orphaned from its bullets. Add \\needspace before that "
|
||||
"\\cventry, or shorten it"
|
||||
)
|
||||
return problems
|
||||
|
||||
|
||||
def report(path: Path, pages: list[Page]) -> list[str]:
|
||||
problems: list[str] = []
|
||||
print(f"{path}: {len(pages)} page(s) (page count is verify_pdf.py's job, not checked here)")
|
||||
|
||||
for i, page in enumerate(pages, 1):
|
||||
if page.empty:
|
||||
problems.append(f"p{i} contains no text")
|
||||
print(f" p{i}: EMPTY")
|
||||
continue
|
||||
|
||||
gap, gap_y = page.largest_gap()
|
||||
share = page.bottom_space / page.height
|
||||
print(
|
||||
f" p{i}: text y {page.body[0].top:.0f}..{page.body[-1].bottom:.0f}"
|
||||
f" of {page.height:.0f}pt | bottom {page.bottom_space:.0f}pt ({share * 100:.0f}%)"
|
||||
f" | largest gap {gap:.0f}pt at y{gap_y:.0f}"
|
||||
)
|
||||
|
||||
if gap > GAP_LIMIT_PT:
|
||||
problems.append(
|
||||
f"p{i} has a {gap:.0f}pt hole at y{gap_y:.0f} (~{gap / 14:.0f} blank lines). "
|
||||
"A moderncv \\cventry is an unbreakable tabular: shorten the entry that "
|
||||
"follows the hole so it fits, or move a shorter section above it"
|
||||
)
|
||||
if i < len(pages) and share > BOTTOM_LIMIT_FRACTION:
|
||||
problems.append(
|
||||
f"p{i} ends {page.bottom_space:.0f}pt ({share * 100:.0f}%) early although "
|
||||
"more pages follow, which reads as a broken page break"
|
||||
)
|
||||
if page.footer_crowded:
|
||||
problems.append(
|
||||
f"p{i} has body text inside the bottom margin band, colliding with the "
|
||||
"footer; stop stretching the page with \\enlargethispage and cut content"
|
||||
)
|
||||
if i == len(pages) > 1 and share > LAST_PAGE_THIN_FRACTION:
|
||||
problems.append(
|
||||
f"p{i} is the last page and {share * 100:.0f}% empty, which reads as an "
|
||||
"unfinished document; restore the highest-relevance content previously cut"
|
||||
)
|
||||
|
||||
problems.extend(find_orphans(pages))
|
||||
return problems
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("pdf", nargs="?", type=Path)
|
||||
args = ap.parse_args()
|
||||
|
||||
if not args.pdf:
|
||||
ap.error("pdf is required")
|
||||
if not args.pdf.exists():
|
||||
print(f"error: {args.pdf} not found", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
try:
|
||||
pages = parse_pdf(args.pdf)
|
||||
except RuntimeError as exc:
|
||||
print(f"skipped: {exc}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
problems = report(args.pdf, pages)
|
||||
if problems:
|
||||
print("\nLAYOUT PROBLEMS:")
|
||||
for m in problems:
|
||||
print(f" - {m}")
|
||||
return 1
|
||||
print("layout: clean")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user