feat(verify_pdf): add --ascii-dates, the non-folding check for the documented date-range rule (#385) (#498)

* feat(verify_pdf): add --ascii-dates, the non-folding check for the documented date-range rule (#385)

05-cv-templates.md records a real Workday import that dropped a role's
end date because `2016--2024` reaches the text layer as 2016<U+2013>2024,
and tells Step 5d to confirm every entry's years are joined by an ASCII
hyphen. Nothing executed that. The natural probe, --contains "2016-2024",
failed identically for "en-dashed" and "absent" before #458 and passes on
both since, because the fold maps the en-dash back to "-" on purpose.

Add --ascii-dates: scan the raw text layer (never the fold) for a year
joined to any Unicode dash or the minus sign and fail naming each hit
with its line and code point. A year on either side of the dash is
enough (Mar 2016 - Jul 2016, 2016 - Present); a numeric range with no
year is not a date. The dump is still written before the check fails.

Wire it in: Step 5d's extraction command carries the flag and the
"Dates recognizable" item points at it, keeping the bare single-year
half as a read-through; the guide's "add this to the step 5d checks"
paragraph names the command (framework_version 1.4.4 -> 1.4.5); the
upstream-only CI assertion runs it on the stock CV.

Demonstrated on the stock template compiled with lualatex through both
extractors: a \cventry{2016--2024} build fails with U+2013 while
--contains "2016-2024" passes on it; the \cventry{2016-2024} build passes
both; the untouched stock CV passes the CI assertion.

Ten new test_verify_pdf.py cases. Proposed by 9scorp4 in Discussion #385.

* fix(verify_pdf): --ascii-dates only joins a year and a dash across horizontal whitespace

`\s*` between the year and the dash also matched a newline, so a heading
ending in an ASCII date ("Heading 1988-1994\n- note") joined the dash
that merely opened the next line and was reported as a U+2013 range; the
mirror shape ("Six Sigma -\n2016 onwards") did the same. Both are real
text-layer shapes: a dash used as a bullet, a wrapped prose line.

Match horizontal whitespace only ([^\S\n]*). Every existing case still
passes; three fixtures pin the two newline shapes and the same-line
spaced dash that must still be caught.

Found in review by yang2632, with the regex and the fixture.

tools/verify_pdf.py

@@ -9,6 +9,11 @@ extractable characters. Poppler remains the fallback.
 Unicode normalization form (NFC), and the typographic substitutions LaTeX makes to
 the source text. The fold is comparison-time only - the `--dump-text` output stays
 the raw text layer an ATS parser actually sees.
+
+`--ascii-dates` is the one check that deliberately does NOT fold: it scans the raw
+layer for a year joined to a Unicode dash - the en-dash LaTeX makes from `--`, which
+a Workday import dropped along with the date (`05-cv-templates.md`, "Date fields
+must be ASCII ranges") - and fails naming each hit with its code point.
 """
 
 import argparse
@@ -84,6 +89,40 @@ def normalize_text(text):
     return " ".join(text.split())
 
 
+# Dash-like code points a year must never be joined to in a date range. U+2013 is
+# what LaTeX makes from `--` and what the documented Workday import dropped; the
+# rest are the other Unicode dashes and the minus sign, which a parser that splits
+# a range only on U+002D treats the same way.
+NON_ASCII_DASHES = "\u2010\u2011\u2012\u2013\u2014\u2015\u2212"
+_YEAR = r"(?:19|20)\d{2}"
+# Horizontal whitespace only between the year and the dash: `\s*` also matched
+# a newline, so a year ending one line joined a dash opening the next (a bullet,
+# a wrapped prose line) and an ASCII date was reported as U+2013.
+NON_ASCII_DATE_RANGE = re.compile(
+    rf"{_YEAR}[^\S\n]*[{NON_ASCII_DASHES}]|[{NON_ASCII_DASHES}][^\S\n]*{_YEAR}"
+)
+
+
+def find_non_ascii_date_ranges(text):
+    """Return (line, dash) for every year joined to a non-ASCII dash in raw text.
+
+    Works on the raw text layer, never on `normalize_text()` output: the fold
+    maps U+2013 back to `-` so `--contains "2016-2024"` can match what the
+    template renders, which is exactly why `--contains` cannot see this defect.
+    A year on either side of the dash is enough ("Mar 2016 - Jul 2016",
+    "2016 - Present"), so the check is locale-agnostic; a numeric range with
+    no year ("EUR 600k-1M") is not a date and is left alone.
+    """
+    hits = []
+    for match in NON_ASCII_DATE_RANGE.finditer(text):
+        dash = next(c for c in match.group(0) if c in NON_ASCII_DASHES)
+        line_start = text.rfind("\n", 0, match.start()) + 1
+        line_end = text.find("\n", match.end())
+        line = text[line_start : line_end if line_end != -1 else len(text)]
+        hits.append((" ".join(line.split()), dash))
+    return hits
+
+
 def _extract_pypdf(pdf_path):
     """Return (text, pages) or None if pypdf is unavailable, raises, or yields no text."""
     try:
@@ -120,7 +159,14 @@ def extract_text_layer(pdf_path):
     return text, pages, "pdftotext"
 
 
-def verify_pdf(pdf_path, expected_pages=None, min_chars=1, required_text=(), dump_text=None):
+def verify_pdf(
+    pdf_path,
+    expected_pages=None,
+    min_chars=1,
+    required_text=(),
+    dump_text=None,
+    ascii_dates=False,
+):
     pdf_path = Path(pdf_path)
     if not pdf_path.is_file():
         raise VerificationError(f"PDF does not exist: {pdf_path}")
@@ -158,6 +204,17 @@ def verify_pdf(pdf_path, expected_pages=None, min_chars=1, required_text=(), dum
             raise VerificationError(
                 f"text layer is missing required text: {required!r} (extractor: {extractor})"
             )
+
+    if ascii_dates:
+        hits = find_non_ascii_date_ranges(extracted_text)
+        if hits:
+            listed = "; ".join(f"U+{ord(dash):04X} in {line[:80]!r}" for line, dash in hits)
+            raise VerificationError(
+                f"{len(hits)} date range(s) joined by a non-ASCII dash - an ATS that splits "
+                f"ranges on U+002D drops the date; write the date argument with a single "
+                f"ASCII hyphen (05-cv-templates.md, 'Date fields must be ASCII ranges'): "
+                f"{listed} (extractor: {extractor})"
+            )
     return extractor, extracted_text, actual_pages
 
 
@@ -188,6 +245,15 @@ def build_parser():
         type=Path,
         help="write the extracted text layer to this path (UTF-8)",
     )
+    parser.add_argument(
+        "--ascii-dates",
+        action="store_true",
+        help=(
+            "fail if the raw text layer has a year joined to a Unicode dash (the "
+            "en-dash LaTeX makes from --), which ATS date parsers drop; this check "
+            "never folds, unlike --contains"
+        ),
+    )
     return parser
 
 
@@ -214,6 +280,7 @@ def main(argv=None):
             args.min_chars,
             args.contains,
             dump_text=args.dump_text,
+            ascii_dates=args.ascii_dates,
         )
     except VerificationError as exc:
         print(f"Error: {args.pdf}: {exc}", file=sys.stderr)