fix(salary): require corroboration before accepting a header row (#415)

* fix(salary): require corroboration before accepting a header row

Header-row detection accepted the first row (of the first 10) where any
cell merely contained a company-pattern word - no check that the row
actually looked like a header. A source-citation row above the real
header table (standard in real Danish union/statistics exports, e.g.
"Kilde: ... opdelt efter arbejdsgiver ...") tripped it purely because
"arbejdsgiver" appeared in prose. The real header row then parsed as
data (its "Firma" cell became a bogus company), and every genuine
company silently lost all its salary data - exit 0, no warning.

A candidate row is now only accepted when a second cell also matches a
city/count/index pattern, and a sheet that ends up with zero detected
salary columns prints a warning instead of reporting success silently.

Fixes #414.

* fix(salary): require cross-cell corroboration, fall back for untyped columns

Two edge cases found in review of the corroboration fix:

- Same-cell corroboration wasn't enough: a citation sentence can pack a
  count-pattern word into the same sentence as the company-pattern one
  ("...opdelt efter arbejdsgiver, antal svar 1234"), which still passed
  the gate. Corroboration must now come from a different cell.

- The corroboration requirement itself broke sheets whose only real
  header has purely untyped salary columns (e.g. "Base pay 2025" /
  "Bonus 2025" - neither matches a known city/count/index pattern), so
  header detection found nothing at all. Falls back to the original
  any-cell-mentions-company rule when the strict pass finds no row in
  the first 10.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

tools/convert_salary_excel.py

@@ -127,15 +127,47 @@ def detect_column_type(header):
 
 def parse_sheet(ws, sheet_label=None):
     """Parse a single worksheet into a list of company entries and detected categories."""
-    # Find header row
+    # Find header row. Two passes:
+    #
+    # Strict pass: a candidate row needs a company-pattern cell AND a
+    # DIFFERENT cell matching a city/count/index pattern. Corroboration must
+    # come from a separate cell - a single free-text sentence can pack both
+    # a company-pattern word and a count-pattern word together (e.g. "...
+    # opdelt efter arbejdsgiver, antal svar 1234"), and that must not read
+    # as a header any more than a citation mentioning just one of them does.
+    # A real header row always has these as separate columns.
+    #
+    # Fallback pass: some real headers have no recognizable city/count/index
+    # column at all (e.g. "Company | Base pay 2025 | Bonus 2025" - neither
+    # data header matches a known pattern, so they're picked up later as
+    # untyped/standalone categories). Nothing can corroborate a company match
+    # there, so if the strict pass finds no row in the first 10, fall back to
+    # the original any-cell-mentions-company rule.
+    rows = list(ws.iter_rows(min_row=1, max_row=10, values_only=False))
+
+    def _cell_texts(row):
+        return [str(cell.value).strip() for cell in row if cell.value]
+
     header_row = None
-    for row_idx, row in enumerate(ws.iter_rows(min_row=1, max_row=10, values_only=False), start=1):
-        for cell in row:
-            if cell.value and header_matches(str(cell.value), COMPANY_PATTERNS):
+    for row_idx, row in enumerate(rows, start=1):
+        cell_texts = _cell_texts(row)
+        company_idxs = {i for i, t in enumerate(cell_texts) if header_matches(t, COMPANY_PATTERNS)}
+        if not company_idxs:
+            continue
+        other_idxs = {
+            i
+            for i, t in enumerate(cell_texts)
+            if header_matches(t, CITY_PATTERNS) or header_matches(t, COUNT_PATTERNS) or header_matches(t, INDEX_PATTERNS)
+        }
+        if other_idxs - company_idxs:
+            header_row = row_idx
+            break
+
+    if header_row is None:
+        for row_idx, row in enumerate(rows, start=1):
+            if any(header_matches(t, COMPANY_PATTERNS) for t in _cell_texts(row)):
                 header_row = row_idx
                 break
-        if header_row:
-            break
 
     if header_row is None:
         print(f"Warning: Could not find header row in sheet '{ws.title}'. Skipping.", file=sys.stderr)
@@ -222,6 +254,14 @@ def parse_sheet(ws, sheet_label=None):
     for col_idx, col_header in untyped_cols:
         categories.append({"name": col_header.lower().replace(" ", "_"), "value_col": col_idx})
 
+    if not categories:
+        print(
+            f"Warning: No salary data columns detected in sheet '{ws.title}' "
+            "(only a company/city column was found) - the header row may be "
+            "wrong, or this sheet has no salary data.",
+            file=sys.stderr,
+        )
+
     # Parse data rows
     companies = []
     for row in ws.iter_rows(min_row=header_row + 1, values_only=True):