fix(salary-tools): decide number locale by last separator, pair compound headers

Review findings F7 and F8 (2026-08-19):

- parse_numeric_cell's both-separators branch always assumed European
  locale, silently turning a US "1,234.56" into 1.23456 - a 1000x
  corruption written to salary_data.json with no warning. The separator
  that appears last is now treated as the decimal separator, which also
  makes multi-group values ("1,234,567.89") parse instead of raising a
  raw float error. Single-separator ambiguity guards are unchanged.

- strip_type_patterns stripped only whole tokens, so the compound header
  "Lønindeks alle" kept its type word and could never pair with "Antal
  alle" - failing exactly for the compound-word locale COMPOUND_PATTERNS
  exists to support. It now also strips compound patterns as substrings,
  mirroring header_matches.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Mads Lorentzen
2026-08-19 19:48:50 +02:00
co-authored by Claude Opus 5
parent 7aba0b4a9d
commit 1c19f6c45f
3 changed files with 78 additions and 2 deletions
+16 -2
View File
@@ -65,7 +65,13 @@ def parse_numeric_cell(value):
if not text:
raise ValueError("not numeric")
if "," in text and "." in text:
text = text.replace(".", "").replace(",", ".")
# The separator that appears last is the decimal separator: European
# "1.234,56" and US "1,234.56" are both unambiguous here, unlike the
# single-separator cases below.
if text.rfind(",") > text.rfind("."):
text = text.replace(".", "").replace(",", ".")
else:
text = text.replace(",", "")
elif "," in text:
if re.fullmatch(r"[+-]?\d+,\d{3}", text):
raise ValueError("ambiguous comma separator")
@@ -95,10 +101,18 @@ def header_matches(header, patterns):
def strip_type_patterns(header, patterns):
"""Remove count/index words from a header to derive a category name."""
"""Remove count/index words from a header to derive a category name.
Mirrors ``header_matches``: patterns strip as whole tokens, and any
pattern also listed in ``COMPOUND_PATTERNS`` additionally strips as a
substring - otherwise a compound header like "Lønindeks alle" keeps the
type word in its category name and can never pair with "Antal alle".
"""
name = header.lower()
for p in patterns:
name = re.sub(rf"(?<![a-zæøåöäü0-9]){re.escape(p)}(?![a-zæøåöäü0-9])", "", name)
if p in COMPOUND_PATTERNS:
name = name.replace(p, "")
return name.strip(" _-")