py/ai-job-search
1
0
Fork 0
mirror of https://github.com/MadsLorentzen/ai-job-search.git synced 2026-09-17 00:26:26 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor(salary): make compound-word matching locale-agnostic (#94)

Browse Source 
* refactor(salary): make compound-word matching locale-agnostic

The Excel column detector hardcoded a DANISH_COMPOUND_PATTERNS set inside
header_matches(), so the compound-word matching that helps Danish headers
(e.g. "lønindeks") was baked into the algorithm by name and unavailable to
any other locale without editing the source.

Rename it to COMPOUND_PATTERNS and pass it as a parameter (default
unchanged, so the Danish demonstration data behaves identically). A
different-locale spreadsheet can now supply its own compound tokens via
header_matches(..., compound_patterns=...). Add a test covering both the
preserved default and the parameterized path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(salary): drop unused compound_patterns parameter

Per review: keep the DANISH_COMPOUND_PATTERNS -> COMPOUND_PATTERNS
rename (universal template naming, defaults still Danish), but remove
the compound_patterns= parameter. No caller passes a custom set, and a
fork adapting another locale edits the module-level constant either way,
so parameterizing it is speculative generality per CONTRIBUTING.md.

header_matches() now reads COMPOUND_PATTERNS directly. Test updated to
verify compound-vs-whole-token matching against the constant.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Erik Pastor Rios
2026-07-10 08:06:05 +02:00
committed by GitHub
co-authored by Claude Opus 4.8
parent cd7c22325b
commit a278ad7a50
2 changed files with 27 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/convert_salary_excel.py
+13 -3
View File
@@ -42,18 +42,28 @@ COMPANY_PATTERNS = {"firma", "company", "virksomhed", "employer", "arbejdsgiver"
CITY_PATTERNS = {"by", "city", "kommune", "location", "lokation", "sted"}
COUNT_PATTERNS = {"antal", "count", "number", "n", "employees", "medarbejdere"}
INDEX_PATTERNS = {"indeks", "index", "idx", "salary", "løn", "median", "average", "gennemsnit"}
DANISH_COMPOUND_PATTERNS = {"antal", "indeks", "løn", "gennemsnit", "medarbejdere"}
# "Compound" tokens: pattern words allowed to match as a substring of a larger
# header token, for languages that glue words together (e.g. Danish "lønindeks"
# -> løn + indeks). Languages that write headers as separate words need none.
# Ships populated for this repo's Danish demonstration data; a fork targeting
# another locale edits this constant.
COMPOUND_PATTERNS = {"antal", "indeks", "løn", "gennemsnit", "medarbejdere"}
def header_matches(header, patterns):
"""Return True when a header contains a meaningful pattern match."""
"""Return True when a header contains a meaningful pattern match.
Patterns match whole tokens; any pattern also listed in
``COMPOUND_PATTERNS`` may additionally match as a substring, to handle
languages that form compound words.
"""
h = header.lower().strip()
tokens = set(re.findall(r"[a-zæøåöäü0-9]+", h))
for p in patterns:
if p in tokens:
return True
if p in DANISH_COMPOUND_PATTERNS and p in h:
if p in COMPOUND_PATTERNS and p in h:
return True
return False