refactor(salary): optimize search match scoring and normalize Excel category keys (#101)

This commit improves the performance and consistency of the salary tools:

- Redundant query normalization and word extraction are eliminated in salary_lookup.py by pre-calculating representations once before the search loop.
- A match_score_optimized helper is introduced to perform the comparison using the pre-calculated query data, preserving full backward compatibility for match_score.
- Normalization in tools/convert_salary_excel.py is unified: paired column headers now consistently substitute spaces and dashes with underscores (e.g. 'software_engineering') to match the single-column formatting.
- Unit test coverage is significantly expanded in tests/test_salary_lookup.py and tests/test_convert_salary_excel.py to cover normalization, anglicization, search filtering, and matching behaviors.
This commit is contained in:
♦ jabarii♦
2026-07-10 15:24:20 +02:00
committed by GitHub
parent 7e8df35819
commit c134eef553
4 changed files with 112 additions and 18 deletions
+4
View File
@@ -145,6 +145,8 @@ def parse_sheet(ws, sheet_label=None):
cat_name = strip_type_patterns(col_header, COUNT_PATTERNS)
if not cat_name:
cat_name = f"category_{len(categories)+1}"
else:
cat_name = cat_name.replace(" ", "_").replace("-", "_")
categories.append({
"name": cat_name,
"count_col": col_idx,
@@ -156,6 +158,8 @@ def parse_sheet(ws, sheet_label=None):
cat_name = strip_type_patterns(col_header, INDEX_PATTERNS)
if not cat_name:
cat_name = f"category_{len(categories)+1}"
else:
cat_name = cat_name.replace(" ", "_").replace("-", "_")
categories.append({
"name": cat_name,
"index_col": col_idx,