Commit Graph
3 Commits
Author SHA1 Message Date
Alaa-TaiebandTunic Assistant 55ba1c1652 fix(salary): validate category shape and add --validate preflight (#156)
validate_data() accepted category values that are not {count?, index?}
objects. They slipped through to format_entry(), which then raised
AttributeError on a normal table lookup (or serialized a malformed shape
under --json). It also accepted duplicate company names silently.

- collect_validation_issues() now also flags a non-object category value
  (and non-numeric count / non number-or-string index) as a hard error,
  and duplicate company names as a warning.
- validate_data() keeps its eager-fail behavior (same messages), so
  existing tests and load_data() are unchanged.
- --validate runs the checks standalone and prints an actionable report
  (exit 1 on errors, 0 on warnings-only/clean), letting users pre-flight
  their BYO salary_data.json.

Reproduced on master: validate_data({'companies':[{'company':'Acme',
'categories':{'eng':'not_a_dict'}}]}) returns without error, but
format_entry then raises AttributeError.

Co-authored-by: Tunic Assistant <assistant@tunic.local>
2026-07-15 07:53:01 +02:00
Alaa-Taieb 1417e3cbdf fix(salary): skip non-numeric and identifier columns in Excel conversion (#152)
parse_sheet treated every column that was not company/city as a salary category, with no check that the column actually held numeric salary data. This turned free-text columns (e.g. Notes) into bogus string categories and numeric identifier columns (e.g. Id) into mistaken salary indexes.

- Drop identifier headers (ID_PATTERNS = {id, personnummer}) at classification time.

- Skip non-numeric standalone values and fully-null count/index pairs at row-processing time.

- Adds regression tests (skips_free_text_column, skips_numeric_identifier_column, keeps_numeric_salary_column) that fail on master and pass after the fix.
2026-07-14 20:11:11 +02:00
Alaa-Taieb 4128ca0318 fix(salary): detect company column from header token, not exact match (#151)
convert_salary_excel.py detected the company column via exact membership
in COMPANY_PATTERNS, so common real-world headers like "Company Name" or
"Employer Name" were never matched. parse_sheet then returned [] for that
sheet, silently dropping it from salary_data.json (or exiting with no
output for a single-sheet file).

Route company-column detection through the existing header_matches()
token matcher (already used for count/index detection). This only adds
detections; inputs that already worked (bare "Company"/"Firma"/...) are
unaffected.

Adds a regression test in tests/test_convert_salary_excel.py that fails
on master (returns []) and passes after the fix.
2026-07-14 14:35:45 +02:00