format_entry appended "* N/A = Too few employees to publish (privacy)"
under every category table, including one where every row has an index,
so the output asserted a suppression that never happened - the residual
noted on #470. Set a flag in the N/A* branch and print the footnote only
when it fired; a table with a suppressed row renders exactly as before.
Two FormatEntryTests cases pin both directions; the "omitted" case fails
on master.
--validate treats an explicit "metadata": null / "categories": null the same
as an omitted key ("...must be an object when provided", None is skipped), but
format_entry read both through dict.get(key, {}), which only substitutes the
default for an *absent* key - a present-but-null value passed through. The
renderer then hit None.get("index_label", ...) (AttributeError) or, via the
numeric-field fallback, None[key] = value (TypeError), so a hand-maintained
salary_data.json using null for "no value" died with an uncaught traceback
right after printing "Found 1 match(es)".
format_entry now coerces both to {} up front, honouring the validator's
existing "when provided" contract at the single consumer that broke it.
Tests (all verified to fail on the unfixed renderer):
- two unit cases calling format_entry with null metadata / null categories
- two end-to-end cases running main() --validate (blesses the file) then the
lookup path (renders it), one per null shape
Plus an [Unreleased] CHANGELOG entry.
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
The A.M.B.A. STRIP_PATTERNS regex ended in a literal dot followed by
\b, but \b can't fire right after a non-word character when the next
char is also non-word (space/end-of-string) - so it never matched any
realistic company name. The sibling undotted 'amba' suffix stripped
fine, so 'Arla Foods A.M.B.A.' and 'Arla Foods amba' normalized to
different strings and scored 86 vs 100 against the same query.
Made the trailing dot optional so the boundary resolves correctly.
validate_data() accepted category values that are not {count?, index?}
objects. They slipped through to format_entry(), which then raised
AttributeError on a normal table lookup (or serialized a malformed shape
under --json). It also accepted duplicate company names silently.
- collect_validation_issues() now also flags a non-object category value
(and non-numeric count / non number-or-string index) as a hard error,
and duplicate company names as a warning.
- validate_data() keeps its eager-fail behavior (same messages), so
existing tests and load_data() are unchanged.
- --validate runs the checks standalone and prints an actionable report
(exit 1 on errors, 0 on warnings-only/clean), letting users pre-flight
their BYO salary_data.json.
Reproduced on master: validate_data({'companies':[{'company':'Acme',
'categories':{'eng':'not_a_dict'}}]}) returns without error, but
format_entry then raises AttributeError.
Co-authored-by: Tunic Assistant <assistant@tunic.local>
This commit improves the performance and consistency of the salary tools:
- Redundant query normalization and word extraction are eliminated in salary_lookup.py by pre-calculating representations once before the search loop.
- A match_score_optimized helper is introduced to perform the comparison using the pre-calculated query data, preserving full backward compatibility for match_score.
- Normalization in tools/convert_salary_excel.py is unified: paired column headers now consistently substitute spaces and dashes with underscores (e.g. 'software_engineering') to match the single-column formatting.
- Unit test coverage is significantly expanded in tests/test_salary_lookup.py and tests/test_convert_salary_excel.py to cover normalization, anglicization, search filtering, and matching behaviors.
Ayobami Adegoke