fix(tools): write UTF-8 on stdout so non-Latin postings stop crashing on Windows (#490)

A piped stdout on Windows defaults to the ANSI code page (cp1252 on most
Western installs). rank_state.py printed titles and companies with
ensure_ascii=False, so one Cyrillic, CJK, Devanagari, Polish or Turkish
posting ended /rank with UnicodeEncodeError. job_key --audit, salary_lookup,
convert_salary_excel and verify_layout failed the same way.

Each tool now reconfigures stdout and stderr to UTF-8 at entry. The new
tests run every tool in a child process with a cp1252 stdout forced through
PYTHONIOENCODING, so Linux CI reproduces the Windows failure.


Claude-Session: https://claude.ai/code/session_01WPXraLZ4i7UW4xGn9tVcox

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Mads LorentzenandClaude Fable 5.1 authored and GitHub committed 2026-09-21 08:37:18 +02:00
1 parent 2e8600d69d
commit ba4f2397a3
10 files changed
+258 -3

No files matched your search

+2 -2
View File
@@ -76,7 +76,7 @@ class CompanyFallbackCLI(unittest.TestCase):
proc = subprocess.run(
[sys.executable, str(TOOL), "--company", company,
"--title", "Software Engineer", "--url", url],
capture_output=True, text=True, check=True,
capture_output=True, text=True, encoding="utf-8", check=True,
)
return proc.stdout.strip()
@@ -126,7 +126,7 @@ class AuditCLI(unittest.TestCase):
json.dump({"seen": seen}, fh)
path = fh.name
proc = subprocess.run(
[sys.executable, str(TOOL), "--audit", path], capture_output=True, text=True
[sys.executable, str(TOOL), "--audit", path], capture_output=True, text=True, encoding="utf-8"
)
return json.loads(proc.stdout), proc.returncode