fix(tools): write UTF-8 on stdout so non-Latin postings stop crashing on Windows (#490)

A piped stdout on Windows defaults to the ANSI code page (cp1252 on most
Western installs). rank_state.py printed titles and companies with
ensure_ascii=False, so one Cyrillic, CJK, Devanagari, Polish or Turkish
posting ended /rank with UnicodeEncodeError. job_key --audit, salary_lookup,
convert_salary_excel and verify_layout failed the same way.

Each tool now reconfigures stdout and stderr to UTF-8 at entry. The new
tests run every tool in a child process with a cp1252 stdout forced through
PYTHONIOENCODING, so Linux CI reproduces the Windows failure.


Claude-Session: https://claude.ai/code/session_01WPXraLZ4i7UW4xGn9tVcox

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Mads LorentzenandClaude Fable 5.1 authored and GitHub committed 2026-09-21 08:37:18 +02:00
1 parent 2e8600d69d
commit ba4f2397a3
10 files changed
+258 -3

No files matched your search

+2 -1
View File
@@ -52,6 +52,7 @@ class RankStateCase(unittest.TestCase):
[sys.executable, str(TOOL), *args, "--state", str(self.state), "--today", TODAY],
capture_output=True,
text=True,
encoding="utf-8",
)
self.assertEqual(proc.returncode, expect, proc.stderr)
return json.loads(proc.stdout)
@@ -148,7 +149,7 @@ class Candidates(RankStateCase):
proc = subprocess.run(
[sys.executable, str(TOOL), "candidates", "--state", str(self.tmp / "nope.json"),
"--tracker", str(self.tmp / "n.csv")],
capture_output=True, text=True,
capture_output=True, text=True, encoding="utf-8",
)
self.assertNotEqual(proc.returncode, 0)
self.assertIn("not found", proc.stderr + proc.stdout)