mirror of
https://github.com/MadsLorentzen/ai-job-search.git
synced 2026-11-02 13:12:50 +00:00
fix(tools): write UTF-8 on stdout so non-Latin postings stop crashing on Windows (#490)
A piped stdout on Windows defaults to the ANSI code page (cp1252 on most Western installs). rank_state.py printed titles and companies with ensure_ascii=False, so one Cyrillic, CJK, Devanagari, Polish or Turkish posting ended /rank with UnicodeEncodeError. job_key --audit, salary_lookup, convert_salary_excel and verify_layout failed the same way. Each tool now reconfigures stdout and stderr to UTF-8 at entry. The new tests run every tool in a child process with a cp1252 stdout forced through PYTHONIOENCODING, so Linux CI reproduces the Windows failure. Claude-Session: https://claude.ai/code/session_01WPXraLZ4i7UW4xGn9tVcox Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
2e8600d69d
commit
ba4f2397a3
10 files changed
+258
-3
No files matched your search
@@ -349,7 +349,21 @@ def cmd_apply(args) -> int:
|
||||
return 1 if errors else 0
|
||||
|
||||
|
||||
def _force_utf8_output() -> None:
|
||||
"""Write UTF-8 whatever the host's default encoding is.
|
||||
|
||||
A piped stdout on Windows defaults to the ANSI code page (cp1252 on most
|
||||
Western installs), so printing a company, title or file name outside it
|
||||
raised UnicodeEncodeError before the workflow saw any output.
|
||||
"""
|
||||
for stream in (sys.stdout, sys.stderr):
|
||||
reconfigure = getattr(stream, "reconfigure", None) # absent on a StringIO under test
|
||||
if reconfigure:
|
||||
reconfigure(encoding="utf-8")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
_force_utf8_output()
|
||||
common = argparse.ArgumentParser(add_help=False)
|
||||
common.add_argument("--state", type=Path, default=STATE)
|
||||
common.add_argument("--today", type=date.fromisoformat, default=date.today())
|
||||
|
||||
Reference in new issue
Block a user