* fix(rank): retry postings that could not be fetched instead of expiring them
A posting /rank could not fetch (403, login wall, 404, timeout, empty
page) and one that is really closed both became `expired`, which is
terminal: `candidates` only selected `new`, so a transient failure on
ranking day removed a live job from the pipeline and the report
presented it as closed.
Shape per the maintainer's decisions on #534:
- Scoring agents return a third status, `unreachable` (with a short
`fetch_error`), after exhausting the 09-web-research.md escalation
order; `expired` stays for positive evidence of closure (closed page,
410, redirect to a listing page, passed deadline). A 404 is
`unreachable` first (#538 is that case).
- `candidates` selects `unreachable` alongside `new`. `apply` increments
`fetch_attempts`, keeps `first_unreachable` (UTC timestamp) and stores
`last_fetch_error` (short reason, no page text). After 3 attempts with
at least 48 hours between the first and the last it writes `expired`
with `expired_reason: "unreachable"` and returns the job once under
`gave_up`, which Step 5 prints under "Gave up, check manually" with
its URL. A later successful fetch, an explicit `expired` or a passed
deadline clears the retry fields. `--all` still revives everything.
- rank_state.py owns the status set (`AGENT_STATUSES`); `apply` rejects
any other status instead of reporting a misleading missing score, and
test_rank_command.py asserts rank.md's JSON contract names the same
set. A new `--now` option makes the 48-hour floor testable.
- rank.md's step text (Step 2 rules and contract, rule 6's --all note,
Step 4 fields, Step 5 report, Important Rule 1) updated together.
CHANGELOG under Changed: no existing entry carries the new status, so
no migration.
Reported by @ai-tvang in #534, who also proposed the time floor and the
give-up listing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(rank): expire an unreachable posting whose deadline has passed
Review on #564:
- The unreachable branch in apply returned before the deadline check the
scored path runs, and candidates hands out `new` entries whose stored
deadline has passed. So a posting that closed on 2026-09-20 and
returned 403 on 2026-10-05 was retried, then listed under "Gave up,
check manually" as possibly live. A passed stored deadline is now
checked first and takes the expired path, dropping the retry fields,
as on master and under rule 5. New test pins it.
- 09-web-research.md step 5 said that in /rank an unretrievable posting
is marked `expired`, so an agent following it could write a 403 off
and bring #534 back. It now says `unreachable` (framework_version
1.1.2 -> 1.1.3); the CHANGELOG entry says so.
- The report shows `attempt N` instead of `N/3`, which read 4/3 while a
job waited out the 48-hour floor, and job-scraper's dedup note names
`unreachable` alongside `ranked`/`expired`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
A piped stdout on Windows defaults to the ANSI code page (cp1252 on most
Western installs). rank_state.py printed titles and companies with
ensure_ascii=False, so one Cyrillic, CJK, Devanagari, Polish or Turkish
posting ended /rank with UnicodeEncodeError. job_key --audit, salary_lookup,
convert_salary_excel and verify_layout failed the same way.
Each tool now reconfigures stdout and stderr to UTF-8 at entry. The new
tests run every tool in a child process with a cp1252 stdout forced through
PYTHONIOENCODING, so Linux CI reproduces the Windows failure.
Claude-Session: https://claude.ai/code/session_01WPXraLZ4i7UW4xGn9tVcox
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* fix(rank): move seen_jobs.json read/write off the state file's own critical path (#395)
/rank's Step 1 read the whole of seen_jobs.json into the conversation to
select candidates by eye, and Step 4 emitted it back to record scores.
That cost is paid on every run regardless of how many jobs are scored,
and it grows for the life of the workspace, since the file is
append-only and most stored entries are `skipped`.
tools/rank_state.py moves that traffic into code:
- `candidates` selects entries per Step 1's existing rules (status
filter, tracker exclusion, focus filter, `--limit`/`--all` from #424)
and projects only the fields a scoring agent needs.
- `sweep` runs rule 6's expiry pass over entries the run did not
re-score - a stored-date comparison, no fetch, no agent - preserving
its defensive parsing of non-ISO deadlines and its `--all`
reversibility.
- `apply` writes scoring results back atomically and prints the
ranked/vetoed/expired rows Step 5's report is built from, preserving
Step 4's existing write-back rules exactly: the `location` ->
`location_verdict` legacy migration, the deadline
null-is-not-a-correction rule, verbatim strengths/gaps persistence,
and idempotent re-scoring.
Step 1, Step 3's rule 6, and Step 4 now route through the tool instead
of describing a manual read/write. Nothing about scoring policy changes
- no new status, no new persisted field, no change to what counts as a
veto. The tracker stays read-only and every write is atomic (temp file
+ rename).
tests/test_rank_state.py (25 tests) covers the three subcommands
directly. The new spec-guard class in test_rank_command.py derives the
fields Step 4 must preserve from Step 2's own JSON schema block rather
than retyping them as a second list, so a future edit to that contract
is what the test reads instead of something that can drift from it.
* fix(rank): add CHANGELOG entry and remove the undefined $SCRATCHPAD reference
Two mechanical fixes from review:
- Step 4 named the results hand-off file via $SCRATCHPAD, a variable
nothing in the repo defines - a reader following the spec literally
has no path to substitute. Named the location in prose instead (a
temporary file outside the repo tree, never committed) and replaced
the shell-variable-looking path in the example command with an
explicit placeholder.
- Added the [Unreleased] entry this change was missing; the one
already in the diff belongs to #424.
* changelog: fold the #395 entry into the existing Fixed section
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013fqqLgQSnwgWkv98twQhHi
---------
Co-authored-by: nox <nox@Mac.home>
Co-authored-by: Mads Lorentzen <madslorentzen17@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
