mirror of
https://github.com/MadsLorentzen/ai-job-search.git
synced 2026-11-02 13:12:50 +00:00
fix(job_key): hash the URL alone so a re-listed non-Latin title keeps one key (#502)
* fix(job_key): hash the URL alone so a re-listed non-Latin title keeps one key * docs(changelog): record the job_key URL-identity fix Adds the [Unreleased] / Fixed entry for #501, naming tools/job_key.py and tests/test_job_key_url_identity.py. The entry states the migration consequence in bold: an entry written before this fix, for a non-Latin title on a portal without a numeric id, re-keys once on the next scrape. It also names where those entries surface, keys_not_matching_current_rule, and that --audit never rewrites them. Verified that claim rather than asserting it. A seen_jobs.json holding one entry under its pre-fix key for a Cyrillic-titled freehire URL reports keys_not_matching_current_rule: 1 under the fixed audit. The same audit over a real 884-entry seen_jobs.json reports 0, because those keys are the company_title form that is_canonical() excludes from the drift check.
This commit is contained in:
1 parent
d7287ec834
commit
a56b8d0414
3 files changed
+56
-1
No files matched your search
@@ -13,6 +13,21 @@ per-file diff commands.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Fixed
|
||||
|
||||
- **One posting on a portal without a numeric id no longer gets a second key when its
|
||||
title is re-listed with different casing** (#501, `tools/job_key.py`,
|
||||
`tests/test_job_key.py`) - the key hashed a slugified title alongside the URL whenever
|
||||
`normalizeId` found no six-digit run in the path, which is every freehire posting. The
|
||||
live API returns both `Инженер` and `инженер` across rows, and the slugifier maps those
|
||||
to different slugs, so one URL could be stored under several keys. `/rank` builds its
|
||||
exclusion set from `seen_jobs.json`, so a posting already seen could be presented again.
|
||||
The key is now a hash of the URL alone: the URL is the identity `/scrape` stores, and
|
||||
the title was never a stable half of it. **Entries written before this fix for a
|
||||
non-Latin title on a portal without a numeric id will re-key once on the next scrape.**
|
||||
`python3 tools/job_key.py --audit` reports those entries under
|
||||
`keys_not_matching_current_rule`; it never rewrites them.
|
||||
|
||||
## [1.7.2] - 2026-09-29
|
||||
|
||||
### Added
|
||||
|
||||
Reference in new issue
Block a user