fix(job_key): hash the URL alone so a re-listed non-Latin title keeps one key (#502)

* fix(job_key): hash the URL alone so a re-listed non-Latin title keeps one key

* docs(changelog): record the job_key URL-identity fix

Adds the [Unreleased] / Fixed entry for #501, naming tools/job_key.py and
tests/test_job_key_url_identity.py.

The entry states the migration consequence in bold: an entry written before
this fix, for a non-Latin title on a portal without a numeric id, re-keys once
on the next scrape. It also names where those entries surface,
keys_not_matching_current_rule, and that --audit never rewrites them.

Verified that claim rather than asserting it. A seen_jobs.json holding one
entry under its pre-fix key for a Cyrillic-titled freehire URL reports
keys_not_matching_current_rule: 1 under the fixed audit. The same audit over a
real 884-entry seen_jobs.json reports 0, because those keys are the
company_title form that is_canonical() excludes from the drift check.
This commit is contained in:
Ahmed Ashraf authored and GitHub committed 2026-10-01 21:45:19 +02:00
1 parent d7287ec834
commit a56b8d0414
3 files changed
+56 -1

No files matched your search

+15
View File
@@ -13,6 +13,21 @@ per-file diff commands.
## [Unreleased]
### Fixed
- **One posting on a portal without a numeric id no longer gets a second key when its
title is re-listed with different casing** (#501, `tools/job_key.py`,
`tests/test_job_key.py`) - the key hashed a slugified title alongside the URL whenever
`normalizeId` found no six-digit run in the path, which is every freehire posting. The
live API returns both `Инженер` and `инженер` across rows, and the slugifier maps those
to different slugs, so one URL could be stored under several keys. `/rank` builds its
exclusion set from `seen_jobs.json`, so a posting already seen could be presented again.
The key is now a hash of the URL alone: the URL is the identity `/scrape` stores, and
the title was never a stable half of it. **Entries written before this fix for a
non-Latin title on a portal without a numeric id will re-key once on the next scrape.**
`python3 tools/job_key.py --audit` reports those entries under
`keys_not_matching_current_rule`; it never rewrites them.
## [1.7.2] - 2026-09-29
### Added