fix(notion-sync): re-key a row matched by URL when its Key has drifted (#761)

* fix(notion-sync): re-key a row matched by URL when its Key has drifted

`tools/job_key.py --audit --migrate` (#637) renames keys in
seen_jobs.json, but Step 4 upserted on Key alone, so the next sync
created a duplicate row for every migrated key. On a Key miss, look up
keyed rows whose URL equals the job's URL exactly: one match gets the
new Key and is updated, several are reported and left untouched, none
creates as before. Rows without a Key stay invisible, and the report
counts re-keyed rows separately.

Shape from Mads Lorentzen's review on #637.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore: drop the CHANGELOG hunk, frozen by #800 (release notes come from the PR title)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* test(notion-sync): pin the key-collision guard and name the skip counter for both ambiguous cases

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Thomas Christian VangandClaude Opus 5.5 authored and GitHub committed 2026-11-03 20:20:19 +01:00
1 parent a3ba937be8
commit a813602972
2 files changed
+34 -6

No files matched your search

+10 -6
View File
@@ -89,10 +89,13 @@ Validate the cheap, local precondition before creating anything external. A run
For each job in the sync set:
1. Query the database for a page whose `Key` equals the job's key.
2. **No match** → if the job is an expired-only entry, skip it - do not create a page or write a body. Otherwise, create the page with all properties from the Step 3 table, then write its body (Step 5).
3. **Match, expired-only entry** → update **Status only** to `expired`. Do not update other properties or touch the page body.
2. **No match on `Key`** → fall back to the posting URL before creating anything. A job's key can change after its page was created (a key migration such as `tools/job_key.py --audit --migrate` renames keys in `job_scraper/seen_jobs.json`), and creating on a key miss would give that job a second row. Query the database for pages whose `URL` equals the job's URL exactly (the same string, not a similar one) **and** whose `Key` is non-empty - rows without a `Key` stay invisible here too (item 5). Skip the lookup when the job has no URL (e.g. a tracker-only row).
- **Exactly one URL match** → re-key it: set its `Key` to the job's key, then handle it as a match - item 3 for an expired-only entry, item 4 otherwise. If the row's current `Key` is the key of another job in this sync set, it is not drifted; treat it like a multiple match instead.
- **More than one URL match** → create nothing and update nothing for this job; report it with the `Key` of every matching row so the user can resolve the duplicates by hand.
- **No URL match** → if the job is an expired-only entry, skip it - do not create a page or write a body. Otherwise, create the page with all properties from the Step 3 table, then write its body (Step 5).
3. **Match, expired-only entry** → update **Status only** to `expired` (plus `Key` when item 2 re-keyed the row). Do not update other properties or touch the page body.
4. **Match, any other effective status** → update **properties only**: Status, Score, Verdict, Deadline, Ranked, Applied on, Channel, CV file, Cover letter. Properties are the always-current surface (bodies are write-once), so tracker updates recorded by `/outcome` reach the destination exclusively through them. Do not touch the page body - the user may have added their own notes there, and clobbering them breaks trust in the whole view. (`--rebuild` is the sole exception for non-expired jobs.)
5. Never delete or archive pages, even for jobs that turned `expired` - set Status to `expired` instead. Rows the user added to the database by hand (no `Key` value) are invisible to this command.
5. Never delete or archive pages, even for jobs that turned `expired` - set Status to `expired` instead. Rows the user added to the database by hand (no `Key` value) are invisible to this command, including to the URL fallback in item 2, which only considers rows that already carry a `Key`.
**Normalise the Status value before writing.** The tracker may hold legacy space spellings (`no response`, `offer declined`) from before the canonical forms were locked. Map them to `no_response` / `offer_declined` per the **Tracker status vocabulary** in `/outcome` before setting Status on create or update - never push a space form to Notion, which would auto-create a separate select option per unique string. Pre-existing space-form options in an existing database simply go unused; Notion never auto-removes select options.
@@ -119,15 +122,16 @@ Keep the page under ~40 blocks; this is a briefing, not a mirror of the posting.
## Pipeline Sync - YYYY-MM-DD
Database: <database_url>
Synced <N> jobs (threshold: score ≥ <T>): <C> created, <U> updated, <S> unchanged, <F> failed.
Synced <N> jobs (threshold: score ≥ <T>): <C> created, <U> updated, <R> re-keyed by URL, <S> unchanged, <A> skipped (ambiguous URL match), <F> failed.
| | Title | Company | Status | Score |
|---|-------|---------|--------|-------|
| ✚ | ... | ... | ranked | 78 |
| ↻ | ... | ... | interview | 71 |
| ⇄ | ... | ... | applied | 74 |
```
List failures with their error and the suggestion to re-run - the upsert is idempotent, so a re-run only touches what failed. Update `last_sync` in the sync-state file.
A re-keyed row (⇄, Step 4.2) counts under re-keyed, not updated. List each skipped job with its URL and the `Key` of every row that URL matched. List failures with their error and the suggestion to re-run - the upsert is idempotent, so a re-run only touches what failed. Update `last_sync` in the sync-state file.
Remind the user once (first run only): the repo files remain the source of truth - edits made in the destination never flow back, and `/outcome` is still how application results get recorded.
@@ -136,7 +140,7 @@ Remind the user once (first run only): the repo files remain the source of truth
## Important Rules
1. **One-way, always.** Destination content never flows back into `job_scraper/seen_jobs.json`, the tracker, or any repo file. This command reads repo state and writes the destination - both repo files are read-only to it, and the gitignored sync-state file is its **only** local write.
2. **Idempotent upsert on `Key`.** Re-running creates nothing twice; matching is on the stored key, never on fuzzy title comparison.
2. **Idempotent upsert on `Key`.** Re-running creates nothing twice; matching is on the stored key, never on fuzzy title comparison. The one fallback, when no row has the key, is an exact match on the `URL` property of a row that has a `Key` (Step 4.2) - still an exact string match, not a fuzzy one - and it rewrites that row's `Key`, so the next run matches on the key again.
3. **Page bodies are write-once.** Property updates keep rows current; bodies belong to the user after creation. Only `--rebuild` may rewrite them, and it says so before doing it.
4. **Never fabricate.** A dead posting URL gets an explicit "unavailable" note, not a reconstruction. Every page claim traces to stored state or fetched content.
5. **Job data only.** The candidate profile, behavioral notes, and evaluation framework never sync - this is a pipeline view, not a profile export.
+24
View File
@@ -80,6 +80,30 @@ class NotionSyncCommandSpec(unittest.TestCase):
self.assertIn("Match, expired-only entry** → update **Status only** to `expired`", step4)
self.assertIn("Never write or rebuild a body for an expired-only entry, including when `--rebuild` is set.", step5)
def test_key_miss_falls_back_to_an_exact_url_match_and_rekeys_the_row(self):
# `job_key.py --audit --migrate` (#637) renames keys in seen_jobs.json, but
# Step 4 upserted on Key alone, so the next sync created a second row for
# every migrated job. On a Key miss, one keyed row with the same URL gets
# the new Key; several are reported and left alone.
text = COMMAND.read_text(encoding="utf-8")
step4 = text.split("## Step 4:", 1)[1].split("## Step 5:", 1)[0]
step6 = text.split("## Step 6:", 1)[1].split("## Important Rules", 1)[0]
rules = text.split("## Important Rules", 1)[1].split("## Adapting", 1)[0]
self.assertIn("**No match on `Key`** → fall back to the posting URL before creating anything", step4)
self.assertIn("whose `URL` equals the job's URL exactly", step4)
self.assertIn("**Exactly one URL match** → re-key it: set its `Key` to the job's key", step4)
self.assertIn("If the row's current `Key` is the key of another job in this sync set, it is not drifted; treat it like a multiple match instead.", step4)
self.assertIn("**More than one URL match** → create nothing and update nothing for this job", step4)
self.assertIn("**No URL match** → if the job is an expired-only entry, skip it", step4)
# Hand-added rows (no Key) stay invisible, fallback included.
self.assertIn("**and** whose `Key` is non-empty", step4)
self.assertIn("invisible to this command, including to the URL fallback", step4)
self.assertIn("<R> re-keyed by URL", step6)
self.assertIn("<A> skipped (ambiguous URL match)", step6)
self.assertIn("never on fuzzy title comparison", rules)
self.assertIn("exact match on the `URL` property", rules)
@unittest.skipUnless(
_HAVE_YAML,
"PyYAML not installed (the CI Python-test job omits it; the lint job runs lint_skills.py directly)",