fix(notion-sync): keep expired-only jobs update-only (#666)

* fix(notion-sync): include expired entries in sync set as update-only (#597)

* changelog: place the #666 entry under [Unreleased]

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Mads Lorentzen <madslorentzen_17@hotmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
authored and GitHub committed 2026-10-19 20:09:23 +02:00
1 parent e7778f58ed
commit 58f04bc463
3 files changed
+38 -10

No files matched your search

+13 -10
View File
@@ -41,9 +41,10 @@ Validate the cheap, local precondition before creating anything external. A run
1. Read `job_scraper/seen_jobs.json` and `job_search_tracker.csv` (either may be missing). 1. Read `job_scraper/seen_jobs.json` and `job_search_tracker.csv` (either may be missing).
2. Select `job_scraper/seen_jobs.json` entries with status `ranked` whose `rank_score` meets the threshold from Step 0. `--all` lifts the threshold entirely. 2. Select `job_scraper/seen_jobs.json` entries with status `ranked` whose `rank_score` meets the threshold from Step 0. `--all` lifts the threshold entirely.
3. Every tracker row joins the sync set (an applied-to job always syncs, ranked or not), matched to `job_scraper/seen_jobs.json` entries case-insensitively on company + title where possible (the tracker's `role` column against the entry's `title` field — entries carry `title`, not `role`). Tracker rows with no `job_scraper/seen_jobs.json` entry sync too - build their Key as `<company>_<role>` lowercased with underscores. 3. Every tracker row joins the sync set (an applied-to job always syncs, ranked or not), matched to `job_scraper/seen_jobs.json` entries case-insensitively on company + title where possible (the tracker's `role` column against the entry's `title` field — entries carry `title`, not `role`). Tracker rows with no `job_scraper/seen_jobs.json` entry sync too - build their Key as `<company>_<role>` lowercased with underscores.
4. **Status precedence:** the tracker wins. A job that is `ranked` in `job_scraper/seen_jobs.json` but `interview` in the tracker syncs as `interview`. Jobs only in `job_scraper/seen_jobs.json` keep their stored status. **Deadline precedence: the tracker wins too** - the tracker's `deadline` (written by `/apply` from the posting the application was actually built on) overrides the `job_scraper/seen_jobs.json` value; jobs only in `job_scraper/seen_jobs.json` keep the scraper's stored deadline. Omit the property when neither states one, and **never reconcile the two by picking the earlier or later date** - both were read from the posting at different times, and the safe-looking `min()` substitutes a date the user never applied against. 4. `job_scraper/seen_jobs.json` entries with status `expired` join the sync set regardless of score as update-only candidates. An entry selected only because it is `expired` and has no matching tracker row can update an existing page, but never creates a page or writes a body. If a matching tracker row exists, it remains an ordinary sync item and tracker status precedence applies.
5. **If the sync set is empty** (no ranked entries meet the threshold and there are no tracker rows), say "Nothing to sync - run `/scrape` and `/rank` first" (or, when jobs exist but all score below the threshold, say so and suggest `--min-score`/`--all`) and **stop**. 5. **Status precedence:** the tracker wins. A job that is `ranked` in `job_scraper/seen_jobs.json` but `interview` in the tracker syncs as `interview`. Jobs only in `job_scraper/seen_jobs.json` keep their stored status. **Deadline precedence: the tracker wins too** - the tracker's `deadline` (written by `/apply` from the posting the application was actually built on) overrides the `job_scraper/seen_jobs.json` value; jobs only in `job_scraper/seen_jobs.json` keep the scraper's stored deadline. Omit the property when neither states one, and **never reconcile the two by picking the earlier or later date** - both were read from the posting at different times, and the safe-looking `min()` substitutes a date the user never applied against.
6. State the counts before touching the destination: how many rows will be created or checked, and the threshold in effect. 6. **If the sync set is empty** (no ranked entries meet the threshold and there are no tracker rows), say "Nothing to sync - run `/scrape` and `/rank` first" (or, when jobs exist but all score below the threshold, say so and suggest `--min-score`/`--all`) and **stop**.
7. State the counts before touching the destination: how many rows will be created or checked, and the threshold in effect.
--- ---
@@ -54,7 +55,7 @@ Validate the cheap, local precondition before creating anything external. A run
{ "database_id": "...", "database_url": "...", "last_sync": "YYYY-MM-DD" } { "database_id": "...", "database_url": "...", "last_sync": "YYYY-MM-DD" }
``` ```
2. If it exists, verify the database id still resolves in Notion. If the database was deleted, treat this as a first run. 2. If it exists, verify the database id still resolves in Notion. If the database was deleted, treat this as a first run.
3. **First run:** search the workspace for a database named "Job Search Pipeline". If none exists, ask the user where to create it (top-level page or an existing page they name), then create it with exactly these properties: 3. **First run:** search the workspace for a database named "Job Search Pipeline". If none exists and the sync set contains only expired-only entries (`job_scraper/seen_jobs.json` entries with status `expired` and no matching tracker row), stop without asking where to create a database and without creating the database or writing `job_scraper/notion_sync.json`: there are no existing pages to update. Otherwise, if no database exists, ask the user where to create it (top-level page or an existing page they name), then create it with exactly these properties:
| Property | Type | Values / notes | | Property | Type | Values / notes |
|----------|------|----------------| |----------|------|----------------|
@@ -86,9 +87,10 @@ Validate the cheap, local precondition before creating anything external. A run
For each job in the sync set: For each job in the sync set:
1. Query the database for a page whose `Key` equals the job's key. 1. Query the database for a page whose `Key` equals the job's key.
2. **No match** → create the page with all properties from the Step 3 table, then write its body (Step 5). 2. **No match** → if the job is an expired-only entry, skip it - do not create a page or write a body. Otherwise, create the page with all properties from the Step 3 table, then write its body (Step 5).
3. **Match** → update **properties only**: Status, Score, Verdict, Deadline, Ranked, Applied on, Channel, CV file, Cover letter. Properties are the always-current surface (bodies are write-once), so tracker updates recorded by `/outcome` reach the destination exclusively through them. Do not touch the page body - the user may have added their own notes there, and clobbering them breaks trust in the whole view. (`--rebuild` is the sole exception.) 3. **Match, expired-only entry** → update **Status only** to `expired`. Do not update other properties or touch the page body.
4. Never delete or archive pages, even for jobs that turned `expired` - set Status to `expired` instead. Rows the user added to the database by hand (no `Key` value) are invisible to this command. 4. **Match, any other effective status** → update **properties only**: Status, Score, Verdict, Deadline, Ranked, Applied on, Channel, CV file, Cover letter. Properties are the always-current surface (bodies are write-once), so tracker updates recorded by `/outcome` reach the destination exclusively through them. Do not touch the page body - the user may have added their own notes there, and clobbering them breaks trust in the whole view. (`--rebuild` is the sole exception for non-expired jobs.)
5. Never delete or archive pages, even for jobs that turned `expired` - set Status to `expired` instead. Rows the user added to the database by hand (no `Key` value) are invisible to this command.
**Normalise the Status value before writing.** The tracker may hold legacy space spellings (`no response`, `offer declined`) from before the canonical forms were locked. Map them to `no_response` / `offer_declined` per the **Tracker status vocabulary** in `/outcome` before setting Status on create or update - never push a space form to Notion, which would auto-create a separate select option per unique string. Pre-existing space-form options in an existing database simply go unused; Notion never auto-removes select options. **Normalise the Status value before writing.** The tracker may hold legacy space spellings (`no response`, `offer declined`) from before the canonical forms were locked. Map them to `no_response` / `offer_declined` per the **Tracker status vocabulary** in `/outcome` before setting Status on create or update - never push a space form to Notion, which would auto-create a separate select option per unique string. Pre-existing space-form options in an existing database simply go unused; Notion never auto-removes select options.
@@ -100,9 +102,10 @@ Batch politely: if the MCP server rate-limits, back off and continue; report any
The page body is what makes a row worth clicking. Build it **only from stored data and actually fetched content**: The page body is what makes a row worth clicking. Build it **only from stored data and actually fetched content**:
1. **Fit summary** - a short section from `job_scraper/seen_jobs.json` fields: score, verdict, quick-fit level, first-seen and ranked dates. If the job is in the tracker, add the application timeline (date applied, channel, current status, dated notes from the `notes` column) and name the submitted documents from `cv_file`/`cover_letter_file` (filenames only - the documents themselves never sync). **When the status is `drafted`, write "drafted YYYY-MM-DD, not yet submitted" instead of a date applied, and call the files drafts rather than submitted documents** (page bodies are write-once - Step 4.3). 1. Never write or rebuild a body for an expired-only entry, including when `--rebuild` is set. Step 4 skips an expired-only entry when no matching page exists.
2. **The posting** - WebFetch the job URL and write a readable digest: what the role is, key requirements, practical details (location, deadline, salary if stated). Retry a 403 with browser headers per `.claude/skills/job-application-assistant/09-web-research.md` first. If the fetch still fails or redirects to a listing page, write "Posting no longer available (checked YYYY-MM-DD)" - **never reconstruct a posting from memory**. 2. **Fit summary** - a short section from `job_scraper/seen_jobs.json` fields: score, verdict, quick-fit level, first-seen and ranked dates. If the job is in the tracker, add the application timeline (date applied, channel, current status, dated notes from the `notes` column) and name the submitted documents from `cv_file`/`cover_letter_file` (filenames only - the documents themselves never sync). **When the status is `drafted`, write "drafted YYYY-MM-DD, not yet submitted" instead of a date applied, and call the files drafts rather than submitted documents** (page bodies are write-once - Step 4.4).
3. **Links** - the posting URL; derive `<company>_<role>` by the **Subfolder naming** rule in `documents/README.md`, and if that archive exists locally, name its path (plain text - the destination cannot link into the filesystem). 3. **The posting** - WebFetch the job URL and write a readable digest: what the role is, key requirements, practical details (location, deadline, salary if stated). Retry a 403 with browser headers per `.claude/skills/job-application-assistant/09-web-research.md` first. If the fetch still fails or redirects to a listing page, write "Posting no longer available (checked YYYY-MM-DD)" - **never reconstruct a posting from memory**.
4. **Links** - the posting URL; derive `<company>_<role>` by the **Subfolder naming** rule in `documents/README.md`, and if that archive exists locally, name its path (plain text - the destination cannot link into the filesystem).
Keep the page under ~40 blocks; this is a briefing, not a mirror of the posting. Keep the page under ~40 blocks; this is a briefing, not a mirror of the posting.
+8
View File
@@ -179,6 +179,14 @@ per-file diff commands.
Poppler installed. It now probes each distinct `pdftotext` on PATH with `-h`, runs the first that Poppler installed. It now probes each distinct `pdftotext` on PATH with `-h`, runs the first that
lists `-bbox` by its full path, and keeps the `skipped:` exit 2 for when none does. lists `-bbox` by its full path, and keeps the `skipped:` exit 2 for when none does.
- **`/notion-sync` can set a page's Status to `expired`** (#666, fixes #597) (`.claude/commands/notion-sync.md` Steps 2-5,
`tests/test_notion_sync_command.py`) - Step 2 built the sync set from `ranked` entries and tracker rows only, so a job
synced while ranked and later retired by `/rank`'s sweep never re-entered the set and its page stayed `ranked`, although
Step 4 says to set Status to `expired`. `expired` entries now join the sync set as update-only candidates: a matching page
gets Status `expired` and nothing else, a Key with no page is skipped rather than created, no body is written for them even
under `--rebuild`, and a first run with only such entries stops before creating the database. An expired posting with a
tracker row still syncs as an ordinary row.
## [1.7.3] - 2026-10-19 ## [1.7.3] - 2026-10-19
### Added ### Added
+17
View File
@@ -63,6 +63,23 @@ class NotionSyncCommandSpec(unittest.TestCase):
"spec lost the rule that CV/cover-letter content never syncs to Notion", "spec lost the rule that CV/cover-letter content never syncs to Notion",
) )
def test_expired_jobs_are_update_only_and_never_create_or_rebuild_pages(self):
text = COMMAND.read_text(encoding="utf-8")
step2 = text.split("## Step 2:", 1)[1].split("## Step 3:", 1)[0]
step3 = text.split("## Step 3:", 1)[1].split("## Step 4:", 1)[0]
step4 = text.split("## Step 4:", 1)[1].split("## Step 5:", 1)[0]
step5 = text.split("## Step 5:", 1)[1].split("## Step 6:", 1)[0]
self.assertIn("entries with status `expired` join the sync set regardless of score as update-only candidates", step2)
self.assertIn("An entry selected only because it is `expired` and has no matching tracker row", step2)
self.assertIn("If a matching tracker row exists, it remains an ordinary sync item and tracker status precedence applies.", step2)
self.assertIn("sync set contains only expired-only entries", step3)
self.assertIn("stop without asking where to create a database", step3)
self.assertIn("without creating the database or writing `job_scraper/notion_sync.json`", step3)
self.assertIn("if the job is an expired-only entry, skip it - do not create a page or write a body", step4)
self.assertIn("Match, expired-only entry** → update **Status only** to `expired`", step4)
self.assertIn("Never write or rebuild a body for an expired-only entry, including when `--rebuild` is set.", step5)
@unittest.skipUnless( @unittest.skipUnless(
_HAVE_YAML, _HAVE_YAML,
"PyYAML not installed (the CI Python-test job omits it; the lint job runs lint_skills.py directly)", "PyYAML not installed (the CI Python-test job omits it; the lint job runs lint_skills.py directly)",