diff --git a/.claude/commands/notion-sync.md b/.claude/commands/notion-sync.md index 36e50d2e..86c14d4d 100644 --- a/.claude/commands/notion-sync.md +++ b/.claude/commands/notion-sync.md @@ -40,7 +40,7 @@ Validate the cheap, local precondition before creating anything external. A run 1. Read `job_scraper/seen_jobs.json` and `job_search_tracker.csv` (either may be missing). 2. Select `seen_jobs.json` entries with status `ranked` whose `rank_score` meets the threshold from Step 0. `--all` lifts the threshold entirely. -3. Every tracker row joins the sync set (an applied-to job always syncs, ranked or not), matched to `seen_jobs.json` entries case-insensitively on company + role where possible. Tracker rows with no `seen_jobs.json` entry sync too - build their Key as `_` lowercased with underscores. +3. Every tracker row joins the sync set (an applied-to job always syncs, ranked or not), matched to `seen_jobs.json` entries case-insensitively on company + title where possible (the tracker's `role` column against the entry's `title` field — entries carry `title`, not `role`). Tracker rows with no `seen_jobs.json` entry sync too - build their Key as `_` lowercased with underscores. 4. **Status precedence:** the tracker wins. A job that is `ranked` in `seen_jobs.json` but `interview` in the tracker syncs as `interview`. Jobs only in `seen_jobs.json` keep their stored status. **Deadline precedence: the tracker wins too** - the tracker's `deadline` (written by `/apply` from the posting the application was actually built on) overrides the `seen_jobs.json` value; jobs only in `seen_jobs.json` keep the scraper's stored deadline. Omit the property when neither states one, and **never reconcile the two by picking the earlier or later date** - both were read from the posting at different times, and the safe-looking `min()` substitutes a date the user never applied against. 5. **If the sync set is empty** (no ranked entries meet the threshold and there are no tracker rows), say "Nothing to sync - run `/scrape` and `/rank` first" (or, when jobs exist but all score below the threshold, say so and suggest `--min-score`/`--all`) and **stop**. 6. State the counts before touching the destination: how many rows will be created or checked, and the threshold in effect. diff --git a/.claude/skills/upskill/SKILL.md b/.claude/skills/upskill/SKILL.md index 57b58220..db43c7ef 100644 --- a/.claude/skills/upskill/SKILL.md +++ b/.claude/skills/upskill/SKILL.md @@ -54,7 +54,7 @@ Extract required and preferred technical skills from each job source: ### Aggregate mode This mode now merges two sources — tracker rows (Step 2.1) and ranked postings from `seen_jobs.json` (Step 2.3) — so the same job is never double-counted and recorded gaps are preferred over inferred ones: -1. **Dedupe.** Match tracker rows against ranked entries on case-insensitive company + role (casefold + strip on both fields) — the same match `/notion-sync`'s Step 2 describes. A job present in both counts once. +1. **Dedupe.** Match tracker rows against ranked entries on case-insensitive company + title: the tracker's `role` column against the entry's `title` field (`seen_jobs.json` entries carry `title`, not `role`), casefold + strip on both sides — the same match `/notion-sync`'s Step 2 describes and `tools/rank_state.py` implements for the tracker exclusion. A job present in both counts once. 2. **Recorded gaps beat inferred skills.** For any job that has a recorded `gaps` array (from a ranked entry, or from a tracker row that matched one), use those gap bullets directly as the skill list for that job instead of inferring from `role`/`sector`/`notes`. For a ranked-only job with no `gaps` (already skipped and counted in Step 2.3) or a tracker-only row, fall back to inferring likely required skills from `role`, `sector`, and `notes` — optionally WebFetch the row's `source` URL for more detail, but skip if the URL is missing or dead. 3. **One weight per job**, both 0–100 on the same scale: `(100 - fit_rating) / 100` for tracker rows, `(100 - rank_score) / 100` for ranked-only rows. If a job is in both (Step 3.1 matched it), prefer the tracker's numeric `fit_rating` for the weight. A **blank or non-numeric `fit_rating`** (rows `/outcome` creates for applications made outside the workflow never got a fit evaluation) contributes no weight: fall back to a matched ranked entry's `rank_score` when Step 3.1 found one, otherwise skip the row, count it, and report the count once in the terminal — the same treatment Step 2.3 gives a missing `gaps` field, and for the same reason. Never treat a blank as 0: that reads as weight 1.0, the maximum, and lets the one job the framework knows nothing about dominate the heatmap. 4. **Score.** Build a **skill frequency map**: for each extracted skill (recorded gap bullet or inferred skill), count how many jobs mention it, then multiply each job's contribution by its weight from Step 3.3. Track whether each contribution came from a recorded gap or an inferred one, for Step 5's provenance column. diff --git a/CHANGELOG.md b/CHANGELOG.md index a8b9d8ff..90610806 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -68,6 +68,18 @@ per-file diff commands. ### Fixed +- **`/upskill` and `/notion-sync` now name the field ranked entries actually carry when joining + them to tracker rows** (`.claude/skills/upskill/SKILL.md` Step 3, `.claude/commands/notion-sync.md` + Step 2, `tests/test_upskill_skill.py`, `tests/test_notion_sync_command.py`) - both said to match + "on company + role" with "casefold + strip on both fields", but a `seen_jobs.json` entry has + `title` and no `role` key (the job-scraper schema), so read literally the lookup never matched: + `/upskill` then weighted the same job twice, once as a tracker row and once as a ranked entry, + the double-count its dedupe step exists to prevent, and `/notion-sync` split one job into two + rows with different Keys. `tools/rank_state.py` already does the cross-name join (tracker + `role` against entry `title`); both specs now say so. Two derived tests read the entry schema + and require the join wording to name `title` for the entry and `role` for the tracker; they + fail on master. + - **`/reset profile` keeps the `### Languages` table header under Identity** (`.claude/commands/reset.md` Step 3, `tests/test_reset_command.py`) - `04-job-evaluation.md`'s Language Gate and `job-scraper/search-queries.md`'s query-language generation both read "your diff --git a/tests/test_notion_sync_command.py b/tests/test_notion_sync_command.py index 0f48892a..c49aea25 100644 --- a/tests/test_notion_sync_command.py +++ b/tests/test_notion_sync_command.py @@ -43,6 +43,18 @@ class NotionSyncCommandSpec(unittest.TestCase): "notion_sync.json is personal state and must never be committable", ) + def test_step2_join_names_the_field_ranked_entries_actually_carry(self): + # seen_jobs.json entries carry `title` (job-scraper SKILL.md schema), the + # tracker carries `role`. "matched ... on company + role" read as a + # lookup of `role` on the entry, which never matches, so one job became + # two Notion rows with different Keys. The join is role-vs-title. + text = COMMAND.read_text(encoding="utf-8") + line = next(l for l in text.splitlines() if "matched to `seen_jobs.json` entries" in l) + self.assertIn("company + title", line) + self.assertIn("`title`", line) + self.assertIn("`role`", line) + self.assertNotIn("company + role", line) + def test_privacy_rule_documents_never_sync(self): text = COMMAND.read_text(encoding="utf-8") self.assertIn( diff --git a/tests/test_upskill_skill.py b/tests/test_upskill_skill.py index 62eaa46a..7227ab89 100644 --- a/tests/test_upskill_skill.py +++ b/tests/test_upskill_skill.py @@ -19,6 +19,7 @@ except ImportError: REPO = Path(__file__).resolve().parent.parent SKILL = REPO / ".claude" / "skills" / "upskill" / "SKILL.md" +SCRAPER = REPO / ".claude" / "skills" / "job-scraper" / "SKILL.md" def _sections(text: str) -> dict[str, str]: @@ -68,7 +69,7 @@ class UpskillSkillSpec(unittest.TestCase): def test_step3_documents_dedupe_and_gap_precedence(self): sections = _sections(SKILL.read_text(encoding="utf-8")) step3 = sections.get("Step 3: Pass 1 — Hard Skill Diff", "") - self.assertIn("case-insensitive company + role", step3, "Step 3 must specify the dedupe key") + self.assertIn("case-insensitive company + title", step3, "Step 3 must specify the dedupe key") self.assertIn( "/notion-sync", step3, @@ -82,6 +83,24 @@ class UpskillSkillSpec(unittest.TestCase): self.assertIn("(100 - fit_rating) / 100", step3) self.assertIn("(100 - rank_score) / 100", step3) + def test_step3_dedupe_names_the_field_ranked_entries_actually_carry(self): + # The tracker has a `role` column; a seen_jobs.json entry has `title` + # (job-scraper SKILL.md schema) and no `role` key at all. Step 3 told + # the reader to match on "company + role" with "casefold + strip on + # both fields", which reads as a literal lookup of `role` on the entry: + # that never matches, so the dedupe silently failed and Step 3.2's + # "one weight per job" counted the same job twice. The join is + # role-vs-title, as tools/rank_state.py implements it. + schema = SCRAPER.read_text(encoding="utf-8") + schema_block = schema[schema.index('""'): schema.index("```", schema.index('""'))] + self.assertIn('"title"', schema_block) + self.assertNotIn('"role"', schema_block, "if entries gain a role key, revisit the join wording") + step3 = _sections(SKILL.read_text(encoding="utf-8")).get("Step 3: Pass 1 — Hard Skill Diff", "") + dedupe = step3[step3.index("**Dedupe.**"): step3.index("2. **Recorded gaps")] + self.assertIn("`title`", dedupe, "the dedupe must name the entry field, title") + self.assertIn("`role`", dedupe, "the dedupe must name the tracker column, role") + self.assertNotIn("company + role", dedupe, "entries have no role field to match on") + def test_step3_handles_blank_fit_rating(self): """/outcome creates tracker rows for applications made outside the workflow, and no rule anywhere fills fit_rating on that path - yet