py/ai-job-search
1
0
Fork 0
mirror of https://github.com/MadsLorentzen/ai-job-search.git synced 2026-09-17 08:36:25 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(apply): factual grounding audit against the union fact base (#185)

Browse Source 
Reviewer-side Factual Grounding Audit: every date, employer, job title, and quantitative metric in both drafts is checked against the union of 01-candidate-profile.md + cv/main_example.tex + CLAUDE.md's Candidate Profile section (grounded if ANY source supports it; inter-source mismatches surfaced as profile-consistency warnings; draft drift returned as Part A edits with reason "grounding"). Drafter-side rule makes those three sources the sole source of truth for facts - existing tailored CVs are structure/phrasing reference only. Fixes the structural drift loop where tailored output fed back as source material; companion to the maintainer-side #178 setup fix.

Reported, diagnosed, and implemented by @jovin-nicholas (#177); the failure mode was validated empirically by the output benchmark (blind judges found the exact escalation class in pre-audit outputs).

Closes #177
This commit is contained in:
Jovin Nicholas
2026-07-19 19:22:46 +02:00
committed by GitHub
parent 8ac6965170
commit f51a766e72
1 changed files with 13 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.claude/commands/apply.md
+13 -5
View File
@@ -64,6 +64,8 @@ Also read the most recent existing CV and cover letter files for concrete struct
- Read any existing `cv/main_*.tex` file as a LaTeX template reference - Read any existing `cv/main_*.tex` file as a LaTeX template reference
- Read any existing `cover_letters/cover_*.tex` or `cover_letters/Cover_*.tex` file as a template reference - Read any existing `cover_letters/cover_*.tex` or `cover_letters/Cover_*.tex` file as a template reference
*The master candidate profile (`01-candidate-profile.md`), the master CV (`cv/main_example.tex`), and CLAUDE.md's Candidate Profile section are the sole source of truth for facts; existing tailored CVs may be read for structure and phrasing only, never as a source of claims.*
### Requirement coverage (both documents) ### Requirement coverage (both documents)
- **Every requirement the posting states gets addressed - matched or honestly gapped, never silently omitted.** A stated requirement the candidate lacks (a tool, a clearance, years of experience) is acknowledged with an honest bridge ("not in my daily toolkit yet; a natural extension of X"), because omission reads as hiding once an interviewer asks. Build the requirement list from Step 1 and check both drafts against it before Step 3. - **Every requirement the posting states gets addressed - matched or honestly gapped, never silently omitted.** A stated requirement the candidate lacks (a tool, a clearance, years of experience) is acknowledged with an honest bridge ("not in my daily toolkit yet; a natural extension of X"), because omission reads as hiding once an interviewer asks. Build the requirement list from Step 1 and check both drafts against it before Step 3.
- **Engage nice-to-haves by name** where the profile supports honest adjacency (e.g. "conceptually aligned with <named tool>"), and use the posting's own term over a synonym wherever it is truthfully applicable - including in CV section headings (a posting hiring for "MLOps" should find a heading containing "MLOps", not only a paraphrase). - **Engage nice-to-haves by name** where the profile supports honest adjacency (e.g. "conceptually aligned with <named tool>"), and use the posting's own term over a synonym wherever it is truthfully applicable - including in CV section headings (a posting hiring for "MLOps" should find a heading containing "MLOps", not only a paraphrase).
@@ -75,6 +77,7 @@ Also read the most recent existing CV and cover letter files for concrete struct
- Tailor the profile statement and experience bullets to the specific role - Tailor the profile statement and experience bullets to the specific role
- Reframe skills and achievements to match job requirements - Reframe skills and achievements to match job requirements
- Keep to 2 pages - Keep to 2 pages
- **Grounding Audit:** Before writing to disk, audit all tailored bullet points against the union of three sources: `.claude/skills/job-application-assistant/01-candidate-profile.md` + the master CV (`cv/main_example.tex`) + `CLAUDE.md`'s Candidate Profile section to verify that all dates, roles, and metrics match exactly (zero profile drift or fabrication).
### Cover Letter (`cover_letters/cover_<company>_<role>.tex`) ### Cover Letter (`cover_letters/cover_<company>_<role>.tex`)
- **Match the language of the job posting** (Danish posting -> Danish cover letter, English posting -> English cover letter) - **Match the language of the job posting** (Danish posting -> Danish cover letter, English posting -> English cover letter)
@@ -111,15 +114,20 @@ Use WebSearch and WebFetch to research, starting **only** from the company ident
- Company culture and values - Company culture and values
### 2. Read Reference Materials (content-critique only) ### 2. Read Reference Materials (content-critique only)
Read these four files — and only these — to ground your critique: Read these reference files — and only these — to ground your critique:
- `.claude/skills/job-application-assistant/01-candidate-profile.md` - `.claude/skills/job-application-assistant/01-candidate-profile.md`
- `.claude/skills/job-application-assistant/02-behavioral-profile.md` — use this specifically to check whether the cover letter's voice matches the candidate's natural register. A "Collaborator" PI profile, for example, should not be given a combative, solo-hero tone; a "Persuader" profile should not be given over-hedged, apologetic phrasing. - `.claude/skills/job-application-assistant/02-behavioral-profile.md` — use this specifically to check whether the cover letter's voice matches the candidate's natural register. A "Collaborator" PI profile, for example, should not be given a combative, solo-hero tone; a "Persuader" profile should not be given over-hedged, apologetic phrasing.
- `.claude/skills/job-application-assistant/03-writing-style.md` - `.claude/skills/job-application-assistant/03-writing-style.md`
- `.claude/skills/job-application-assistant/04-job-evaluation.md` - `.claude/skills/job-application-assistant/04-job-evaluation.md`
- The master CV baseline template (`cv/main_example.tex`)
- The workspace root `CLAUDE.md` file (specifically the Candidate Profile section)
Do NOT read `05-cv-templates.md` or `06-cover-letter-templates.md` — those govern LaTeX structure the drafter already applied and are not needed for content critique. Do NOT read `05-cv-templates.md` or `06-cover-letter-templates.md` — those govern LaTeX structure the drafter already applied and are not needed for content critique.
### 3. Drafts to Review ### 3. Factual Grounding Audit
Compare every date, employer, job title, and quantitative metric in both drafts against the union of three sources: `.claude/skills/job-application-assistant/01-candidate-profile.md` + the master CV baseline template (`cv/main_example.tex`) + `CLAUDE.md`'s Candidate Profile section. A claim is grounded if ANY of these sources supports it. Mismatches between these three sources themselves must be reported to the user as a profile-consistency warning rather than treated as draft drift. Draft mismatches must be flagged as Part A edits with `"reason": "grounding"` so they can be distinguished from style changes. Keep the tolerance honest: reframed emphasis is fine; changed facts and escalated numbers are not.
### 4. Drafts to Review
Both drafts are provided inline below. Do NOT use the Read tool on the draft files — use these exact texts. Both drafts are provided inline below. Do NOT use the Read tool on the draft files — use these exact texts.
<CV_DRAFT file="cv/main_<COMPANY>_<ROLE>.tex"> <CV_DRAFT file="cv/main_<COMPANY>_<ROLE>.tex">
@@ -130,12 +138,12 @@ Both drafts are provided inline below. Do NOT use the Read tool on the draft fil
<INSERT_COVER_LETTER_DRAFT_HERE> <INSERT_COVER_LETTER_DRAFT_HERE>
</COVER_LETTER_DRAFT> </COVER_LETTER_DRAFT>
### 4. Job Posting ### 5. Job Posting
<JOB_POSTING> <JOB_POSTING>
<INSERT_JOB_POSTING_TEXT_HERE> <INSERT_JOB_POSTING_TEXT_HERE>
</JOB_POSTING> </JOB_POSTING>
### 5. Produce Feedback ### 6. Produce Feedback
Return your feedback in **two parts**: Return your feedback in **two parts**:
@@ -146,7 +154,7 @@ A JSON array of concrete edits the drafter can apply directly without re-reading
"file": "cv/main_<COMPANY>_<ROLE>.tex" | "cover_letters/cover_<COMPANY>_<ROLE>.tex", "file": "cv/main_<COMPANY>_<ROLE>.tex" | "cover_letters/cover_<COMPANY>_<ROLE>.tex",
"old_string": "<exact text currently in the draft>", "old_string": "<exact text currently in the draft>",
"new_string": "<replacement text>", "new_string": "<replacement text>",
"reason": "<one-line rationale: keyword match / company angle / reframing / style>" "reason": "<one-line rationale: keyword match / company angle / reframing / style / grounding>"
} }
``` ```
Only use this format when you can quote the exact `old_string` from the drafts above. Make `old_string` unique — include enough surrounding context so it matches exactly once per file. Only use this format when you can quote the exact `old_string` from the drafts above. Make `old_string` unique — include enough surrounding context so it matches exactly once per file.