mirror of
https://github.com/MadsLorentzen/ai-job-search.git
synced 2026-09-17 08:36:25 +00:00
* fix(gmail-sync): strip CSV-breaking characters from the email subject Step 7a interpolated the raw subject line of a received email into the `notes` column of job_search_tracker.csv. No writer here emits a quoted tracker field, so an unescaped comma splits the row - for csv.DictReader just as much as for a naive split, which matters because tools/rank_state.py is the repo's only machine reader and uses exactly that. `notes` is column 10 of 14, so a subject as ordinary as "Re: Your application, Data Scientist" shifted cv_file, cover_letter_file and source a column left. A line break is worse: it ends the row and starts a second one. The rule now sits on the append instruction itself rather than in a general note a writer can miss. The subject survives verbatim in the archive's outcome.md, which is Markdown and carries no such constraint. /outcome Step 4 (outcome.md:195) is also free-form and has the same exposure, but its text is model-authored in a turn the user is watching rather than copied from third-party mail unattended. Left out deliberately, to be filed separately. * fix(outcome): keep the Step 4 tracker note free of CSV-breaking characters /outcome Step 4 appended "a short dated note" to `notes` with no constraint on its content, the same exposure /gmail-sync Step 7a had: nothing quotes a tracker field, so `rejected, no feedback given` shifts cv_file, cover_letter_file and source a column left under csv.DictReader, and a line break ends the row. The append instruction now requires a note with no commas, double quotes or line breaks. Folded in at the maintainer's request on #455 so one entry and one rule cover both free-form writers. The CSV-safety tests move out of test_gmail_sync_command.py into test_tracker_notes_csv_safe.py, where a CASES table pins the rule on each writer's append line.
This commit is contained in:
Jakob Stender Guldberg
@@ -124,7 +124,9 @@ Approving the whole batch in one reply is expected UX - the requirement is that
|
||||
|
||||
For every row the user approved:
|
||||
|
||||
1. **Tracker (`job_search_tracker.csv`):** update the matched row's `status` column per the Step 5 table, and append to `notes`: `<date> gmail-sync: <signal> ("<email subject>")`. Never restructure the CSV, reorder rows, or touch unrelated rows - same rule `/outcome` follows. The rewrite touches only `status`, `notes` (and `date` when the drafted-rule below fires): preserve every other field of the row, parsed or not, so the `deadline` column written by `/apply` Step 6b - or any column added in the future - is never blanked by a status sync.
|
||||
1. **Tracker (`job_search_tracker.csv`):** update the matched row's `status` column per the Step 5 table, and append to `notes`: `<date> gmail-sync: <signal> ("<email subject>")`, **with every comma, double quote and line break deleted from the subject first**. No writer here emits a quoted tracker field and no reader unquotes one, so an unescaped comma splits the row identically for a naive split and for the `csv.DictReader` the shipped reader actually uses (`tools/rank_state.py`): `cv_file`, `cover_letter_file` and `source` each shift a column left. A line break is worse - it ends the row and starts a second one. The double quote is stripped as cheap insurance for the day something does quote a field; on today's readers it is harmless. The subject is a human-readable breadcrumb here, not data anything reads back - item 2 below keeps it verbatim in `outcome.md`, which is Markdown and carries no such constraint. This matters more than it looks: `/gmail-sync` is the only tracker writer that copies *third-party* text, and the only one that runs unattended, so nobody is watching the row it edits.
|
||||
|
||||
Never restructure the CSV, reorder rows, or touch unrelated rows - same rule `/outcome` follows. The rewrite touches only `status`, `notes` (and `date` when the drafted-rule below fires): preserve every other field of the row, parsed or not, so the `deadline` column written by `/apply` Step 6b - or any column added in the future - is never blanked by a status sync.
|
||||
|
||||
**If the matched row was still `drafted`,** also set `date` to the email's date. The employer replying proves the user submitted by hand without running `/outcome`, so the drafting date now in that column is wrong. The email's date is an upper bound on the real submission date, tight for an ack and loose for a rejection weeks later, which is why Step 6 shows it and lets the user supply the actual date instead.
|
||||
2. **`outcome.md`:** tick the relevant stage checkbox (adding the date in parentheses) or update `Status`/`Date resolved` per the table. Append a dated entry to `## Notes`, never overwrite existing Notes history:
|
||||
|
||||
@@ -192,7 +192,7 @@ Update rules: tick stage checkboxes as they are reached (add the date in parenth
|
||||
|
||||
## Step 4: Update the Tracker
|
||||
|
||||
Update the matched row's `status` column using the canonical spellings from **Tracker status vocabulary** above (e.g. `drafted` → `applied` → `interview` → `offer` → `hired` / `rejected` / `no_response` / `offer_declined` / `withdrawn`) and append a short dated note to the `notes` column. Never restructure the CSV, reorder rows, or touch other rows. The rewrite touches only the `status` and `notes` columns: preserve every other field of the row, parsed or not, so a value the row carries - the `deadline` written by `/apply` Step 6b, or any column added in the future - is never blanked by a status update.
|
||||
Update the matched row's `status` column using the canonical spellings from **Tracker status vocabulary** above (e.g. `drafted` → `applied` → `interview` → `offer` → `hired` / `rejected` / `no_response` / `offer_declined` / `withdrawn`) and append a short dated note to the `notes` column, **containing no commas, double quotes or line breaks**. No writer here emits a quoted tracker field, so a comma in the note shifts `cv_file`, `cover_letter_file` and `source` a column left for `csv.DictReader` (`tools/rank_state.py`) as much as for a naive split, and a line break ends the row - `rejected, no feedback given` is exactly the sentence that corrupts it; write `rejected - no feedback given`. `/gmail-sync` Step 7a applies the same rule to the email subjects it appends. Never restructure the CSV, reorder rows, or touch other rows. The rewrite touches only the `status` and `notes` columns: preserve every other field of the row, parsed or not, so a value the row carries - the `deadline` written by `/apply` Step 6b, or any column added in the future - is never blanked by a status update.
|
||||
|
||||
**Moving a row off `drafted`:** rows written by `/apply` Step 6b carry the date the documents were drafted, not the date they were sent. Whenever this step advances such a row to any other status - `applied`, or straight to `interview` or `rejected` when the user reports an outcome for something they submitted without recording it - overwrite its `date` column with the actual submission date. The `date` column is read as "applied on" by `/notion-sync` and drives `/html-report`'s year/season grouping and this command's own days-quiet count, so leaving the draft date in place would misreport the application.
|
||||
|
||||
|
||||
@@ -118,6 +118,26 @@ per-file diff commands.
|
||||
wrappers, plus a new `detail-backoff.test.ts` that exercises the handler path itself -
|
||||
its retry cases fail against the bare `fetch()`.
|
||||
|
||||
- **Free-form tracker notes no longer break the CSV row** (#454) (`.claude/commands/gmail-sync.md`,
|
||||
`.claude/commands/outcome.md`, `tests/test_tracker_notes_csv_safe.py`) - two writers put
|
||||
free-form text into the `notes` column of `job_search_tracker.csv`: `/gmail-sync` Step 7a
|
||||
copied the raw subject of a received email, and `/outcome` Step 4 appended "a short dated
|
||||
note" with no constraint on its content. No writer in the framework emits a quoted tracker
|
||||
field, so an unescaped comma splits the row for a naive split and for `csv.DictReader` alike -
|
||||
the latter being what the repo's only machine reader of the tracker uses
|
||||
(`tools/rank_state.py`). `notes` is column 10 of 14, so a subject as ordinary as
|
||||
`Re: Your application, Data Scientist`, or a note as natural as `rejected, no feedback given`,
|
||||
shifted `cv_file`, `cover_letter_file` and `source` a column left. A line break is worse: it
|
||||
ends the row and starts a second one. Nothing validated the row afterwards, and the
|
||||
`/gmail-sync` half was written unattended, so the corruption was silent. Both append
|
||||
instructions now carry the rule themselves - `/gmail-sync` deletes commas, double quotes and
|
||||
line breaks from the subject, `/outcome` writes its note without them - rather than a general
|
||||
note a writer can miss. Nothing is lost on the `/gmail-sync` side: Step 7a item 2 still
|
||||
records the subject verbatim in the archive's `outcome.md`, which is Markdown and carries no
|
||||
such constraint. The fixed-format writers (`followed up YYYY-MM-DD`,
|
||||
`stale resolved no_response (YYYY-MM-DD)`, `redrafted`) could never contain these characters
|
||||
and are unchanged.
|
||||
|
||||
- **`jobindex-search detail` no longer fetches arbitrary URLs or invents posting-shaped
|
||||
output** (#447) - the command fetched any `http(s)` input verbatim (no host check) and,
|
||||
when the path didn't match its one pattern, silently used the whole input URL as the job
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
"""Guards for the rule that keeps free-form `notes` from breaking a tracker row.
|
||||
|
||||
No writer in this framework emits a quoted tracker field, so a comma inside
|
||||
`notes` splits the row - for the `csv.DictReader` in `tools/rank_state.py` as
|
||||
much as for a naive split - and shifts `cv_file`, `cover_letter_file` and
|
||||
`source` a column left. A line break is worse: it ends the row. Two writers put
|
||||
free-form text into `notes`: `/gmail-sync` Step 7a copies an email subject, and
|
||||
`/outcome` Step 4 writes a short note of its own. The fixed-format writers
|
||||
(`followed up YYYY-MM-DD`, `stale resolved no_response (YYYY-MM-DD)`,
|
||||
`redrafted`) cannot contain the characters and are not listed.
|
||||
|
||||
The spec IS the implementation, so the guard is `CASES`: each rule must sit on
|
||||
the line that instructs the append, not merely somewhere in the section. The
|
||||
shape tests below it parse with `csv.DictReader` and document why the rule
|
||||
exists; they pass on master too.
|
||||
"""
|
||||
import csv
|
||||
import io
|
||||
import re
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
REPO = Path(__file__).resolve().parent.parent
|
||||
COMMANDS = REPO / ".claude" / "commands"
|
||||
GMAIL_SYNC = COMMANDS / "gmail-sync.md"
|
||||
OUTCOME = COMMANDS / "outcome.md"
|
||||
|
||||
TRACKER_HEADER = (
|
||||
"date,company,sector,role,role_type,channel,status,contact_person,"
|
||||
"fit_rating,notes,cv_file,cover_letter_file,source,deadline"
|
||||
)
|
||||
|
||||
|
||||
def section(path, heading):
|
||||
"""The body of one markdown section, up to the next heading of any depth."""
|
||||
text = path.read_text(encoding="utf-8")
|
||||
start = text.index(heading) + len(heading)
|
||||
rest = text[start:]
|
||||
end = re.search(r"^#{1,4} ", rest, re.MULTILINE)
|
||||
return rest[: end.start()] if end else rest
|
||||
|
||||
|
||||
class FreeFormNotesWritersStateTheRule(unittest.TestCase):
|
||||
"""Format: (path, heading, line_anchor, rule, why)"""
|
||||
|
||||
CASES = [
|
||||
(
|
||||
GMAIL_SYNC,
|
||||
"### Step 7a: Write Approved Updates",
|
||||
"append to `notes`",
|
||||
"with every comma, double quote and line break deleted from the subject first",
|
||||
"Step 7a item 2 deliberately keeps the subject verbatim in `outcome.md`, "
|
||||
"so the rule must sit on the tracker append, not anywhere in the step",
|
||||
),
|
||||
(
|
||||
OUTCOME,
|
||||
"## Step 4: Update the Tracker",
|
||||
"append a short dated note",
|
||||
"containing no commas, double quotes or line breaks",
|
||||
"Step 4 is the primary status-update path and its note is written "
|
||||
"free-form - `rejected, no feedback given` is the natural sentence",
|
||||
),
|
||||
]
|
||||
|
||||
def test_rule_is_stated_where_the_append_happens(self):
|
||||
for path, heading, anchor, rule, why in self.CASES:
|
||||
with self.subTest(path=path.name, heading=heading):
|
||||
lines = [l for l in section(path, heading).splitlines() if anchor in l]
|
||||
self.assertEqual(len(lines), 1, f"expected one append instruction: {why}")
|
||||
self.assertIn(rule, lines[0], why)
|
||||
|
||||
|
||||
class NotesShapeUnderTheShippedReader(unittest.TestCase):
|
||||
"""Why the rule exists, demonstrated with the reader the repo ships."""
|
||||
|
||||
SUBJECT = 'Re: Your application, Data Scientist - "next steps"'
|
||||
|
||||
def test_sanitised_note_keeps_the_row_parseable(self):
|
||||
safe = self.SUBJECT.replace(",", "").replace('"', "")
|
||||
rows = self._parse(f'2026-09-12 gmail-sync: acknowledged ("{safe}")')
|
||||
|
||||
self.assertEqual(len(rows), 1, "the note must not end the row early")
|
||||
row = rows[0]
|
||||
self.assertIsNone(row.get(None), "the row must be no wider than the header")
|
||||
self.assertEqual(row["cv_file"], "cv/main_acme_data_scientist.tex")
|
||||
self.assertEqual(
|
||||
row["cover_letter_file"], "cover_letters/cover_acme_data_scientist.tex"
|
||||
)
|
||||
self.assertEqual(row["source"], "linkedin")
|
||||
|
||||
def test_a_comma_in_the_note_shifts_the_columns(self):
|
||||
for note in (
|
||||
f'2026-09-12 gmail-sync: acknowledged ("{self.SUBJECT}")',
|
||||
"2026-09-12 rejected, no feedback given",
|
||||
):
|
||||
with self.subTest(note=note):
|
||||
row = self._parse(note)[0]
|
||||
self.assertIsNotNone(row.get(None), "the comma must widen the row")
|
||||
self.assertNotEqual(row["cv_file"], "cv/main_acme_data_scientist.tex")
|
||||
self.assertNotEqual(row["source"], "linkedin")
|
||||
|
||||
def test_a_line_break_in_the_note_splits_the_row_in_two(self):
|
||||
rows = self._parse('2026-09-12 gmail-sync: acknowledged ("Re: update\nlater")')
|
||||
self.assertEqual(len(rows), 2)
|
||||
self.assertIsNone(rows[0]["cv_file"], "the first row ends mid-note")
|
||||
|
||||
@classmethod
|
||||
def _parse(cls, notes):
|
||||
stream = io.StringIO(TRACKER_HEADER + "\n" + cls._row(notes) + "\n")
|
||||
return list(csv.DictReader(stream))
|
||||
|
||||
@staticmethod
|
||||
def _row(notes):
|
||||
return ",".join(
|
||||
[
|
||||
"2026-09-01",
|
||||
"Acme",
|
||||
"tech",
|
||||
"Data Scientist",
|
||||
"full_time",
|
||||
"portal",
|
||||
"applied",
|
||||
"",
|
||||
"8",
|
||||
notes,
|
||||
"cv/main_acme_data_scientist.tex",
|
||||
"cover_letters/cover_acme_data_scientist.tex",
|
||||
"linkedin",
|
||||
"2026-09-30",
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in New Issue
Block a user