fix(gmail-sync,outcome): keep free-form tracker notes free of CSV-breaking characters (#454) (#455)

* fix(gmail-sync): strip CSV-breaking characters from the email subject

Step 7a interpolated the raw subject line of a received email into the
`notes` column of job_search_tracker.csv. No writer here emits a quoted
tracker field, so an unescaped comma splits the row - for csv.DictReader
just as much as for a naive split, which matters because
tools/rank_state.py is the repo's only machine reader and uses exactly
that. `notes` is column 10 of 14, so a subject as ordinary as
"Re: Your application, Data Scientist" shifted cv_file,
cover_letter_file and source a column left. A line break is worse: it
ends the row and starts a second one.

The rule now sits on the append instruction itself rather than in a
general note a writer can miss. The subject survives verbatim in the
archive's outcome.md, which is Markdown and carries no such constraint.

/outcome Step 4 (outcome.md:195) is also free-form and has the same
exposure, but its text is model-authored in a turn the user is watching
rather than copied from third-party mail unattended. Left out
deliberately, to be filed separately.

* fix(outcome): keep the Step 4 tracker note free of CSV-breaking characters

/outcome Step 4 appended "a short dated note" to `notes` with no
constraint on its content, the same exposure /gmail-sync Step 7a had:
nothing quotes a tracker field, so `rejected, no feedback given` shifts
cv_file, cover_letter_file and source a column left under
csv.DictReader, and a line break ends the row. The append instruction
now requires a note with no commas, double quotes or line breaks.

Folded in at the maintainer's request on #455 so one entry and one rule
cover both free-form writers. The CSV-safety tests move out of
test_gmail_sync_command.py into test_tracker_notes_csv_safe.py, where a
CASES table pins the rule on each writer's append line.
This commit is contained in:
Jakob Stender Guldberg
2026-09-16 06:45:55 +02:00
committed by GitHub
parent 1b65f7198a
commit c93609cd22
4 changed files with 159 additions and 2 deletions
+3 -1
View File
@@ -124,7 +124,9 @@ Approving the whole batch in one reply is expected UX - the requirement is that
For every row the user approved: For every row the user approved:
1. **Tracker (`job_search_tracker.csv`):** update the matched row's `status` column per the Step 5 table, and append to `notes`: `<date> gmail-sync: <signal> ("<email subject>")`. Never restructure the CSV, reorder rows, or touch unrelated rows - same rule `/outcome` follows. The rewrite touches only `status`, `notes` (and `date` when the drafted-rule below fires): preserve every other field of the row, parsed or not, so the `deadline` column written by `/apply` Step 6b - or any column added in the future - is never blanked by a status sync. 1. **Tracker (`job_search_tracker.csv`):** update the matched row's `status` column per the Step 5 table, and append to `notes`: `<date> gmail-sync: <signal> ("<email subject>")`, **with every comma, double quote and line break deleted from the subject first**. No writer here emits a quoted tracker field and no reader unquotes one, so an unescaped comma splits the row identically for a naive split and for the `csv.DictReader` the shipped reader actually uses (`tools/rank_state.py`): `cv_file`, `cover_letter_file` and `source` each shift a column left. A line break is worse - it ends the row and starts a second one. The double quote is stripped as cheap insurance for the day something does quote a field; on today's readers it is harmless. The subject is a human-readable breadcrumb here, not data anything reads back - item 2 below keeps it verbatim in `outcome.md`, which is Markdown and carries no such constraint. This matters more than it looks: `/gmail-sync` is the only tracker writer that copies *third-party* text, and the only one that runs unattended, so nobody is watching the row it edits.
Never restructure the CSV, reorder rows, or touch unrelated rows - same rule `/outcome` follows. The rewrite touches only `status`, `notes` (and `date` when the drafted-rule below fires): preserve every other field of the row, parsed or not, so the `deadline` column written by `/apply` Step 6b - or any column added in the future - is never blanked by a status sync.
**If the matched row was still `drafted`,** also set `date` to the email's date. The employer replying proves the user submitted by hand without running `/outcome`, so the drafting date now in that column is wrong. The email's date is an upper bound on the real submission date, tight for an ack and loose for a rejection weeks later, which is why Step 6 shows it and lets the user supply the actual date instead. **If the matched row was still `drafted`,** also set `date` to the email's date. The employer replying proves the user submitted by hand without running `/outcome`, so the drafting date now in that column is wrong. The email's date is an upper bound on the real submission date, tight for an ack and loose for a rejection weeks later, which is why Step 6 shows it and lets the user supply the actual date instead.
2. **`outcome.md`:** tick the relevant stage checkbox (adding the date in parentheses) or update `Status`/`Date resolved` per the table. Append a dated entry to `## Notes`, never overwrite existing Notes history: 2. **`outcome.md`:** tick the relevant stage checkbox (adding the date in parentheses) or update `Status`/`Date resolved` per the table. Append a dated entry to `## Notes`, never overwrite existing Notes history:
+1 -1
View File
@@ -192,7 +192,7 @@ Update rules: tick stage checkboxes as they are reached (add the date in parenth
## Step 4: Update the Tracker ## Step 4: Update the Tracker
Update the matched row's `status` column using the canonical spellings from **Tracker status vocabulary** above (e.g. `drafted` → `applied` → `interview` → `offer` → `hired` / `rejected` / `no_response` / `offer_declined` / `withdrawn`) and append a short dated note to the `notes` column. Never restructure the CSV, reorder rows, or touch other rows. The rewrite touches only the `status` and `notes` columns: preserve every other field of the row, parsed or not, so a value the row carries - the `deadline` written by `/apply` Step 6b, or any column added in the future - is never blanked by a status update. Update the matched row's `status` column using the canonical spellings from **Tracker status vocabulary** above (e.g. `drafted` → `applied` → `interview` → `offer` → `hired` / `rejected` / `no_response` / `offer_declined` / `withdrawn`) and append a short dated note to the `notes` column, **containing no commas, double quotes or line breaks**. No writer here emits a quoted tracker field, so a comma in the note shifts `cv_file`, `cover_letter_file` and `source` a column left for `csv.DictReader` (`tools/rank_state.py`) as much as for a naive split, and a line break ends the row - `rejected, no feedback given` is exactly the sentence that corrupts it; write `rejected - no feedback given`. `/gmail-sync` Step 7a applies the same rule to the email subjects it appends. Never restructure the CSV, reorder rows, or touch other rows. The rewrite touches only the `status` and `notes` columns: preserve every other field of the row, parsed or not, so a value the row carries - the `deadline` written by `/apply` Step 6b, or any column added in the future - is never blanked by a status update.
**Moving a row off `drafted`:** rows written by `/apply` Step 6b carry the date the documents were drafted, not the date they were sent. Whenever this step advances such a row to any other status - `applied`, or straight to `interview` or `rejected` when the user reports an outcome for something they submitted without recording it - overwrite its `date` column with the actual submission date. The `date` column is read as "applied on" by `/notion-sync` and drives `/html-report`'s year/season grouping and this command's own days-quiet count, so leaving the draft date in place would misreport the application. **Moving a row off `drafted`:** rows written by `/apply` Step 6b carry the date the documents were drafted, not the date they were sent. Whenever this step advances such a row to any other status - `applied`, or straight to `interview` or `rejected` when the user reports an outcome for something they submitted without recording it - overwrite its `date` column with the actual submission date. The `date` column is read as "applied on" by `/notion-sync` and drives `/html-report`'s year/season grouping and this command's own days-quiet count, so leaving the draft date in place would misreport the application.
+20
View File
@@ -118,6 +118,26 @@ per-file diff commands.
wrappers, plus a new `detail-backoff.test.ts` that exercises the handler path itself - wrappers, plus a new `detail-backoff.test.ts` that exercises the handler path itself -
its retry cases fail against the bare `fetch()`. its retry cases fail against the bare `fetch()`.
- **Free-form tracker notes no longer break the CSV row** (#454) (`.claude/commands/gmail-sync.md`,
`.claude/commands/outcome.md`, `tests/test_tracker_notes_csv_safe.py`) - two writers put
free-form text into the `notes` column of `job_search_tracker.csv`: `/gmail-sync` Step 7a
copied the raw subject of a received email, and `/outcome` Step 4 appended "a short dated
note" with no constraint on its content. No writer in the framework emits a quoted tracker
field, so an unescaped comma splits the row for a naive split and for `csv.DictReader` alike -
the latter being what the repo's only machine reader of the tracker uses
(`tools/rank_state.py`). `notes` is column 10 of 14, so a subject as ordinary as
`Re: Your application, Data Scientist`, or a note as natural as `rejected, no feedback given`,
shifted `cv_file`, `cover_letter_file` and `source` a column left. A line break is worse: it
ends the row and starts a second one. Nothing validated the row afterwards, and the
`/gmail-sync` half was written unattended, so the corruption was silent. Both append
instructions now carry the rule themselves - `/gmail-sync` deletes commas, double quotes and
line breaks from the subject, `/outcome` writes its note without them - rather than a general
note a writer can miss. Nothing is lost on the `/gmail-sync` side: Step 7a item 2 still
records the subject verbatim in the archive's `outcome.md`, which is Markdown and carries no
such constraint. The fixed-format writers (`followed up YYYY-MM-DD`,
`stale resolved no_response (YYYY-MM-DD)`, `redrafted`) could never contain these characters
and are unchanged.
- **`jobindex-search detail` no longer fetches arbitrary URLs or invents posting-shaped - **`jobindex-search detail` no longer fetches arbitrary URLs or invents posting-shaped
output** (#447) - the command fetched any `http(s)` input verbatim (no host check) and, output** (#447) - the command fetched any `http(s)` input verbatim (no host check) and,
when the path didn't match its one pattern, silently used the whole input URL as the job when the path didn't match its one pattern, silently used the whole input URL as the job
+135
View File
@@ -0,0 +1,135 @@
"""Guards for the rule that keeps free-form `notes` from breaking a tracker row.
No writer in this framework emits a quoted tracker field, so a comma inside
`notes` splits the row - for the `csv.DictReader` in `tools/rank_state.py` as
much as for a naive split - and shifts `cv_file`, `cover_letter_file` and
`source` a column left. A line break is worse: it ends the row. Two writers put
free-form text into `notes`: `/gmail-sync` Step 7a copies an email subject, and
`/outcome` Step 4 writes a short note of its own. The fixed-format writers
(`followed up YYYY-MM-DD`, `stale resolved no_response (YYYY-MM-DD)`,
`redrafted`) cannot contain the characters and are not listed.
The spec IS the implementation, so the guard is `CASES`: each rule must sit on
the line that instructs the append, not merely somewhere in the section. The
shape tests below it parse with `csv.DictReader` and document why the rule
exists; they pass on master too.
"""
import csv
import io
import re
import unittest
from pathlib import Path
REPO = Path(__file__).resolve().parent.parent
COMMANDS = REPO / ".claude" / "commands"
GMAIL_SYNC = COMMANDS / "gmail-sync.md"
OUTCOME = COMMANDS / "outcome.md"
TRACKER_HEADER = (
"date,company,sector,role,role_type,channel,status,contact_person,"
"fit_rating,notes,cv_file,cover_letter_file,source,deadline"
)
def section(path, heading):
"""The body of one markdown section, up to the next heading of any depth."""
text = path.read_text(encoding="utf-8")
start = text.index(heading) + len(heading)
rest = text[start:]
end = re.search(r"^#{1,4} ", rest, re.MULTILINE)
return rest[: end.start()] if end else rest
class FreeFormNotesWritersStateTheRule(unittest.TestCase):
"""Format: (path, heading, line_anchor, rule, why)"""
CASES = [
(
GMAIL_SYNC,
"### Step 7a: Write Approved Updates",
"append to `notes`",
"with every comma, double quote and line break deleted from the subject first",
"Step 7a item 2 deliberately keeps the subject verbatim in `outcome.md`, "
"so the rule must sit on the tracker append, not anywhere in the step",
),
(
OUTCOME,
"## Step 4: Update the Tracker",
"append a short dated note",
"containing no commas, double quotes or line breaks",
"Step 4 is the primary status-update path and its note is written "
"free-form - `rejected, no feedback given` is the natural sentence",
),
]
def test_rule_is_stated_where_the_append_happens(self):
for path, heading, anchor, rule, why in self.CASES:
with self.subTest(path=path.name, heading=heading):
lines = [l for l in section(path, heading).splitlines() if anchor in l]
self.assertEqual(len(lines), 1, f"expected one append instruction: {why}")
self.assertIn(rule, lines[0], why)
class NotesShapeUnderTheShippedReader(unittest.TestCase):
"""Why the rule exists, demonstrated with the reader the repo ships."""
SUBJECT = 'Re: Your application, Data Scientist - "next steps"'
def test_sanitised_note_keeps_the_row_parseable(self):
safe = self.SUBJECT.replace(",", "").replace('"', "")
rows = self._parse(f'2026-09-12 gmail-sync: acknowledged ("{safe}")')
self.assertEqual(len(rows), 1, "the note must not end the row early")
row = rows[0]
self.assertIsNone(row.get(None), "the row must be no wider than the header")
self.assertEqual(row["cv_file"], "cv/main_acme_data_scientist.tex")
self.assertEqual(
row["cover_letter_file"], "cover_letters/cover_acme_data_scientist.tex"
)
self.assertEqual(row["source"], "linkedin")
def test_a_comma_in_the_note_shifts_the_columns(self):
for note in (
f'2026-09-12 gmail-sync: acknowledged ("{self.SUBJECT}")',
"2026-09-12 rejected, no feedback given",
):
with self.subTest(note=note):
row = self._parse(note)[0]
self.assertIsNotNone(row.get(None), "the comma must widen the row")
self.assertNotEqual(row["cv_file"], "cv/main_acme_data_scientist.tex")
self.assertNotEqual(row["source"], "linkedin")
def test_a_line_break_in_the_note_splits_the_row_in_two(self):
rows = self._parse('2026-09-12 gmail-sync: acknowledged ("Re: update\nlater")')
self.assertEqual(len(rows), 2)
self.assertIsNone(rows[0]["cv_file"], "the first row ends mid-note")
@classmethod
def _parse(cls, notes):
stream = io.StringIO(TRACKER_HEADER + "\n" + cls._row(notes) + "\n")
return list(csv.DictReader(stream))
@staticmethod
def _row(notes):
return ",".join(
[
"2026-09-01",
"Acme",
"tech",
"Data Scientist",
"full_time",
"portal",
"applied",
"",
"8",
notes,
"cv/main_acme_data_scientist.tex",
"cover_letters/cover_acme_data_scientist.tex",
"linkedin",
"2026-09-30",
]
)
if __name__ == "__main__":
unittest.main()