jobyssey/.claude/commands/sync-twenty.md

/sync-twenty - Push Ranked Jobs, Companies, and Applications to Twenty CRM

You are publishing a read-only view of the job search into the user's Twenty CRM: one Opportunity record per job, linked to one enriched Company record per employer. The repo files stay the system of record - job_scraper/seen_jobs.json owns scraped/ranked jobs and job_search_tracker.csv (when present) owns applications. Twenty is a disposable presentation layer on top of them; nothing ever syncs back.

This command requires the Twenty MCP server (.omp/mcp.json → twenty, OAuth). It reads local state, upserts company and opportunity records, and stops - it never ranks, applies, or edits repo files. Twenty is the pipeline backend; the sync contract itself is tool-agnostic (see "Adapting to Another Tool" at the end - only the sections marked (Twenty binding) are tool-specific).

Follow these steps in order.

---

Step 0: Parse Input

$ARGUMENTS may contain:

• Nothing → sync every ranked job (any score) plus every tracked application
• --min-score <N> → only sync ranked jobs whose rank_score ≥ N (applications always sync regardless)
• --all → alias for the default (every ranked job)
• --no-companies → skip Steps 4-5's company enrichment/creation; opportunities sync unlinked (or keep existing links)

---

Step 1: Preflight the Connection (Twenty binding)

The command is silently optional: when the destination is not reachable, the outcome is one clear message and a clean exit - nothing else in the framework notices this command exists.

1. Confirm the Twenty MCP tools are available in this session from the session's own tool list only (the meta-tools get_tool_catalog, learn_tools, execute_tool, or the mounted mcp__twenty_* devices). Never shell out to probe the server. If unavailable, stop and tell the user:

   Twenty MCP isn't connected. Add the twenty server to .omp/mcp.json (HTTP transport, OAuth), start a new session, complete the OAuth login, then re-run /sync-twenty.

2. Verify the connection with one cheap call: execute_tool → find_many_opportunities with { "select": ["id"], "limit": 1 }. An auth error → tell the user to re-authenticate and stop. Never retry in a loop.
3. Twenty MCP is interactively authenticated, so "connected but not authenticable right now" (expired OAuth, headless/CI) gets the same graceful exit as "not configured": state the reason in one line and stop. Never initiate the OAuth flow from this command.

All record operations go through the execute_tool meta-tool, e.g. execute_tool(toolName="create_many_opportunities", arguments={...}). If a tool's schema is unfamiliar, call learn_tools for it first; never guess field shapes. Parse every response defensively: a failed call may return a plain error string instead of JSON - catch it, record the batch as failed, and continue.

---

Step 2: Build the Sync Set (local data only - no external calls yet)

Validate the cheap, local precondition before creating anything external. A run with nothing to sync must exit with zero side effects.

1. Read job_scraper/seen_jobs.json (structure: { "seen": { "<url>": { ...job } } }) and job_search_tracker.csv (may be missing).
2. Select seen_jobs.json entries with status ranked. Apply the --min-score threshold from Step 0 when given; otherwise take all ranked entries. skipped entries never sync (they failed the location/language/salary gates and were never evaluated as opportunities). expired entries only sync when they carry a rank_score - a bare seen-then-expired job was never an opportunity.
3. Every tracker row joins the sync set (an applied-to job always syncs, ranked or not), matched to seen_jobs.json entries case-insensitively on company + role. Tracker rows with no seen_jobs.json entry sync too.
4. Status precedence: the tracker wins. A job ranked in seen_jobs.json but interview in the tracker syncs as interview. Jobs only in seen_jobs.json keep their stored status. Deadline precedence: the tracker wins too - the tracker's deadline overrides seen_jobs.json; never reconcile the two dates with min()/max().
5. If the sync set is empty, say "Nothing to sync - run /scrape and /rank first" (or, when jobs exist but all score below --min-score, say so and suggest lowering it) and stop.
6. Derive the employer set: unique companies across the sync set, keyed by a normalized name - lowercase, punctuation collapsed to spaces, legal/noise suffixes stripped (inc, llc, ltd, corp, co, labs, the, ...). Keep the first-seen display spelling for the record name.
7. State the counts before touching the destination: how many opportunities and how many companies will be created or checked, and the threshold in effect.

---

Step 3: Object Schemas and Sync State (Twenty binding)

The Twenty workspace already carries the required custom fields; this command assumes the schema exists and never mutates metadata. Confirm with get_object_metadata objectName="opportunity"|"company" includeFields=true only if a field is missing.

Opportunity (create_many_opportunities / upsert_many_opportunities):

Field	Type	Source / notes
name	title (string)	"<Role> — <Company>" on create; cleaned to the role only by Step 7 once companyId is set - the suffix is a readable fallback for unlinked records, redundant after linking
stage	select enum	job status → enum (Step 6 mapping); required on create
score	number	rank_score (0-100); omit when unranked
verdict	select enum	rank_verdict → enum (Step 6 mapping); omit when unranked
url	links	{ "primaryLinkUrl": <posting url>, "primaryLinkLabel": <role title> } - the dedup key
location	address	parse the location string: addressCity = text before the last comma, addressState = the remainder; whole string in addressCity when there is no comma
closeDate	date-time	deadline (tracker wins) as "YYYY-MM-DDT00:00:00.000Z"; omit when none
companyId	relation uuid	the employer's Company record (Step 5); omit under --no-companies
amount	currency	Salary - stated posting salary (max of the stated range) as { "amountMicros": <usd×1_000_000>, "currencyCode": "USD" }; filled by Step 7's enrichment ladder; omit when no grounded value exists

Records passed to create_many_*/upsert_many_* also require position - use "last".

Company (create_many_companies) - stock Twenty fields:

Field	Type	Source / notes
name	title (string)	display spelling from the sync set - the dedup key (normalized)
domainName	links	{ "primaryLinkUrl": "https://<domain>", "primaryLinkLabel": <name> } - strip scheme/www./trailing slash from the source value first
linkedinLink	links	LinkedIn company page URL (http:// → https://)
address	address	HQ addressCity / addressState / addressCountry - only sub-fields actually known
annualRevenue	currency	organization revenue in USD micros; omit when 0/unknown

Sync state: read/write job_scraper/twenty_sync.json (personal, gitignored - never commit):

{ "workspace_url": "https://crm.prad.nu", "last_sync": "YYYY-MM-DD" }

It exists only to record last_sync; posting URL (opportunities) and normalized name (companies) are the durable dedup anchors, so no id map is stored.

---

Step 4: Enrich the Employer Set (skip under --no-companies)

Enrichment fills domainName, linkedinLink, address, annualRevenue per employer. Sources, in priority order:

1. Local first: python3 tools/twenty_enrich.py. The script encodes the deterministic layer - it matches each normalized employer name against documents/postings/jobs.db (companies table; homepage domain joined through postings.company_domain, most frequent domain wins), sanitizes stringified-NaN/empty values (never emit https://nan - Twenty rejects invalid link URLs and fails the whole batch on one bad record), and emits job_scraper/enrichment_plan.json with an unresolved section listing exactly what local data cannot ground.
2. Web for the unresolved. Fan out scout subagents in parallel batches (~14 companies each), each given the company name plus a sample job title, location, and posting URL for disambiguation (generic names like "Sigma" or "Picnic" collide badly without it). Each scout web-searches and returns {company, domain, linkedin, city, state, country} with null for anything search results don't clearly confirm - never guessed.
3. Perplexity for the stragglers. Companies still unresolved after the scout wave (name collisions, recruiters, very obscure startups) go to the Perplexity MCP (perplexity_ask) in one batched question carrying the same disambiguation context; keep only facts it verifies with citations, and accept "unverified" as the answer.

An employer with no verified data still gets a record - name-only. Never invent a domain, LinkedIn slug, or HQ to make a record look complete; genuinely unverifiable employers (stealth startups, generic recruiter names) stay bare.

---

Step 5: Upsert Company Records (skip under --no-companies)

1. Load existing CRM companies once: page find_many_companies { "select": ["id","name"], "limit": 100, "offset": … } until hasNextPage is false. Map normalized name → id.
2. Reuse, don't recreate. Employers whose normalized name already exists in the CRM (including hand-added or seed records) keep the existing id for linking. Never overwrite an existing company's fields - it may carry the user's own edits.
3. Create the rest in batches of ≤ 20 via create_many_companies, each record position: "last" plus the Step 3 field map. Capture returned ids in input order into the name→id map. On a batch failure with a validation error (e.g. "The URL of the link is not valid"), bisect: retry that batch's records one-by-one to isolate and report the offender; fix by dropping the invalid field, not by guessing a replacement.
4. Never delete or archive company records.

---

Step 6: Upsert Opportunity Records

Twenty has no server-side upsert keyed on a custom field, so dedup client-side on the posting URL:

1. Load existing rows once: page find_many_opportunities with { "select": ["id","name","stage","url"], "limit": 100, "offset": … } until hasNextPage is false. Build a map primaryLinkUrl → id.
2. Split the sync set into records whose URL is absent from the map (create) and those present (update).
3. Create in batches of ≤ 20 via create_many_opportunities. Each record carries position: "last", stage, companyId (from Step 5's map, keyed by the job's normalized company name), and every mapped field from Step 3. Batch politely; on a rate-limit, back off and continue; report any batch that failed rather than retrying indefinitely.
4. Update matched rows in batches of ≤ 20 via upsert_many_opportunities, each record carrying the explicit id plus the required position/stage and the changed fields: stage, score, verdict, closeDate, companyId, amount. Passing id makes the upsert match-and-update (verified: created: 0, updated: N); assert the response reports zero creates - an unexpected create means the id didn't match and a duplicate now exists. Do not rewrite url on update - the URL is the identity. name is rewritten only by Step 7's cleanup.
5. Never delete or archive records, even for expired jobs - set stage to EXPIRED instead. Rows the user added by hand (no matching posting URL) are invisible to this command.

Status → stage enum (normalise legacy space spellings first):

job status	stage
ranked	RANKED
drafted	DRAFTED
applied	APPLIED
interview	INTERVIEW
offer	OFFER
hired	HIRED
rejected	REJECTED
no_response / no response	NORESPONSE
offer_declined / offer declined	OFFERDECLINED
withdrawn	WITHDRAWN
expired	EXPIRED

rank_verdict → verdict enum: Strong Fit→STRONG_FIT, Good Fit→GOOD_FIT, Moderate Fit→MODERATE_FIT, Weak Fit→WEAK_FIT, Poor Fit→POOR_FIT. Omit verdict when the job is unranked or the string is unrecognised - never invent an enum value.

---

Step 7: Enrich Opportunities, then Clean Names

Run after Step 6 so every opportunity already has its companyId.

1. Salary (amount) ladder - first grounded source wins, encoded in tools/twenty_enrich.py:
   1. seen_jobs.json salary_max_stated (stated in the posting at rank time)
   2. jobs.db postings salary_max matched on normalized company + title (stated on an ATS board)
   3. jobs.db companies levels.fyi SWE median (company-level estimate - its provenance is named in the Fit Analysis note, never presented as a posting-stated figure)
   4. Still missing → scout subagents re-fetch the posting page (salary often sits in JSON-LD) or find the same posting mirrored on another board; a posting that states no salary keeps an empty amount - an unstated salary is a fact, not a gap to fill.
2. Apply the script's plan: --apply with TWENTY_API_KEY set, else upsert the plan's patches through the MCP session (explicit id, batches ≤ 20, assert zero creates).
3. Name cleanup (last). For every opportunity whose companyId is set and whose name still carries the " — <Company>" suffix, rewrite name to the role only (build_name_cleanup_plan in the script). This runs after linking and enrichment because the suffix is the only company identification an unlinked record has.

---

Step 8: Sync Fit Analysis Notes

One note per synced opportunity, attached through noteTargets:

1. Idempotency guard: find_many_notes with title.startsWith "Fit Analysis —"; skip opportunities that already have one.
2. Create via create_many_notes (batches ≤ 20): title = "Fit Analysis — <Role> @ <Company>", bodyV2.markdown built from stored seen_jobs.json data only - score/verdict/quick-fit line, portal + dates line, employer tier / AI-tools context, salary gate (with the note naming a levels.fyi figure as an estimate), ## Strengths and ## Gaps bullet lists, likely interview style, and the posting link. Never fetch or reconstruct posting text here.
3. Link each returned note id via create_many_note_targets with { "noteId", "targetOpportunityId", "position": "last" }.
4. Notes are write-once: this step creates, never edits - the user may annotate them in Twenty. Email-derived context notes belong to /gather-recent-context, not this command.

---

Step 9: Verify and Report

Verify before reporting: find_many_opportunities with { "companyId": { "is": "NOT_NULL" } } must count every synced opportunity (all of them, unless --no-companies); total opportunity count must equal pre-run count + creates (no upsert-spawned duplicates); the Fit Analysis note count must equal the opportunity count.

## Twenty CRM Sync - YYYY-MM-DD

Workspace: <workspace_url>
Opportunities: <N> synced (threshold: score ≥ <T | none>): <C> created, <U> updated, <F> failed.
Companies: <M> employers: <C> created, <R> reused, <E> enriched (domain/linkedin), <B> name-only.
Linked: <L>/<N> opportunities → companies · Salary: <S>/<N> · Notes: <K>/<N>.

| | Opportunity | Company | Stage | Score |
|---|-----------|---------|-------|-------|
| ✚ | Senior Backend Engineer - Performance | Teleport | RANKED | 86 |
| ↻ | Software Engineer, Bridge | Stripe | INTERVIEW | 86 |

List failures with their error and the suggestion to re-run - the upsert is idempotent (posting URL for opportunities, normalized name for companies), so a re-run only touches what failed. Update last_sync in job_scraper/twenty_sync.json.

Remind the user once (first run only): the repo files remain the source of truth - edits made in Twenty never flow back, and /outcome is still how application results get recorded.

---

Important Rules

1. One-way, always. Twenty content never flows back into seen_jobs.json, the tracker, or any repo file. The gitignored twenty_sync.json is this command's only local write.
2. Idempotent upserts. Opportunities dedup on url.primaryLinkUrl; companies dedup on normalized name. Re-running creates nothing twice; matching is never fuzzy title comparison.
3. Metadata is read-only. Assume the custom fields already exist; never create, retype, or delete object/field metadata from this command.
4. Batch cap is 20. Every create_many_*/upsert_many_* call rejects more; always chunk. One invalid record fails its whole batch - bisect on validation errors.
5. Never fabricate. Omit a field rather than guess it: no invented enum values, deadlines, domains, or LinkedIn slugs. Enrichment is jobs.db or verified web search - unverifiable employers stay name-only.
6. Existing companies belong to the user. Reuse their id for linking; never overwrite or delete their fields.
7. Job data only. The candidate profile, behavioral notes, and evaluation framework never sync - this is a pipeline view, not a profile export.
8. Documents never leave the machine. CVs and cover letters are not uploaded or embedded; the local repo and documents/applications/ archive are their only home.

---

Adapting to Another Tool (forks)

The sync contract is tool-agnostic; only the sections marked (Twenty binding) are tool-specific. A fork targeting a different CRM keeps Steps 0, 2, 4, 7-9, and every Important Rule unchanged - build the same sync set, enrich the same employer set, upsert on the same posting-URL/company-name keys, keep the destination read-only-to-source and documents local - and swaps only:

• Step 1 (connection preflight) for the target tool's MCP server or access check, keeping the silently-optional bar: not configured or not authenticable both end in one message and a clean exit.
• Steps 3, 5-6 (schemas, upsert mechanics) for the target tool's records API, reusing the same field tables and a renamed sync-state file.