docs(add-portal): specify credential handling for portals needing an API token (#304)

* docs(add-portal): specify credential handling for portals needing an API token

/add-portal can already scaffold a skill for a portal that returns usable
content only through a paid fetching service, but the spec says nothing
about the credential such a skill needs: no guidance on where it lives, no
requirement to document it, no defined failure mode when it is unset, and
no point at which the user learns the portal costs money per query.

Three additions, all to the generator spec:

- Step 2 gains a reconnaissance point for whether the portal needs a
  credential at all, to be raised with the user before scaffolding. A
  portal that bills per query is a different proposition from a free one
  and the user should get to decline it.
- The portal-skill contract gains a Credentials rule: environment variable
  only, named <SERVICE>_API_TOKEN, never a CLI flag (flags leak into shell
  history and process listings), never in url-reference.md or a fixture,
  and a MISSING_CREDENTIALS exit rather than a fallthrough to an
  unauthenticated request that fails confusingly.
- SKILL.md for such a skill must carry a Setup section naming the service,
  the variable, and the per-call billing.

Spec only. All six shipped portals are free and unauthenticated, so no
existing skill changes and the zero-dependency default is untouched.

Evidence this is not speculative: running /add-portal against portals that
sit behind bot-detection produced four token-requiring skills in my own
fork (Bright Data Web Unlocker), each of which had to invent its own
convention for reading, documenting, and failing on the token, because the
spec defines none.

Verified: lint_skills, check_framework_version, security_guards all OK;
python3 -m unittest discover -s tests 196 passed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(add-portal): subordinate the credential check to the robots.txt decision

Review catch: as written, Step 2.5 could be read as "robots.txt or the
terms said no, but a paid unlocker gets through, so here is the convention
for that" - which would let a paid fetcher launder a robots refusal.

That is not the intent and it contradicts the posture settled in #277/#286
(09-web-research.md, tools/robots_check.py): robots.txt is decided first
and honestly, and is never overridden by better fetching machinery.

State the subordination explicitly in 2.5 rather than leaving it implied by
step order, and record it in the changelog entry so the constraint survives
later edits to the step. The credential path is for portals whose robots.txt
permits access but whose bot protection blocks ordinary fetches - paying for
access a site allows, never paying past a site's no.

Wording follows the reviewer's suggestion.

Verified: lint_skills, check_framework_version, security_guards all OK;
python3 -m unittest discover -s tests 219 passed.

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

.claude/commands/add-portal.md

@@ -41,6 +41,8 @@ Do reconnaissance before writing any code. Use WebFetch (or `curl` via Bash) on
    - If the portal requires login/authentication to view listings, **stop**: this pattern only works on public pages. Tell the user and suggest checking whether the portal has an official API.
    - If robots.txt disallows the paths or the portal's terms prohibit automated access, tell the user plainly and let them decide whether to proceed for personal use. If they proceed, the generated `SKILL.md` **must** carry a prominent personal-use-only warning (copy the tone of `linkedin-search`'s "⚠️ Personal use only" section: keep volume low, no commercial or bulk use, own responsibility).
 
+5. **Check whether the portal can be reached without a credential.** Some portals return usable content only through a third-party fetching service (a paid unlocker/proxy API). **This step never overrides Step 2.4:** if `robots.txt` or the portal's terms disallow access, that is decided there, and a paid fetching service does not change the answer. The credential path exists for portals whose `robots.txt` permits access but whose bot protection blocks ordinary fetches. Where that applies and the test fetch succeeds only through such a service, say so to the user **before scaffolding** - a portal that bills per query is a different proposition from a free one, and they may prefer to skip it. Note which service and which environment variable; the handling rules are in the portal-skill contract in Step 3.
+
 Record everything you found - endpoints, parameters, field anchors, quirks - you will write it into `url-reference.md` in Step 3.
 
 ---
@@ -80,11 +82,12 @@ These conventions are what make portal skills interchangeable for `/scrape` and
 - **Fetching:** browser User-Agent, exponential backoff with jitter on 429/5xx (max ~6 retries), `""`/`null` on 404 rather than a crash.
 - **HTML parsing:** split the response into per-result chunks and parse each independently, so one malformed card cannot break the rest (see `parseJobCards` in `linkedin-search/cli/src/helpers.ts`).
 - **Dependencies:** default to **zero runtime dependencies** (plain `bun` + `fetch` + regex parsing) like `linkedin-search` - `bun install` should only pull dev types. Only add a parsing library if the portal's markup genuinely defeats chunked regex parsing, and say so in the README.
+- **Credentials:** a skill that needs an API key (Step 2.5) reads it **only** from an environment variable named `<SERVICE>_API_TOKEN`. Never hardcode it, never accept it as a CLI flag (flags leak into shell history and process listings), and never write a real token into `url-reference.md`, a README example, or a test fixture. If the variable is unset, exit `1` with the standard stderr JSON error and code `MISSING_CREDENTIALS`, naming the variable to set - never fall through to an unauthenticated request that fails confusingly. The repo `.gitignore` covers `.env`; do not commit one.
 
 ### File specifics
 
 - **`SKILL.md` frontmatter:** `name`, `version: 1.0.0`, a `description` written for skill triggering - it must name the portal, the market, and include trigger phrases in English **and** the market's language; `context: fork`; `allowed-tools: Bash(bun run skills/<name>/cli/src/cli.ts *)`.
-- **`SKILL.md` body:** what the skill searches, the personal-use warning if Step 2 found terms restrictions, command reference with flags, 4-6 usage examples using the user's market (real cities, realistic roles), output-format table, and a Notes section recording portal quirks found in Step 2.
+- **`SKILL.md` body:** what the skill searches, the personal-use warning if Step 2 found terms restrictions, command reference with flags, 4-6 usage examples using the user's market (real cities, realistic roles), output-format table, and a Notes section recording portal quirks found in Step 2. If Step 2.5 found the portal needs a credential, add a **Setup** section naming the service, the exact environment variable to export, and the fact that every call is billed - stated where the user reads it before running the skill, not after.
 - **`url-reference.md`:** the endpoints, parameters table, and response-structure notes from Step 2 - this is the file a future maintainer needs when the portal changes its markup.
 - **`package.json`:** name `<portal>-cli`, `"type": "module"`, scripts `start`, `test` (`bun test --timeout 30000`), and `typecheck` (`tsc --noEmit`); dev-only dependencies in the zero-dependency default.
 - **`tests/`:** copy `runCLI`/`parseJSON` from `jobindex-search/cli/tests/helpers.ts`, then add a small live smoke-test file: `search` with the test query returns exit code 0 and ≥1 result with non-null `id`/`title`/`url`; a bogus flag or missing required arg exits 1 with a JSON error on stderr.
@@ -154,3 +157,4 @@ Present a summary:
 - The portal-skill contract keeps every generated skill interchangeable with the shipped ones: same commands, same flags, same output shape, same error convention.
 - Zero runtime dependencies by default, matching `linkedin-search` - a portal skill should run on a fresh clone with nothing but `bun`.
 - Access rules are surfaced, not silently bypassed: auth-walled portals are declined, robots.txt/ToS restrictions are reported to the user, and restricted portals get a prominent personal-use-only warning in the generated skill.
+- Credentials live in the environment, never in the repo: a generated skill reads its token from an environment variable, fails loudly when it is unset, and never commits it. Per-call cost is disclosed before the skill is generated, not discovered afterwards.

CHANGELOG.md

@@ -113,6 +113,19 @@ per-file diff commands.
 
 ### Changed
 
+- **`/add-portal` now specifies how a generated skill handles an API token** - the command
+  could already scaffold a skill for a portal reachable only through a paid fetching
+  service, but said nothing about the credential such a skill needs. It now checks for that
+  case during reconnaissance and raises the per-call cost with the user *before*
+  scaffolding. That check is explicitly subordinate to the `robots.txt`/terms decision
+  in Step 2.4 - a paid fetching service never launders a refusal, and the credential
+  path exists only for portals whose `robots.txt` permits access but whose bot
+  protection blocks ordinary fetches. The portal-skill contract requires the token to come from a
+  `<SERVICE>_API_TOKEN` environment variable (never a CLI flag, never a fixture) and to
+  fail with `MISSING_CREDENTIALS` when unset; and such a skill's `SKILL.md` must carry a
+  Setup section naming the service, the variable, and the billing. Spec only - no shipped
+  portal needs a credential, so no existing skill changes.
+
 - **The four Danish demo portals now ship disabled** (#288) - `jobindex-search`,
   `jobbank-search`, `jobdanmark-search`, and `jobnet-search` default to `enabled: false`,
   and `/setup`'s job-portals question now acts on the answer: it flips them to