docs(releases): add CHANGELOG + release-based update guidance; sharpen real-path bar (#225)

Addresses #213 (how to keep up with a fast-moving upstream) and closes the
verification loophole surfaced in the 2026-07-22 triage audit.

- Add CHANGELOG.md (Keep a Changelog + semver), with v1.0.0 as the first
  tagged baseline and an Unreleased section for going forward.
- SETUP.md section 8: recommend updating to a tagged release (a vetted,
  described checkpoint) over pulling raw master; fetch --tags and merge a tag.
- README: add a "Staying up to date" pointer to Releases, the CHANGELOG, and
  check_upstream_updates.py.
- CONTRIBUTING.md: sharpen "Claims get verified" - a test that distinguishes
  master from the fix is necessary but not sufficient; the failing input must
  be one the workflow actually produces, not one the test hand-builds. Fixes
  demonstrated only through a synthetic input the real code path never receives
  get declined even when their test is green.

Note: the git tag / GitHub Release for v1.0.0 is intentionally left for the
maintainer to cut.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Mads Lorentzen
2026-07-22 11:36:47 +02:00
committed by GitHub
co-authored by Claude Opus 4.8
parent d88c023683
commit 905f6e0946
4 changed files with 52 additions and 0 deletions
+1
View File
@@ -32,6 +32,7 @@ A new command therefore faces a high bar. The test that admitted the existing on
Reviews here are empirical. Bug reports are reproduced on master before the fix is considered; "all tests green" is checked against whether the tests can distinguish master from the fix. PRs whose premise doesn't reproduce get declined even when the code is fine - it has happened ([#35]'s converter fix, [#52]'s first version). You can make this fast:
- State the failing case and how to reproduce it.
- **Reproduce on the real path, not a constructed input.** A test that fails on master and passes on the fix is necessary but not sufficient: the failing input has to be one the workflow actually produces, not one the test hand-builds. Show the failure through the path the code really runs - the documented CLI invocation, real portal output, an actual data file - not a synthetic value fed straight to the function. A fix whose only demonstration is an input the real code path never receives gets declined even though its test is green.
- Put CLI tests in `.agents/skills/<name>/cli/tests/` (bun test, network-free where possible); Python tool tests in `tests/`.
- Run what CI runs: `python3 tools/lint_skills.py`, `python3 tools/check_framework_version.py`, `bun run typecheck` in touched CLIs, and the relevant test suites.