feat(free-router): run freellmapi under bun on localhost:9876 as a user service
This commit is contained in:
1 parent
f50717dfac
commit
6f06d5c89a
19 files changed
+230
-10
No files matched your search
+2
-1
@@ -50,9 +50,10 @@ That command installs chezmoi, clones [env/omarchy](https://prad.codes/env/omarc
|
||||
- [Hyprland](desktop/hyprland.md)
|
||||
- [Waybar](desktop/waybar.md)
|
||||
- [Fonts and apps](desktop/fonts-and-apps.md)
|
||||
- [AI agents](ai/README.md): Claude Code, omp, and the shared agent skill library.
|
||||
- [AI agents](ai/README.md): Claude Code, omp, the shared agent skill library, and the free-router LLM endpoint.
|
||||
- [Overview](ai/README.md)
|
||||
- [Skills](ai/skills.md)
|
||||
- [free-router](ai/free-router.md)
|
||||
|
||||
## Machine roles
|
||||
|
||||
|
||||
+3
-1
@@ -1,16 +1,18 @@
|
||||
# AI agents
|
||||
|
||||
The shared agent skill library and the shell entry points for coding agents.
|
||||
The shared agent skill library, a local free-tier LLM router, and the shell entry points for coding agents.
|
||||
|
||||
The repo manages the agent skill library only. Claude Code (`~/.claude`) and omp (`~/.omp`) configs are not in this repo. omp, opencode, and crush are global mise tools (`packages.shared.mise`, installed by [15-tools](../chezmoi/scripts.md#15-tools)) and are updated by topgrade's `mise upgrade`; no other agent CLI is installed by chezmoi.
|
||||
|
||||
- [Skills](skills.md): Every agent skill in [prdlk/skills](https://github.com/prdlk/skills), installed into `~/.agents/skills` and `~/.claude/skills` by an apply script.
|
||||
- [free-router](free-router.md): The freellmapi router on `localhost:9876` as a systemd user service, with `FREE_ROUTER_API_KEY` exported to every shell.
|
||||
|
||||
## Pages
|
||||
|
||||
| Page | Source in repo | Target in `$HOME` | Covers |
|
||||
| --- | --- | --- | --- |
|
||||
| [Skills](skills.md) | `.chezmoidata/skills.yaml`, `.chezmoiscripts/run_onchange_after_25-skills.sh.tmpl` | `~/.agents/skills/`, `~/.claude/skills/` | The prdlk/skills source and skill list; installed by the Skills CLI, not vendored |
|
||||
| [free-router](free-router.md) | `dot_config/free-router/env`, `dot_local/bin/executable_free-router`, `dot_config/systemd/user/free-router.service`, `.chezmoiexternals/freellmapi.toml` | `~/.config/free-router/env`, `~/.local/bin/free-router`, `~/.local/share/freellmapi` | freellmapi under Bun on `localhost:9876`, the pinned API key, the service, updates |
|
||||
|
||||
## Shell entry points
|
||||
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
# free-router
|
||||
|
||||
[freellmapi](https://github.com/tashfeenahmed/freellmapi) (an OpenAI/Anthropic-compatible router over free LLM provider tiers) running under Bun as a systemd user service on `http://localhost:9876`. Shells get the endpoint's port and API key as `FREE_ROUTER_PORT` and `FREE_ROUTER_API_KEY`. Linux desktop/laptop roles only.
|
||||
|
||||
| Piece | Source in repo | Target in `$HOME` |
|
||||
| --- | --- | --- |
|
||||
| Settings | `dot_config/free-router/env` | `~/.config/free-router/env` (symlink) |
|
||||
| Launcher | `dot_local/bin/executable_free-router` | `~/.local/bin/free-router` |
|
||||
| Unit | `dot_config/systemd/user/free-router.service` | `~/.config/systemd/user/free-router.service` |
|
||||
| Router source | `.chezmoiexternals/freellmapi.toml` | `~/.local/share/freellmapi` (upstream checkout) |
|
||||
| State | none | `~/.local/state/free-router/` (`freeapi.db`, `encryption-key`, `built-head`) |
|
||||
|
||||
## Settings
|
||||
|
||||
**~/.config/free-router/env**
|
||||
|
||||
```sh
|
||||
FREE_ROUTER_PORT=9876
|
||||
FREE_ROUTER_API_KEY=sk_gz58b6RRX5
|
||||
```
|
||||
|
||||
One file, three readers: `free-router.service` loads it with `EnvironmentFile=`, bash and zsh source it with `set -a` (`.bashrc`, `.zshrc`), and fish exports each `KEY=VALUE` line (`conf.d/10-env.fish`). Keep it to plain `KEY=VALUE` lines so all three parse it. After a change, run `systemctl --user restart free-router` and open a new shell.
|
||||
|
||||
## Launcher
|
||||
|
||||
`free-router` is a Bun script (`#!/usr/bin/env bun`). It needs `FREE_ROUTER_PORT` and `FREE_ROUTER_API_KEY` in its environment and exits 1 without them or without the checkout.
|
||||
|
||||
1. **Build, once per upstream commit.** If `git rev-parse HEAD` of the checkout differs from `built-head`, it runs `bun install --no-save` and `bun --bun run build` in `client/` (the dashboard), then records the commit. `--no-save` keeps a `bun.lock` out of the checkout, so chezmoi's `--ff-only` pulls never conflict.
|
||||
2. **Encryption key.** Generates `encryption-key` (32 random bytes, hex, mode `0600`) on first run and passes it as `ENCRYPTION_KEY`. Provider keys in the database are encrypted with it: losing the file orphans them.
|
||||
3. **Environment.** `NODE_ENV=production`, `HOST=127.0.0.1`, `PORT=$FREE_ROUTER_PORT`, `FREEAPI_DB_PATH=~/.local/state/free-router/freeapi.db`.
|
||||
4. **SQLite driver.** Bun refuses to load `better-sqlite3` ([oven-sh/bun#4290](https://github.com/oven-sh/bun/issues/4290)). A Bun plugin serves `require('better-sqlite3')` from the server's own `node:sqlite` adapter (`server/src/db/node-sqlite.ts`, its Android/Termux driver).
|
||||
5. **API key.** Runs the migrations, writes `FREE_ROUTER_API_KEY` to the `unified_api_key` setting, then imports `server/src/index.ts`. The key is set before the server listens, so the random key upstream generates is never accepted.
|
||||
|
||||
Regenerating the unified key in the dashboard replaces it until the next restart, which writes `FREE_ROUTER_API_KEY` back.
|
||||
|
||||
## Service
|
||||
|
||||
```ini
|
||||
[Service]
|
||||
EnvironmentFile=%h/.config/free-router/env
|
||||
ExecStart=%h/.local/bin/free-router
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
```
|
||||
|
||||
[`30-user-services`](../chezmoi/scripts.md#30-user-services) enables it with `--now`. It starts with the user's systemd manager at login and is restarted 5 s after any exit. Without lingering (`loginctl enable-linger`), it does not run before the first login after a reboot. The first start after a new upstream commit takes longer (install and dashboard build); `journalctl --user -u free-router -f` shows progress.
|
||||
|
||||
## Using it
|
||||
|
||||
1. Open `http://localhost:9876`, create the dashboard account (a browser on this machine needs no setup code), and add provider keys on the **Keys** page. Until then, completions fail with `no_providers`.
|
||||
2. Point clients at `http://localhost:$FREE_ROUTER_PORT/v1` with `$FREE_ROUTER_API_KEY`:
|
||||
|
||||
```sh
|
||||
curl -s http://localhost:$FREE_ROUTER_PORT/v1/chat/completions \
|
||||
-H "Authorization: Bearer $FREE_ROUTER_API_KEY" -H 'content-type: application/json' \
|
||||
-d '{"model":"auto","messages":[{"role":"user","content":"hi"}]}'
|
||||
```
|
||||
|
||||
## Updating
|
||||
|
||||
chezmoi fast-forwards `~/.local/share/freellmapi` at most once a week (`refreshPeriod = "168h"`), or now with `chezmoi apply --refresh-externals ~/.local/share/freellmapi`. The running service keeps the old code; `systemctl --user restart free-router` rebuilds and starts the new commit.
|
||||
@@ -24,6 +24,7 @@ The repo is a [chezmoi](https://chezmoi.io) source directory. It lives at `~/Cod
|
||||
.chezmoiexternal.toml
|
||||
.chezmoiexternals/
|
||||
agent-root.toml
|
||||
freellmapi.toml
|
||||
ghostty.toml
|
||||
herdr.toml
|
||||
kitty.toml
|
||||
@@ -86,7 +87,7 @@ README.md
|
||||
| `.chezmoi.toml.tmpl` | `~/.config/chezmoi/chezmoi.toml` | Config template rendered by `chezmoi init`. See [Config and roles](config.md). |
|
||||
| `.chezmoidata/` | — | YAML merged into template data (`.packages`, `.aliases`, `.skills`). See [Packages and aliases](data.md) and [Skills](../ai/skills.md). |
|
||||
| `.chezmoiexternal.toml` | — | Git repos cloned into `$HOME` (nvim config, a bat syntax, tpm). |
|
||||
| `.chezmoiexternals/` | — | One external per app or tool: agent-root, ghostty, herdr, kitty, omarchy, omp, television, tmux, tmux-palette, yazi. Plain config repos cloned into `~/.config/<app>`. See [Externals and ignore rules](externals-and-ignore.md#chezmoiexternals). |
|
||||
| `.chezmoiexternals/` | — | One external per app or tool: agent-root, freellmapi, ghostty, herdr, kitty, omarchy, omp, television, tmux, tmux-palette, yazi. Mostly plain config repos cloned into `~/.config/<app>`. See [Externals and ignore rules](externals-and-ignore.md#chezmoiexternals). |
|
||||
| `.gitmodules` | — | `dot_config/hypr` and `dot_config/fish` are git submodules. See [Submodules](externals-and-ignore.md#submodules). |
|
||||
| `.chezmoiignore.tmpl` | — | Target paths chezmoi skips; gates the Linux desktop layer by role. |
|
||||
| `.chezmoiscripts/`, root `run_*` | — | Install and housekeeping scripts. See [Apply scripts](scripts.md). |
|
||||
|
||||
@@ -42,7 +42,7 @@ To update explicitly, run `chezmoi apply --refresh-externals` (`-R`), which pull
|
||||
|
||||
## .chezmoiexternals/
|
||||
|
||||
Per-app config repos, one file each in `.chezmoiexternals/<app>.toml`. Each is a `git-repo` external with `refreshPeriod = "168h"` and `pull.args = ["--ff-only"]`, so `chezmoi apply` fast-forwards them at most once a week. The repos hold plain config files (no chezmoi prefixes) and are not part of `chezmoi managed` contents. The exception is [`agent-root.toml`](#agent-root), a single downloaded script.
|
||||
Per-app config repos, one file each in `.chezmoiexternals/<app>.toml`. Each is a `git-repo` external with `refreshPeriod = "168h"` and `pull.args = ["--ff-only"]`, so `chezmoi apply` fast-forwards them at most once a week. The repos hold plain config files (no chezmoi prefixes) and are not part of `chezmoi managed` contents. The exceptions are [`agent-root.toml`](#agent-root), a single downloaded script, and [`freellmapi.toml`](#freellmapi), an upstream app checkout.
|
||||
|
||||
**.chezmoiexternals/tmux.toml**
|
||||
|
||||
@@ -83,6 +83,10 @@ Edit these in place under `~/.config/<app>`, then commit and push from that chec
|
||||
refreshPeriod = "168h"
|
||||
```
|
||||
|
||||
### freellmapi
|
||||
|
||||
`freellmapi.toml` clones upstream [tashfeenahmed/freellmapi](https://github.com/tashfeenahmed/freellmapi) to `~/.local/share/freellmapi`, same `git-repo` settings as the config repos. Nothing is edited there: [`free-router`](../ai/free-router.md) runs it, installing deps and building the dashboard into gitignored paths, so the checkout stays clean and `--ff-only` pulls apply. Linux desktop/laptop roles only ([role gate](#chezmoiignoretmpl)), together with its service.
|
||||
|
||||
## Submodules
|
||||
|
||||
Two source directories are git submodules, listed in `.gitmodules`. They keep chezmoi source naming and are applied in symlink mode like the rest of the tree. `chezmoi init` clones them.
|
||||
@@ -162,10 +166,15 @@ mise tool versions stay local to each machine and are not managed.
|
||||
.local/bin/kitty-day-save
|
||||
.local/bin/cp-appid
|
||||
.local/bin/projects-sync
|
||||
.local/bin/free-router
|
||||
.local/share/freellmapi
|
||||
.local/share/freellmapi/**
|
||||
.config/free-router
|
||||
.config/free-router/**
|
||||
{{ end }}
|
||||
```
|
||||
|
||||
The block is emitted, and these paths skipped, unless the OS is `linux` and the role is `desktop` or `laptop`. `server` machines, and any non-Linux machine regardless of role, never receive the Hyprland config (`dot_config/hypr/`), the Waybar config (`dot_config/waybar/`), the uwsm environment (`dot_config/uwsm/`), fontconfig (`dot_config/fontconfig/fonts.conf`), default-app associations (`dot_config/mimeapps.list`, `dot_config/xdg-terminals.list`), the systemd user units (`dot_config/systemd/user/kitty-day-save.service`, `projects-sync.service`, `projects-sync.timer`), or the `agent-root`, `cp-appid`, `kitty-day-save` and `projects-sync` scripts.
|
||||
The block is emitted, and these paths skipped, unless the OS is `linux` and the role is `desktop` or `laptop`. `server` machines, and any non-Linux machine regardless of role, never receive the Hyprland config (`dot_config/hypr/`), the Waybar config (`dot_config/waybar/`), the uwsm environment (`dot_config/uwsm/`), fontconfig (`dot_config/fontconfig/fonts.conf`), default-app associations (`dot_config/mimeapps.list`, `dot_config/xdg-terminals.list`), the systemd user units (`dot_config/systemd/user/kitty-day-save.service`, `projects-sync.service`, `projects-sync.timer`, `free-router.service`), the `agent-root`, `cp-appid`, `kitty-day-save`, `projects-sync` and `free-router` scripts, or free-router's settings (`dot_config/free-router/env`) and freellmapi checkout.
|
||||
|
||||
| Machine | Desktop layer |
|
||||
| --- | --- |
|
||||
|
||||
@@ -34,7 +34,7 @@ Eight scripts run during `chezmoi apply`. Six live in `.chezmoiscripts/`, a dire
|
||||
|
||||
7. **30-user-services (after, once)**
|
||||
|
||||
Enable systemd user units (kitty-day-save, projects-sync.timer).
|
||||
Enable systemd user units (kitty-day-save, projects-sync.timer, free-router).
|
||||
|
||||
8. **40-post (after, onchange)**
|
||||
|
||||
@@ -136,6 +136,7 @@ The strata flags install the binary, `~/.local/share/applications/io.github.lgse
|
||||
|
||||
- Same check for `~/.config/systemd/user/kitty-day-save.service` (from `dot_config/systemd/user/`, so Linux desktop/laptop roles only), then `systemctl --user enable --now kitty-day-save.service`. See [kitty-day-save](../shell/scripts.md#kitty-day-save).
|
||||
- Same check for `~/.config/systemd/user/projects-sync.timer`, then `systemctl --user enable --now projects-sync.timer`. See [projects-sync](../shell/scripts.md#projects-sync).
|
||||
- Same check for `~/.config/systemd/user/free-router.service`, then `systemctl --user enable --now free-router.service`. See [free-router](../ai/free-router.md).
|
||||
- On non-Linux systems it renders to `#!/bin/sh` / `exit 0`.
|
||||
|
||||
Other personal automation units (brain-lens, omp-loop, rclone, hermes, voxtype) are deliberately not enabled; the script comment says to enable them by hand per machine. Their unit files are not in the repo, so they have to exist on the machine already.
|
||||
|
||||
@@ -5,7 +5,7 @@ The Linux desktop layer: Omarchy, the Hyprland config on top of it, the Waybar b
|
||||
> [!NOTE]
|
||||
> **Role gating**
|
||||
>
|
||||
> `.chezmoiignore.tmpl` skips the Linux desktop layer unless the machine runs Linux **and** its role is `desktop` or `laptop`. The gated targets are `~/.config/hypr`, `~/.config/waybar`, `~/.config/uwsm`, `~/.config/fontconfig`, `~/.config/mimeapps.list`, `~/.config/xdg-terminals.list`, `~/.config/systemd`, `~/.local/bin/agent-root`, `~/.local/bin/kitty-day-save`, `~/.local/bin/cp-appid`, and `~/.local/bin/projects-sync`. On the `server` role these are not applied.
|
||||
> `.chezmoiignore.tmpl` skips the Linux desktop layer unless the machine runs Linux **and** its role is `desktop` or `laptop`. The gated targets are `~/.config/hypr`, `~/.config/waybar`, `~/.config/uwsm`, `~/.config/fontconfig`, `~/.config/mimeapps.list`, `~/.config/xdg-terminals.list`, `~/.config/systemd`, `~/.config/free-router`, `~/.local/bin/agent-root`, `~/.local/bin/kitty-day-save`, `~/.local/bin/cp-appid`, `~/.local/bin/projects-sync`, `~/.local/bin/free-router`, and `~/.local/share/freellmapi`. On the `server` role these are not applied.
|
||||
>
|
||||
> The `omarchy-*` scripts in `~/.local/bin` are **not** listed there, so chezmoi applies them on every role.
|
||||
|
||||
|
||||
@@ -26,7 +26,7 @@ Interactive shells only (`[[ $- != *i* ]] && return`). In order:
|
||||
|
||||
1. **PATH.** Prepends `~/.local/bin` if missing, sources `~/.cargo/env` if present, and appends `~/.maestro/bin` if it exists. Exports `GHQ_ROOT` and `CODE_ROOT` as `~/Code`.
|
||||
2. **Go.** `GOPATH=~/.local/share/go`, `GOBIN=$GOPATH/bin`, and `GOBIN` prepended to `PATH`. mise provides `GOROOT`. The same two values are also in `~/.config/go/env` (`dot_config/go/env.tmpl`), Go's own config file, so every `go` invocation uses them even without these variables (cron, `sudo -u`, `env -i`, GUI apps).
|
||||
3. **Android SDK.** `ANDROID_HOME` and `ANDROID_SDK_ROOT` set to `~/.local/share/android`. `platform-tools`, `cmdline-tools/latest/bin`, and `emulator` are appended if they exist. Exports `GITHUB_TOKEN` from `gh auth token` unless it is already set.
|
||||
3. **Android SDK.** `ANDROID_HOME` and `ANDROID_SDK_ROOT` set to `~/.local/share/android`. `platform-tools`, `cmdline-tools/latest/bin`, and `emulator` are appended if they exist. Exports `GITHUB_TOKEN` from `gh auth token` unless it is already set, and every variable in `~/.config/free-router/env` (`FREE_ROUTER_PORT`, `FREE_ROUTER_API_KEY`; see [free-router](../ai/free-router.md)) if that file exists.
|
||||
4. **Aliases.** Sources `~/.config/shell/aliases.sh`.
|
||||
5. **Functions.** Sources every `~/.config/bash/functions/*.bash`.
|
||||
6. **Flyline.** If `~/.local/lib/libflyline.so` exists, loads it with `enable -f … flyline` for richer line editing.
|
||||
@@ -39,7 +39,7 @@ Interactive shells only (`[[ $- != *i* ]] && return`). In order:
|
||||
|
||||
1. **History.** `HISTFILE=~/.zsh_history`, `HISTSIZE=10000`, `SAVEHIST=10000`, `setopt SHARE_HISTORY HIST_IGNORE_DUPS`.
|
||||
2. **Completion.** `autoload -Uz compinit && compinit`.
|
||||
3. The same PATH, Go, Android, and `GITHUB_TOKEN` blocks as `.bashrc`, including `GHQ_ROOT` and `CODE_ROOT`.
|
||||
3. The same PATH, Go, Android, `GITHUB_TOKEN`, and free-router blocks as `.bashrc`, including `GHQ_ROOT` and `CODE_ROOT`.
|
||||
4. Sources `~/.config/shell/aliases.sh`.
|
||||
5. Sources every `~/.config/zsh/functions/*.zsh`.
|
||||
6. Sources `~/.config/zsh/keybinds.zsh`.
|
||||
|
||||
@@ -81,6 +81,7 @@ Runs for every shell, including non-interactive ones, so scripts get the same `P
|
||||
| `GITEA_LOGIN` | `prad.codes` |
|
||||
| `GITLAB_HOST` | `git.sonr.org` |
|
||||
| `GITHUB_TOKEN` | `gh auth token`, unless already set and only if `gh` is logged in |
|
||||
| `FREE_ROUTER_PORT`, `FREE_ROUTER_API_KEY` | each `KEY=VALUE` line of `~/.config/free-router/env`, if it exists. See [free-router](../ai/free-router.md) |
|
||||
| `EDITOR`, `VISUAL` | `nvim` |
|
||||
| `BROWSER` | `/usr/bin/brave-origin` if that binary exists |
|
||||
| `CHROME_EXECUTABLE` | `/usr/bin/chromium` if it exists |
|
||||
|
||||
@@ -35,6 +35,7 @@ Several scripts below also exist as fish functions of the same name (`ai`, `c2p`
|
||||
| [`c2p`](#c2p) | sh | code2prompt with an fzf-picked template |
|
||||
| [`extract_agents`](#extract_agents) | sh | Copy all `AGENTS.md` files into a directory tree |
|
||||
| [`fab`](#fab) | sh | fzf picker for Fabric patterns |
|
||||
| [`free-router`](#free-router) | bun | Run the freellmapi LLM router on `localhost:$FREE_ROUTER_PORT` |
|
||||
| [`gdc`](#gdc) | sh | Discard all git changes after confirmation |
|
||||
| [`gh-sonr-org-migrate`](#gh-sonr-org-migrate) | bash | Consolidate `*sonr*` GitHub orgs into `sonr-io` |
|
||||
| [`ghg-clone`](#ghg-clone) | sh | `ghq get` and print the clone path |
|
||||
@@ -308,6 +309,16 @@ Run by `projects-sync.timer` (`dot_config/systemd/user/`, 30 s after login and e
|
||||
|
||||
**Needs:** GNU `find`. Linux desktop/laptop roles only (`.chezmoiignore.tmpl`).
|
||||
|
||||
## free-router
|
||||
|
||||
```sh
|
||||
free-router
|
||||
```
|
||||
|
||||
Runs the [freellmapi](https://github.com/tashfeenahmed/freellmapi) router under Bun on `127.0.0.1:$FREE_ROUTER_PORT`, with its API key pinned to `$FREE_ROUTER_API_KEY`. Run by `free-router.service`, which [`30-user-services`](../chezmoi/scripts.md#30-user-services) enables. See [free-router](../ai/free-router.md).
|
||||
|
||||
**Needs:** `bun`, `git`, the `~/.local/share/freellmapi` external. Linux desktop/laptop roles only (`.chezmoiignore.tmpl`).
|
||||
|
||||
## mtime-sort
|
||||
|
||||
```sh
|
||||
|
||||
Reference in new issue
Block a user