feat(skill): add DuckDB attach-db, docs, query, and mise-expert skills

.github/skills/attach-db/SKILL.md

@@ -0,0 +1,158 @@
+---
+name: attach-db
+description: >
+  Attach a DuckDB database file for use with /duckdb-skills:query.
+  Explores the schema (tables, columns, row counts) and writes a SQL state file
+  so subsequent queries can restore this session automatically via duckdb -init.
+argument-hint: <path-to-database.duckdb>
+allowed-tools: Bash
+---
+
+You are helping the user attach a DuckDB database file for interactive querying.
+
+Database path given: `$0`
+
+Follow these steps in order, stopping and reporting clearly if any step fails.
+
+**State file convention**: see the "Resolve state directory" section below. All skills share a single `state.sql` file per project. Once resolved, any skill can use it with `duckdb -init "$STATE_DIR/state.sql" -c "<QUERY>"`.
+
+## Step 1 — Resolve the database path
+
+If `$0` is a relative path, resolve it against `$PWD` to get an absolute path (`RESOLVED_PATH`).
+
+```bash
+RESOLVED_PATH="$(cd "$(dirname "$0")" 2>/dev/null && pwd)/$(basename "$0")"
+```
+
+Check the file exists:
+
+```bash
+test -f "$RESOLVED_PATH"
+```
+
+- **File exists** -> continue to Step 2.
+- **File not found** -> ask the user if they want to create a new empty database (DuckDB creates the file on first write). If yes, continue. If no, stop.
+
+## Step 2 — Check DuckDB is installed
+
+```bash
+command -v duckdb
+```
+
+If not found, delegate to `/duckdb-skills:install-duckdb` and then continue.
+
+## Step 3 — Validate the database
+
+```bash
+duckdb "$RESOLVED_PATH" -c "PRAGMA version;"
+```
+
+- **Success** -> continue.
+- **Failure** -> report the error clearly (e.g. corrupt file, not a DuckDB database) and stop.
+
+## Step 4 — Explore the schema
+
+First, list all tables:
+
+```bash
+duckdb "$RESOLVED_PATH" -csv -c "
+SELECT table_name, estimated_size
+FROM duckdb_tables()
+ORDER BY table_name;
+"
+```
+
+If the database has **no tables**, note that it is empty and skip to Step 5.
+
+For each table discovered (up to 20), run:
+
+```bash
+duckdb "$RESOLVED_PATH" -csv -c "
+DESCRIBE <table_name>;
+SELECT count() AS row_count FROM <table_name>;
+"
+```
+
+Collect the column definitions and row counts for the summary.
+
+## Step 5 — Resolve the state directory
+
+Check if a state file already exists in either location:
+
+```bash
+# Option 1: in the project directory
+test -f .duckdb-skills/state.sql && STATE_DIR=".duckdb-skills"
+
+# Option 2: in the home directory, scoped by project root path
+PROJECT_ROOT="$(git rev-parse --show-toplevel 2>/dev/null || echo "$PWD")"
+PROJECT_ID="$(echo "$PROJECT_ROOT" | tr '/' '-')"
+test -f "$HOME/.duckdb-skills/$PROJECT_ID/state.sql" && STATE_DIR="$HOME/.duckdb-skills/$PROJECT_ID"
+```
+
+If **neither exists**, ask the user:
+
+> Where would you like to store the DuckDB session state for this project?
+>
+> 1. **In the project directory** (`.duckdb-skills/state.sql`) — colocated with the project, easy to find. You can choose to gitignore it.
+> 2. **In your home directory** (`~/.duckdb-skills/<project-id>/state.sql`) — keeps the project directory clean.
+
+Based on their choice:
+
+**Option 1:**
+```bash
+STATE_DIR=".duckdb-skills"
+mkdir -p "$STATE_DIR"
+```
+Then ask: *"Would you like to gitignore `.duckdb-skills/`?"* If yes:
+```bash
+echo '.duckdb-skills/' >> .gitignore
+```
+
+**Option 2:**
+```bash
+PROJECT_ROOT="$(git rev-parse --show-toplevel 2>/dev/null || echo "$PWD")"
+PROJECT_ID="$(echo "$PROJECT_ROOT" | tr '/' '-')"
+STATE_DIR="$HOME/.duckdb-skills/$PROJECT_ID"
+mkdir -p "$STATE_DIR"
+```
+
+## Step 6 — Append to the state file
+
+`state.sql` is a shared, accumulative init file used by all duckdb-skills. It may already contain macros, LOAD statements, secrets, or other ATTACH statements written by other skills. **Never overwrite it** — always check for duplicates and append.
+
+Derive the database alias from the filename without extension (e.g. `my_data.duckdb` → `my_data`). Check if this ATTACH already exists:
+
+```bash
+grep -q "ATTACH.*RESOLVED_PATH" "$STATE_DIR/state.sql" 2>/dev/null
+```
+
+If not already present, append:
+
+```bash
+cat >> "$STATE_DIR/state.sql" <<'STATESQL'
+ATTACH IF NOT EXISTS 'RESOLVED_PATH' AS my_data;
+USE my_data;
+STATESQL
+```
+
+Replace `RESOLVED_PATH` and `my_data` with the actual values. If the alias would conflict with an existing one in the file, ask the user for a name.
+
+## Step 7 — Verify the state file works
+
+```bash
+duckdb -init "$STATE_DIR/state.sql" -c "SHOW TABLES;"
+```
+
+If this fails, fix the state file and retry.
+
+## Step 8 — Report
+
+Summarize for the user:
+
+- **Database path**: the resolved absolute path
+- **Alias**: the database alias used in the state file
+- **State file**: the resolved `STATE_DIR/state.sql` path
+- **Tables**: name, column count, row count for each table (or note the DB is empty)
+- Confirm the database is now active for `/duckdb-skills:query`
+
+If the database is empty, suggest creating tables or importing data.

.github/skills/duckdb-docs/SKILL.md

@@ -0,0 +1,151 @@
+---
+name: duckdb-docs
+description: >
+  Search DuckDB and DuckLake documentation and blog posts. Returns relevant
+  doc chunks for a question or keyword using full-text search against a
+  locally cached index.
+argument-hint: <question or keyword>
+allowed-tools: Bash
+---
+
+You are helping the user find relevant DuckDB or DuckLake documentation.
+
+Query: `$@`
+
+Follow these steps in order.
+
+## Step 1 — Check DuckDB is installed
+
+```bash
+command -v duckdb
+```
+
+If not found, delegate to `/duckdb-skills:install-duckdb` and then continue.
+
+## Step 2 — Ensure required extensions are installed
+
+```bash
+duckdb :memory: -c "INSTALL httpfs; INSTALL fts;"
+```
+
+If this fails, report the error and stop.
+
+## Step 3 — Choose the data source and extract search terms
+
+The query is: `$@`
+
+### Data source selection
+
+There are two search indexes available:
+
+| Index | Remote URL | Local cache filename | Versions | Use when |
+|-------|-----------|---------------------|----------|----------|
+| **DuckDB docs + blog** | `https://duckdb.org/data/docs-search.duckdb` | `duckdb-docs.duckdb` | `lts`, `current`, `blog` | Default — any DuckDB question |
+| **DuckLake docs** | `https://ducklake.select/data/docs-search.duckdb` | `ducklake-docs.duckdb` | `stable`, `preview` | Query mentions DuckLake, catalogs, or DuckLake-specific features |
+
+Both indexes share the same schema:
+
+| Column | Type | Description |
+|--------|------|-------------|
+| `chunk_id` | `VARCHAR` (PK) | e.g. `stable/sql/functions/numeric#absx` |
+| `page_title` | `VARCHAR` | Page title from front matter |
+| `section` | `VARCHAR` | Section heading (null for page intros) |
+| `breadcrumb` | `VARCHAR` | e.g. `SQL > Functions > Numeric` |
+| `url` | `VARCHAR` | URL path with anchor |
+| `version` | `VARCHAR` | See table above |
+| `text` | `TEXT` | Full markdown of the chunk |
+
+By default, search **DuckDB docs** and filter to `version = 'lts'`. Use different versions when:
+
+- The user explicitly asks about `current`/nightly features → `version = 'current'`
+- The user asks about a blog post or wants background/motivation → `version = 'blog'`
+- The user asks about DuckLake → search the DuckLake index with `version = 'stable'`
+- When unsure, omit the version filter to search across all versions.
+
+### Search terms
+
+If the input is a **natural language question** (e.g. "how do I find the most frequent value"), extract the key technical terms (nouns, function names, SQL keywords) to form a compact BM25 query string. Drop stop words like "how", "do", "I", "the".
+
+If the input is already a **function name or technical term** (e.g. `arg_max`, `GROUP BY ALL`), use it as-is.
+
+Use the extracted terms as `SEARCH_QUERY` in the next step.
+
+## Step 4 — Ensure local cache is fresh
+
+The cache lives at `$HOME/.duckdb/docs/CACHE_FILENAME` (where `CACHE_FILENAME` is `duckdb-docs.duckdb` or `ducklake-docs.duckdb` per Step 3).
+
+First, ensure the directory exists:
+
+```bash
+mkdir -p "$HOME/.duckdb/docs"
+```
+
+Then check whether the cache file exists and is fresh (≤2 days old):
+
+```bash
+CACHE_FILE="$HOME/.duckdb/docs/CACHE_FILENAME"
+if [ -f "$CACHE_FILE" ]; then
+    MTIME=$(stat -f %m "$CACHE_FILE" 2>/dev/null || stat -c %Y "$CACHE_FILE")
+    CACHE_AGE_DAYS=$(( ( $(date +%s) - MTIME ) / 86400 ))
+else
+    CACHE_AGE_DAYS=999
+fi
+echo "Cache age: $CACHE_AGE_DAYS days"
+```
+
+**If `CACHE_AGE_DAYS` ≤ 2** → skip to Step 5.
+
+**Otherwise** (stale or missing) → fetch the index:
+
+```bash
+duckdb -c "
+LOAD httpfs;
+LOAD fts;
+ATTACH 'REMOTE_URL' AS remote (READ_ONLY);
+ATTACH '$HOME/.duckdb/docs/CACHE_FILENAME.tmp' AS tmp;
+COPY FROM DATABASE remote TO tmp;
+" && mv "$HOME/.duckdb/docs/CACHE_FILENAME.tmp" "$HOME/.duckdb/docs/CACHE_FILENAME"
+```
+
+Replace `REMOTE_URL` and `CACHE_FILENAME` per Step 3. If the fetch fails (network error), report the error and stop.
+
+## Step 5 — Search the docs
+
+```bash
+duckdb "$HOME/.duckdb/docs/CACHE_FILENAME" -readonly -json -c "
+LOAD fts;
+SELECT
+    chunk_id, page_title, section, breadcrumb, url, version, text,
+    fts_main_docs_chunks.match_bm25(chunk_id, 'SEARCH_QUERY') AS score
+FROM docs_chunks
+WHERE score IS NOT NULL
+  AND version = 'VERSION'
+ORDER BY score DESC
+LIMIT 8;
+"
+```
+
+Replace `CACHE_FILENAME`, `SEARCH_QUERY`, and `VERSION` per Step 3. Remove the `AND version = 'VERSION'` line if searching across all versions.
+
+If the user's question could benefit from both DuckDB docs and blog results, run two queries (one with `version = 'stable'`, one with `version = 'blog'`) or omit the version filter entirely.
+
+## Step 6 — Handle errors
+
+- **Extension not installed** (`httpfs` or `fts` not found): run `duckdb :memory: -c "INSTALL httpfs; INSTALL fts;"` and retry.
+- **ATTACH fails / network unreachable**: inform the user that the docs index is unavailable and suggest checking their internet connection. The DuckDB index is hosted at `https://duckdb.org/data/docs-search.duckdb` and the DuckLake index at `https://ducklake.select/data/docs-search.duckdb`.
+- **No results** (all scores NULL or empty result set): try broadening the query — drop the least specific term, or try a single-word version of the query — then retry Step 5. If still no results, tell the user no matching documentation was found and suggest visiting https://duckdb.org/docs or https://ducklake.select/docs directly.
+
+## Step 7 — Present results
+
+For each result chunk returned (ordered by score descending), format as:
+
+```
+### {section} — {page_title}
+{url}
+
+{text}
+
+---
+```
+
+After presenting all chunks, synthesize a concise answer to the user's original question (`$@`) based on the retrieved documentation. If the chunks directly answer the question, lead with the answer before showing the sources.

.github/skills/query/SKILL.md

@@ -0,0 +1,208 @@
+---
+name: query
+description: >
+  Run SQL queries against the attached DuckDB database or ad-hoc against files.
+  Accepts raw SQL or natural language questions. Uses DuckDB Friendly SQL idioms.
+argument-hint: <SQL or question> [--file path]
+allowed-tools: Bash
+---
+
+You are helping the user query data using DuckDB.
+
+Input: `$@`
+
+Follow these steps in order.
+
+## Step 1 — Resolve state and determine the mode
+
+Look for an existing state file in either location:
+
+```bash
+STATE_DIR=""
+test -f .duckdb-skills/state.sql && STATE_DIR=".duckdb-skills"
+PROJECT_ROOT="$(git rev-parse --show-toplevel 2>/dev/null || echo "$PWD")"
+PROJECT_ID="$(echo "$PROJECT_ROOT" | tr '/' '-')"
+test -f "$HOME/.duckdb-skills/$PROJECT_ID/state.sql" && STATE_DIR="$HOME/.duckdb-skills/$PROJECT_ID"
+```
+
+If found, verify the databases it references are still accessible:
+
+```bash
+duckdb -init "$STATE_DIR/state.sql" -c "SHOW DATABASES;"
+```
+
+Now determine the mode:
+
+- **Ad-hoc mode** if: the `--file` flag is present, or the SQL references file paths/literals (e.g. `FROM 'data.csv'`), or `STATE_DIR` is empty.
+- **Session mode** if: `STATE_DIR` is set and the input references table names, is natural language, or is SQL without file references.
+
+If no state file exists and no file is referenced, fall back to ad-hoc mode against `:memory:` — the user must reference files directly in their SQL.
+
+If the state file exists but any ATTACH in it fails, warn the user and fall back to ad-hoc mode.
+
+## Step 2 — Check DuckDB is installed
+
+```bash
+command -v duckdb
+```
+
+If not found, delegate to `/duckdb-skills:install-duckdb` and then continue.
+
+## Step 3 — Generate SQL if needed
+
+If the input is natural language (not valid SQL), generate SQL using the Friendly SQL reference below.
+
+In **session mode**, first retrieve the schema to inform query generation:
+
+```bash
+duckdb -init "$STATE_DIR/state.sql" -csv -c "
+SELECT table_name FROM duckdb_tables() ORDER BY table_name;
+"
+```
+
+Then for relevant tables:
+
+```bash
+duckdb -init "$STATE_DIR/state.sql" -csv -c "DESCRIBE <table_name>;"
+```
+
+Use the schema context and the Friendly SQL reference to generate the most appropriate query.
+
+## Step 4 — Estimate result size
+
+Before executing, estimate whether the query could produce a very large result that would
+consume excessive tokens when returned to this conversation.
+
+**Session mode** — check row counts for the tables involved:
+
+```bash
+duckdb -init "$STATE_DIR/state.sql" -csv -c "
+SELECT table_name, estimated_size, column_count
+FROM duckdb_tables()
+WHERE table_name IN ('<table1>', '<table2>');
+"
+```
+
+**Ad-hoc mode** — probe the source:
+
+```bash
+duckdb :memory: -csv -c "
+SET allowed_paths=['FILE_PATH'];
+SET enable_external_access=false;
+SET allow_persistent_secrets=false;
+SET lock_configuration=true;
+SELECT count() AS row_count FROM 'FILE_PATH';
+"
+```
+
+**Evaluate**:
+- If the query already has a `LIMIT`, `count()`, or other aggregation that bounds the output -> safe, proceed.
+- If the source has **>1M rows** and the query has no LIMIT or aggregation -> tell the user:
+  *"This query would return a very large result set. Displaying it here would consume a lot of tokens and increase cost. I'd recommend adding `LIMIT 1000` or an aggregation to keep the output manageable."*
+  Ask for confirmation before running as-is.
+- If the data size is **>10 GB** -> additionally warn:
+  *"This table is over 10 GB — the query may take a while to complete."*
+  Proceed if the user confirms.
+
+Skip this step for queries that are intrinsically bounded (e.g. `DESCRIBE`, `SUMMARIZE`, aggregations, `count()`).
+
+## Step 5 — Execute the query
+
+**Ad-hoc mode** (sandboxed — only the referenced file is accessible):
+
+```bash
+duckdb :memory: -csv <<'SQL'
+SET allowed_paths=['FILE_PATH'];
+SET enable_external_access=false;
+SET allow_persistent_secrets=false;
+SET lock_configuration=true;
+<QUERY>;
+SQL
+```
+
+Replace `FILE_PATH` with the actual file path extracted from the query or `--file` argument.
+If multiple files are referenced, include all paths in the `allowed_paths` list.
+
+**Session mode** (user-trusted database):
+
+```bash
+duckdb -init "$STATE_DIR/state.sql" -csv -c "<QUERY>"
+```
+
+For multi-line queries, use a heredoc with `-init`:
+
+```bash
+duckdb -init "$STATE_DIR/state.sql" -csv <<'SQL'
+<QUERY>;
+SQL
+```
+
+Always use heredocs (`<<'SQL'`) for multi-line queries to avoid shell quoting issues.
+
+## Step 6 — Handle errors
+
+- **Syntax error**: show the error, suggest a corrected query, and re-run.
+- **Missing extension** (e.g. `Extension "X" not loaded`): delegate to `/duckdb-skills:install-duckdb <ext>`, then retry.
+- **Table not found** (session mode): list available tables with `FROM duckdb_tables()` and suggest corrections.
+- **File not found** (ad-hoc mode): use `find "$PWD" -name "<filename>" 2>/dev/null` to locate the file and suggest the corrected path.
+- **Persistent or unclear DuckDB error**: use `/duckdb-skills:duckdb-docs <error message or relevant keywords>` to search the documentation for guidance, then apply the fix and retry.
+
+## Step 7 — Present results
+
+Show the query output to the user. If the result has more than 100 rows, note the truncation and suggest adding `LIMIT` to the query.
+
+For natural language questions, also provide a brief interpretation of the results.
+
+---
+
+## DuckDB Friendly SQL Reference
+
+When generating SQL, prefer these idiomatic DuckDB constructs:
+
+### Compact clauses
+- **FROM-first**: `FROM table WHERE x > 10` (implicit `SELECT *`)
+- **GROUP BY ALL**: auto-groups by all non-aggregate columns
+- **ORDER BY ALL**: orders by all columns for deterministic results
+- **SELECT * EXCLUDE (col1, col2)**: drop columns from wildcard
+- **SELECT * REPLACE (expr AS col)**: transform a column in-place
+- **UNION ALL BY NAME**: combine tables with different column orders
+- **Percentage LIMIT**: `LIMIT 10%` returns a percentage of rows
+- **Prefix aliases**: `SELECT x: 42` instead of `SELECT 42 AS x`
+- **Trailing commas** allowed in SELECT lists
+
+### Query features
+- **count()**: no need for `count(*)`
+- **Reusable aliases**: use column aliases in WHERE / GROUP BY / HAVING
+- **Lateral column aliases**: `SELECT i+1 AS j, j+2 AS k`
+- **COLUMNS(*)**: apply expressions across columns; supports regex, EXCLUDE, REPLACE, lambdas
+- **FILTER clause**: `count() FILTER (WHERE x > 10)` for conditional aggregation
+- **GROUPING SETS / CUBE / ROLLUP**: advanced multi-level aggregation
+- **Top-N per group**: `max(col, 3)` returns top 3 as a list; also `arg_max(arg, val, n)`, `min_by(arg, val, n)`
+- **DESCRIBE table_name**: schema summary (column names and types)
+- **SUMMARIZE table_name**: instant statistical profile
+- **PIVOT / UNPIVOT**: reshape between wide and long formats
+- **SET VARIABLE x = expr**: define SQL-level variables, reference with `getvariable('x')`
+
+### Data import
+- **Direct file queries**: `FROM 'file.csv'`, `FROM 'data.parquet'`
+- **Globbing**: `FROM 'data/part-*.parquet'` reads multiple files
+- **Auto-detection**: CSV headers and schemas are inferred automatically
+
+### Expressions and types
+- **Dot operator chaining**: `'hello'.upper()` or `col.trim().lower()`
+- **List comprehensions**: `[x*2 FOR x IN list_col]`
+- **List/string slicing**: `col[1:3]`, negative indexing `col[-1]`
+- **STRUCT.* notation**: `SELECT s.* FROM (SELECT {'a': 1, 'b': 2} AS s)`
+- **Square bracket lists**: `[1, 2, 3]`
+- **format()**: `format('{}->{}', a, b)` for string formatting
+
+### Joins
+- **ASOF joins**: approximate matching on ordered data (e.g. timestamps)
+- **POSITIONAL joins**: match rows by position, not keys
+- **LATERAL joins**: reference prior table expressions in subqueries
+
+### Data modification
+- **CREATE OR REPLACE TABLE**: no need for `DROP TABLE IF EXISTS` first
+- **CREATE TABLE ... AS SELECT (CTAS)**: create tables from query results
+- **INSERT INTO ... BY NAME**: match columns by name, not position
+- **INSERT OR IGNORE INTO / INSERT OR REPLACE INTO**: upsert patterns