fix: filter the pile on chat flags and fetch snippets from the message index

Verified against Beeper Desktop 4.3.73 with a real token:

- inbox=primary combined with unreadOnly=true returned 0 of 78 genuinely
  unread chats, and includeMuted=false returned muted chats anyway. Both
  params are gone; muted/archived/low-priority are now decided here from
  isMuted/isArchived/isLowPriority, and a hand-marked-unread chat counts.
- Snippets now come from GET /v1/messages/search (1.1 ms/chat) instead of
  GET /v1/chats/{id}/messages (145 ms/chat), which is kept as the fallback
  for chats the index has not reached. list --limit 50 went 3.33s -> 0.16s.

CHANGELOG.md

@@ -16,7 +16,13 @@ First release.
 - Paging with `n`/`p`, page size 1–50, poll interval 15–3600 s.
 - `bin/omarchy-beeper` CLI: `list`, `read`, `read-all`, `open`, `auth`. Every
   command prints one JSON object and exits 0.
-- Muted, archived, and low-priority chats are excluded from the pile.
+- Muted, archived, and low-priority chats are excluded from the pile, filtered
+  on each chat's own flags: Beeper Desktop 4.3.73 ignores `includeMuted=false`
+  and `inbox=primary` hides genuinely unread chats when combined with
+  `unreadOnly=true`.
+- Row snippets come from `GET /v1/messages/search` (~1 ms per chat) with
+  `GET /v1/chats/{chatID}/messages` as the fallback for chats the message
+  index has not reached yet, and only for the rows on screen.
 - Token stored at `~/.config/omarchy-beeper/secrets/token.json` (600, in a 700
   directory), read with `O_NOFOLLOW`, never passed in argv.
 - All HTTP bound to `http://localhost:23373/v1/`, no redirects, 2 MiB body cap

README.md

@@ -91,13 +91,20 @@ One row is one unread **chat**, not one message, sorted newest activity first.
 
 Not in the pile:
 
-- **muted** chats (`includeMuted=false`),
+- **muted** chats,
 - **archived** chats,
 - **low-priority** chats.
 
-The last two come free with `inbox=primary` on `GET /v1/chats/search`, which
-Beeper defines as non-archived and non-low-priority. If you want a muted chat
-to nag you, unmute it in Beeper.
+A chat you marked unread by hand counts even though it has no unread
+messages. If you want a muted chat to nag you, unmute it in Beeper.
+
+The filtering happens in `lib/beeper.py`, on the `isMuted` / `isArchived` /
+`isLowPriority` flags each chat carries — deliberately **not** with the search
+endpoint's own filters. Measured against Beeper Desktop 4.3.73:
+`includeMuted=false` returns muted chats anyway, and `inbox=primary` combined
+with `unreadOnly=true` hid 77 of 78 genuinely unread chats. The per-chat flags
+are exact, so the plugin asks only for `unreadOnly=true` and decides the rest
+itself.
 
 The badge counts unread chats, capped at 200 — a deeper pile is a Beeper
 problem, not a bar problem. When the cap is hit the panel says so. `A`
@@ -160,11 +167,14 @@ Failures look like `{"ok":false,"error":"…"}`. The two you will actually see:
   one means `POST /v1/focus` with the chat id: Beeper Desktop comes forward
   with that conversation selected. There is no browser hand-off and no deep
   link to validate.
-- **Snippets cost one request per visible row.** The chat objects Beeper
-  returns carry no message preview, so the last message is fetched with
-  `GET /v1/chats/{chatID}/messages` — but only for the rows on screen, never
-  for the whole pile. A snippet that fails to load costs the snippet, not the
-  row.
+- **Snippets are index-first, one request per visible row.** The chat objects
+  Beeper returns carry no message preview, so the last message comes from
+  `GET /v1/messages/search?chatIDs=<chat>` — about 1 ms per chat, against
+  ~145 ms for `GET /v1/chats/{chatID}/messages`, which returns a whole page.
+  The slow endpoint is only used as a fallback for chats the index has not
+  caught up with yet (2 of 40 on a warm install; the fallback recovered both).
+  Either way only the rows on screen are fetched, never the whole pile, and a
+  snippet that fails to load costs the snippet, not the row.
 - **Paging is a numeric offset.** The API is cursor-based; the CLI walks the
   cursors once, caps at 200 chats, sorts, and slices. `nextPage` is the next
   offset, so `p` can step back without replaying cursors.

docs/SETUP.md

@@ -17,14 +17,15 @@ its settings.
 
 In Beeper Desktop: **Settings → Integrations → create an access token.**
 
-Read access is all this plugin needs. It calls exactly six endpoints:
+Read access is all this plugin needs. It calls exactly seven endpoints:
 
 | Call | Why |
 | --- | --- |
 | `GET /v1/info` | Tell "Beeper is down" apart from "the token is wrong" |
 | `GET /v1/accounts` | Network names for the chips, and the account count |
-| `GET /v1/chats/search` | The unread pile |
-| `GET /v1/chats/{chatID}/messages` | The last message, for the row snippet |
+| `GET /v1/chats/search` | The unread pile (`unreadOnly=true`) |
+| `GET /v1/messages/search` | The last message of a visible row, for the snippet |
+| `GET /v1/chats/{chatID}/messages` | Snippet fallback for a chat the index has not reached |
 | `POST /v1/chats/{chatID}/read` | Mark a chat read |
 | `POST /v1/focus` | Bring Beeper Desktop forward on a chat |
 

lib/beeper.py

@@ -9,7 +9,8 @@ Endpoints used (https://developers.beeper.com/desktop-api-reference/):
   GET  /v1/info                     server discovery, no auth
   GET  /v1/accounts                 connected chat accounts
   GET  /v1/chats/search             unread chats, cursor paginated
-  GET  /v1/chats/{chatID}/messages  last message for the row snippet
+  GET  /v1/messages/search          last message for the row snippet
+  GET  /v1/chats/{chatID}/messages  snippet fallback for unindexed chats
   POST /v1/chats/{chatID}/read      mark a chat read
   POST /v1/focus                    focus Beeper Desktop, optionally a chat
 """
@@ -35,6 +36,7 @@ from common import (
 
 REQUEST_TIMEOUT = 30
 SEARCH_PAGE = 200  # /v1/chats/search caps limit at 200
+SNIPPET_SEARCH_LIMIT = 5  # a few, so a deleted newest message is not fatal
 SNIPPET_WORKERS = 8
 
 # Message kinds with no body text. Shown as a short placeholder instead.
@@ -181,13 +183,31 @@ def account_label(account: dict) -> str:
     return one_line(str(account.get("network") or account.get("accountID") or ""), 40)
 
 
-def unread_chats(token: str, cap: int = FETCH_CAP, budget: float = 0.0) -> tuple[list[dict], bool]:
-    """Unread chats, newest first.
+def in_pile(chat: dict) -> bool:
+    """Is this chat one the user still has to look at?
 
-    Muted, archived, and low-priority chats are not part of the pile:
-    `inbox=primary` drops archived and low-priority, `includeMuted=false`
-    drops muted. Returns (chats, truncated) where truncated means the walk
-    hit `cap` or `budget` before the API ran out of pages.
+    Muted, archived, and low-priority chats are out. The decision is made
+    here, on the flags the chat object carries, and not with the search
+    endpoint's `inbox`/`includeMuted` filters: measured against Beeper
+    Desktop 4.3.73, `includeMuted=false` returns muted chats anyway, and
+    `inbox=primary` combined with `unreadOnly=true` hides ordinary unread
+    chats (75 of 80 on the machine this was tested on). The per-chat flags
+    are exact.
+    """
+    if chat.get("isMuted") or chat.get("isArchived") or chat.get("isLowPriority"):
+        return False
+    try:
+        unread = int(chat.get("unreadCount") or 0)
+    except (TypeError, ValueError):
+        unread = 0
+    return unread > 0 or bool(chat.get("isMarkedUnread"))
+
+
+def unread_chats(token: str, cap: int = FETCH_CAP, budget: float = 0.0) -> tuple[list[dict], bool]:
+    """Unread chats, newest first, muted/archived/low-priority removed.
+
+    Returns (chats, truncated) where truncated means the walk hit `cap` or
+    `budget` before the API ran out of pages.
     """
     out: list[dict] = []
     seen: set[str] = set()
@@ -195,12 +215,7 @@ def unread_chats(token: str, cap: int = FETCH_CAP, budget: float = 0.0) -> tuple
     truncated = False
     deadline = (time.monotonic() + budget) if budget > 0 else 0.0
     while True:
-        params = {
-            "unreadOnly": True,
-            "includeMuted": False,
-            "inbox": "primary",
-            "limit": min(SEARCH_PAGE, max(1, cap)),
-        }
+        params: dict = {"unreadOnly": True, "limit": SEARCH_PAGE}
         if cursor:
             params["cursor"] = cursor
             params["direction"] = "before"
@@ -217,7 +232,8 @@ def unread_chats(token: str, cap: int = FETCH_CAP, budget: float = 0.0) -> tuple
             if not chat_id or chat_id in seen:
                 continue
             seen.add(chat_id)
-            out.append(chat)
+            if in_pile(chat):
+                out.append(chat)
         if len(out) >= cap:
             truncated = bool(data.get("hasMore")) or len(out) > cap
             del out[cap:]
@@ -232,25 +248,53 @@ def unread_chats(token: str, cap: int = FETCH_CAP, budget: float = 0.0) -> tuple
     return out, truncated
 
 
-def last_message(token: str, chat_id: str) -> dict:
-    """Newest displayable message in a chat, for the row snippet."""
-    data = request(token, "GET", f"/v1/chats/{urllib.parse.quote(chat_id, safe='')}/messages")
+def _newest(data: object) -> dict:
+    """Newest message in a response that is worth showing as a snippet."""
     items = data.get("items") if isinstance(data, dict) else None
     if not isinstance(items, list):
         return {}
     best: dict = {}
-    best_key = ("", 0.0)
+    best_key = (0.0, "")
     for msg in items:
         if not isinstance(msg, dict):
             continue
         if msg.get("isDeleted") or msg.get("isHidden") or msg.get("type") == "REACTION":
             continue
-        key = (str(msg.get("sortKey") or ""), _ts(msg.get("timestamp")))
-        if not best or key[1] > best_key[1] or (key[1] == best_key[1] and key[0] > best_key[0]):
+        key = (_ts(msg.get("timestamp")), str(msg.get("sortKey") or ""))
+        if not best or key > best_key:
             best, best_key = msg, key
     return best
 
 
+def last_message(token: str, chat_id: str) -> dict:
+    """Newest displayable message in a chat, for the row snippet.
+
+    The search index answers in about a millisecond; listing a chat's
+    messages takes ~145 ms because it returns a whole page. So ask the index
+    first and only fall back for the few chats it has not indexed yet
+    (measured: 2 of 40 on a warm install, both recovered by the fallback).
+    """
+    indexed = _newest(
+        request(
+            token,
+            "GET",
+            "/v1/messages/search",
+            {
+                "chatIDs": [chat_id],
+                "limit": SNIPPET_SEARCH_LIMIT,
+                # Neither filter should ever hide the snippet of a chat that
+                # already earned its place in the pile.
+                "excludeLowPriority": False,
+                "includeMuted": True,
+            },
+        )
+    )
+    if indexed:
+        return indexed
+    path = f"/v1/chats/{urllib.parse.quote(chat_id, safe='')}/messages"
+    return _newest(request(token, "GET", path))
+
+
 def message_preview(msg: dict) -> str:
     text = one_line(str(msg.get("text") or ""))
     if text:

tests/test_beeper.py

@@ -2,6 +2,7 @@ from __future__ import annotations
 
 import re
 import unittest
+import urllib.parse
 from contextlib import redirect_stderr
 from io import StringIO
 from unittest.mock import patch
@@ -25,31 +26,35 @@ import common
 
 TOKEN = "test-token"
 SEARCH = "/v1/chats/search"
+MSG_SEARCH = "/v1/messages/search"
 ACCOUNTS = "/v1/accounts"
 FOCUS = "/v1/focus"
 
 
 def messages_path(chat_id: str) -> str:
-    import urllib.parse
-
     return f"/v1/chats/{urllib.parse.quote(chat_id, safe='')}/messages"
 
 
 def read_path(chat_id: str) -> str:
-    import urllib.parse
-
     return f"/v1/chats/{urllib.parse.quote(chat_id, safe='')}/read"
 
 
 def wire(http: FakeHttp, chats: list[dict], accounts: list[dict] | None = None) -> FakeHttp:
-    """Standard happy-path routing: one search page, snippets, marks."""
+    """Standard happy-path routing: one search page, snippets, marks.
+
+    Snippets come from the message index, so that is what is wired here; the
+    per-chat fallback is only routed by the tests that exercise it.
+    """
     http.route("GET", SEARCH, lambda call: search_page(chats))
     http.route("GET", ACCOUNTS, accounts if accounts is not None else [account("a1", "WhatsApp")])
     http.route("POST", FOCUS, {"success": True})
+    http.route(
+        "GET",
+        MSG_SEARCH,
+        lambda call: {"items": [message(call.one("chatIDs"))], "hasMore": False},
+    )
     for item in chats:
-        chat_id = item["id"]
-        http.route("GET", messages_path(chat_id), {"items": [message(chat_id)], "hasMore": False})
-        http.route("POST", read_path(chat_id), item)
+        http.route("POST", read_path(item["id"]), item)
     return http
 
 
@@ -81,13 +86,45 @@ class UrlBindingTests(unittest.TestCase):
 
 
 class ListContractTests(unittest.TestCase):
-    def test_filters_out_muted_archived_and_low_priority(self) -> None:
+    def test_asks_the_api_only_for_unread(self) -> None:
         http = wire(FakeHttp(), [chat("!a:x")])
         run_list(http)
         search = next(c for c in http.calls if c.path == SEARCH)
         self.assertEqual(search.one("unreadOnly"), "true")
-        self.assertEqual(search.one("includeMuted"), "false")
-        self.assertEqual(search.one("inbox"), "primary")
+        self.assertEqual(search.one("limit"), str(beeper.SEARCH_PAGE))
+        # Beeper Desktop 4.3.73 ignores includeMuted and mis-filters
+        # inbox=primary together with unreadOnly, so neither is sent: the
+        # pile is filtered here, on the flags each chat carries.
+        self.assertNotIn("includeMuted", search.query)
+        self.assertNotIn("inbox", search.query)
+
+    def test_muted_archived_and_low_priority_are_not_in_the_pile(self) -> None:
+        chats = [
+            chat("!keep:x", title="Keep"),
+            dict(chat("!muted:x", title="Muted"), isMuted=True),
+            dict(chat("!archived:x", title="Archived"), isArchived=True),
+            dict(chat("!low:x", title="Low"), isLowPriority=True),
+            dict(chat("!read:x", title="Read"), unreadCount=0),
+        ]
+        payload = run_list(wire(FakeHttp(), chats))
+        self.assertEqual([row["subject"] for row in payload["messages"]], ["Keep"])
+        self.assertEqual(payload["unread"], 1)
+        self.assertEqual(sum(box["unread"] for box in payload["inboxes"]), 1)
+
+    def test_a_chat_marked_unread_by_hand_stays_in_the_pile(self) -> None:
+        marked = dict(chat("!marked:x", title="Marked"), unreadCount=0, isMarkedUnread=True)
+        payload = run_list(wire(FakeHttp(), [marked]))
+        self.assertEqual([row["subject"] for row in payload["messages"]], ["Marked"])
+        self.assertEqual(payload["unread"], 1)
+
+    def test_in_pile_is_the_single_rule(self) -> None:
+        self.assertTrue(beeper.in_pile(chat("!a:x")))
+        self.assertFalse(beeper.in_pile(dict(chat("!a:x"), isMuted=True)))
+        self.assertFalse(beeper.in_pile(dict(chat("!a:x"), isArchived=True)))
+        self.assertFalse(beeper.in_pile(dict(chat("!a:x"), isLowPriority=True)))
+        self.assertFalse(beeper.in_pile(dict(chat("!a:x"), unreadCount=0)))
+        self.assertFalse(beeper.in_pile(dict(chat("!a:x"), unreadCount="nonsense")))
+        self.assertTrue(beeper.in_pile(dict(chat("!a:x"), unreadCount=0, isMarkedUnread=True)))
 
     def test_one_row_per_chat_newest_first(self) -> None:
         chats = [
@@ -146,16 +183,39 @@ class ListContractTests(unittest.TestCase):
         chats = [chat(f"!c{i}:x", last_activity=f"2026-02-11T10:{i:02d}:00Z") for i in range(40)]
         http = wire(FakeHttp(), chats)
         run_list(http, limit=5)
-        message_calls = [p for p in http.paths("GET") if p.endswith("/messages")]
-        self.assertEqual(len(message_calls), 5)
+        self.assertEqual(len([p for p in http.paths("GET") if p == MSG_SEARCH]), 5)
+        self.assertEqual([p for p in http.paths("GET") if p.endswith("/messages")], [])
+
+    def test_snippet_search_is_scoped_and_unfiltered(self) -> None:
+        http = wire(FakeHttp(), [chat("!a:x")])
+        run_list(http)
+        call = next(c for c in http.calls if c.path == MSG_SEARCH)
+        self.assertEqual(call.query["chatIDs"], ["!a:x"])
+        self.assertEqual(call.one("limit"), str(beeper.SNIPPET_SEARCH_LIMIT))
+        # A chat already in the pile must never lose its snippet to an
+        # unrelated filter on the message index.
+        self.assertEqual(call.one("excludeLowPriority"), "false")
+        self.assertEqual(call.one("includeMuted"), "true")
+
+    def test_unindexed_chat_falls_back_to_listing_its_messages(self) -> None:
+        http = wire(FakeHttp(), [chat("!a:x")])
+        http.route("GET", MSG_SEARCH, {"items": [], "hasMore": False})
+        http.route(
+            "GET",
+            messages_path("!a:x"),
+            {"items": [message("!a:x", text="from the chat itself")], "hasMore": False},
+        )
+        payload = run_list(http)
+        self.assertEqual(payload["messages"][0]["snippet"], "from the chat itself")
+        self.assertIn(messages_path("!a:x"), http.paths("GET"))
 
     def test_a_failed_snippet_only_loses_the_snippet(self) -> None:
-        chats = [chat("!a:x")]
-        http = wire(FakeHttp(), chats)
-        http.route("GET", messages_path("!a:x"), lambda call: http_error(500, "boom"))
+        http = wire(FakeHttp(), [chat("!a:x")])
+        http.route("GET", MSG_SEARCH, lambda call: http_error(500, "boom"))
         payload = run_list(http)
         self.assertTrue(payload["ok"])
         self.assertEqual(payload["messages"][0]["snippet"], "")
+        self.assertEqual(payload["messages"][0]["from"], "")
         self.assertEqual(payload["messages"][0]["subject"], "Ada Lovelace")
         self.assertEqual(payload["messages"][0]["ts"], 1770804000)
 
@@ -163,7 +223,7 @@ class ListContractTests(unittest.TestCase):
         http = wire(FakeHttp(), [chat("!a:x")])
         http.route(
             "GET",
-            messages_path("!a:x"),
+            MSG_SEARCH,
             {"items": [message("!a:x", text="", kind="IMAGE")], "hasMore": False},
         )
         payload = run_list(http)
@@ -180,10 +240,18 @@ class ListContractTests(unittest.TestCase):
             ),
             dict(message("!a:x", text="👍", timestamp="2026-02-11T13:00:00Z", sort_key="5"), type="REACTION"),
         ]
-        http.route("GET", messages_path("!a:x"), {"items": items, "hasMore": False})
+        http.route("GET", MSG_SEARCH, {"items": items, "hasMore": False})
         payload = run_list(http)
         self.assertEqual(payload["messages"][0]["snippet"], "newest")
 
+    def test_you_are_named_as_the_sender_of_your_own_last_message(self) -> None:
+        http = wire(FakeHttp(), [chat("!a:x")])
+        mine = dict(message("!a:x", text="on my way", sender="Prad"), isSender=True)
+        http.route("GET", MSG_SEARCH, {"items": [mine], "hasMore": False})
+        payload = run_list(http)
+        self.assertEqual(payload["messages"][0]["from"], "You")
+        self.assertEqual(payload["messages"][0]["snippet"], "on my way")
+
     def test_inboxes_and_account_count_come_from_accounts(self) -> None:
         chats = [
             chat("!a:x", account="wa", network="WhatsApp"),