Files
diwan/PROTOCOL.md
T
marc fc0e556b42 diwan: the tool, split out from personal notes content
Contains only the generic diwan CLI and procedure docs (diwan, PROTOCOL.md,
TICK.md, DO.md, DESIGN.md). No personal notes, no content-repo-specific
AGENTS.md -- those now live in a separate content repo (e.g. diwan-dev-notes
for testing, ~/nc/org-agenda for the real thing), discovered at runtime via
git rev-parse --show-toplevel from wherever diwan is invoked.

History reset: this repo previously doubled as both diwan's own dev copy and
a throwaway test-content repo, with 48 commits of the two tangled together.
Starting fresh now that the split is done -- still v0/prototyping.
2026-11-09 10:13:24 +01:00

131 lines
14 KiB
Markdown

# PROTOCOL.md — the diwan protocol
This is diwan's generic contract: the tag system, the `diwan.org` layout, the concurrency/revert model, and the herdr topology. It describes how diwan works, independent of which notes repo it's pointed at — it ships with diwan itself (this repo), not with any one person's content. If you are a dispatched diwan agent (`tick` or `do`), read this file first (you were given its path directly in your dispatch prompt), then the *content* repo's own `AGENTS.md` — a different repo from this one, containing that deployment's specific notes schema and conventions. The two together are the complete contract; this doc never restates anything from a content repo's `AGENTS.md` (Layout, org conventions already in use) and that file never restates anything from here — if you find yourself unsure which owns something, it's a doc bug worth flagging, not something to resolve by guessing.
## `:diwan:` tag
A generic way to call diwan's attention to a heading: tagging something `:diwan:` means "I want diwan to take care of this" (or, when diwan adds it itself, "this needs diwan's attention"), without saying anything about *how*. An ordinary task that's normally just for the user becomes a delegated request the moment the tag is added:
```org
** TODO Archive all DONE entries in the inbox :diwan:
```
Can be added directly by the user, or by diwan itself when it decides some other heading (a chat reply, a plain task found during triage) needs this kind of attention — see the tag-consistency rule below. What actually happens to a `:diwan:`-tagged heading depends on what the request needs, decided by whichever diwan agent processes it (ordinarily `tick`, per `TICK.md`):
- Trivial (a quick lookup, a one-line answer): handled inline immediately, no further tag, no session.
- Non-trivial but self-contained in this heading's own subtree (independent delegated work, nothing that needs writes elsewhere): escalates to a `:session:` tag and a dedicated `diwan do` session.
- Non-trivial *and* inherently cross-cutting — spans multiple files or headings, can't be parallelized, e.g. archiving, bulk cleanup, moving things between files or folders: handled by `diwan tick` itself, directly, possibly across several tick runs — not escalated to a session, since a subtree-scoped `do` agent would just hit the same wall a human doing it alone would (see the Concurrency contract's scope rule). No `:session:` tag for this case.
## `:session:` tag
A heading tagged `:session:` has (or is about to get) a dedicated, persistent agent session handling it — a `DIWAN_ID` property and a `Close diwan session` checkbox item. Note the org tag syntax when it's stacked alongside `:diwan:`: one colon-delimited run, no space (`:diwan:session:`, not `:diwan: :session:`) — the latter isn't valid org tag syntax and won't parse as two tags.
```org
* Gather information about what I could do on my next trip to greece :diwan:session:
:PROPERTIES:
:DIWAN_ID: do-greece-trip
:END:
** [ ] Close diwan session
```
Can be added by the user directly, when they already know a heading needs its own ongoing session, or by diwan (ordinarily `tick`) when it decides a `:diwan:`-tagged request is non-trivial and self-contained enough to delegate (see above). `DIWAN_ID` is a herdr terminal name (see "herdr", below) — not an arbitrary id. It points directly at the live/persisted agent session handling this heading, so "jump in personally" is just attaching to that terminal, no separate lookup needed. A heading keeps the same `DIWAN_ID`/`:session:` tag across multiple tick passes until its `Close diwan session` item is checked off (`[X]`) by the user, at which point the *next* tick retires the session and leaves the property and tag as a historical record (does not delete them).
## `:chat:` tag
A `:chat:` heading is a conversation thread between the user and diwan, always as a child of the heading it's about. Entries are checklist-style, unread/unacted = `[ ]`, read/acted = `[X]`. diwan signs and timestamps its own entries; the user is not required to:
```org
** Chat :chat:
- [ ] Please investigate some ways I could decorate my kitchen
- [X] [diwan 2026-10-13] I have added various proposals in the Jams header
- [ ] [diwan 2026-10-21] I found the "Invite cellist" item in the inbox and think it belongs here. Please confirm.
```
**Ownership rule:** a `:chat:` heading that lives under a `:session:`-tagged ancestor is handled by that ancestor's `DIWAN_ID` session — don't spin up a separate session just for the chat.
**Promotion rule:** a `:chat:` heading with *no* `:session:`-tagged ancestor is answered directly by whichever diwan agent is processing it (ordinarily `tick`). For each unread (`[ ]`) entry: trivial — answer/act inline, mark `[X]`. Non-trivial — the same escalation decision as any other `:diwan:`-tagged heading above: a `:session:` tag if the work is self-contained, or handled by `tick` directly if it's cross-cutting.
**Tag-consistency rule:** `:session:` and `:chat:` both imply diwan's attention, so each should normally appear together with (or have an ancestor carrying) a `:diwan:` tag. This is fine:
```org
* Heading A :diwan:
** Heading B :session:
```
This is not explicit enough — the next `diwan tick` adds `:diwan:` directly to Heading B (not to Heading A) to fix it:
```org
* Heading A
** Heading B :session:
```
Same rule for `:chat:`.
## Signing format
`[diwan YYYY-MM-DD]` prefix on a chat line, as shown above. Optional on headings/body text diwan writes elsewhere; required on chat entries so the user can tell diwan's messages from their own at a glance.
## `diwan.org`
diwan's own operational file, at the root of the content repo — not a place for the user's personal notes, those belong wherever this deployment's own `AGENTS.md` says. Four top-level headings, each with a distinct job:
### Reports
Every notable thing a diwan agent does — a tick dispatching something new, a tick triaging a batch of inbox items, a `do` session finishing a real piece of work — gets its own short subheading under the top-level "Reports" heading in `diwan.org`. One entry per notable thing, not one giant paragraph per run: this is what keeps each entry short and the whole log skimmable with everything collapsed. **Whoever actually did the thing writes the entry** — `diwan tick` reports its own actions (triage moves, inline answers, promotions, new dispatches), and a `diwan do` session reports its *own* progress once it has something worth telling the user, not before. In particular: a tick merely nudging an idle active session (see the `:session:` section above) is not itself reportable — tick has no idea yet whether anything came of it, so it stays silent; if the nudge leads the session to actually do something, that session writes the entry once it has.
Format: a `[ ]`/`[X]` checkbox right in the subheading title (same pattern as `Close diwan session`), a short title, an optional `:LINK:` property pointing at the one heading the entry is most about (fuzzy heading link, or `id:` link for a note file with an `:ID:` drawer — skip it if there's no single heading to point at), and a signed one-or-two-line body, no more:
```org
* Reports
** [ ] Dispatched kitchen redecoration
:PROPERTIES:
:LINK: [[file:personal.org::*Plan the kitchen redecoration][Plan the kitchen redecoration]]
:END:
[diwan 2026-11-08] Non-trivial, dispatched as do-kitchen.
** [X] Triaged 2 inbox items
[diwan 2026-11-07] Moved into esdeveniments.org and personal.org.
```
Unread = `[ ]`, read = `[X]`, same semantics as `:chat:` — the user checks an entry off once they've seen it. Once an entry is `[X]`, the *next* `diwan tick` deletes the whole subheading, not just the checkbox — this is a disposable log, not a record, and only `tick` ever deletes from it (a `do` session only ever appends its own new entries, never touches an existing one). Questions for the user still go in the relevant heading's own `:chat:`, not here — a report entry may link there, but the report itself is one-way, no back-and-forth expected.
### Tasks
Plain `TODO`/`DONE` items about maintaining diwan's own domain — the notes themselves, not any one topic in them: archiving old entries, bulk cleanup, moving files or folders around, repo-wide renames. Anyone (the user or diwan) can add one directly as a plain heading here; living under `diwan.org`'s Tasks heading is itself the signal, no `:diwan:` tag needed on the item (the heading itself may carry one, that's fine, just not required). `diwan tick` treats every open item here as inherently cross-cutting work — see the `:diwan:` tag section above — and handles it directly in its own small commits, across however many tick runs it takes, marking the item `DONE` once actually finished, or asking via a `:chat:` child first if a real judgment call needs the user's input.
### Config
Reserved for configuration values governing diwan's own behavior — tick cadence, tab retention, which harness to dispatch, that kind of thing — as one legible place instead of scattered environment variables. **Not implemented yet.** The `diwan` CLI doesn't read this heading for configuration as of this writing; it's a placeholder for a future change, not a convention in force today. Env vars (`DIWAN_HARNESS`, `DIWAN_PERMISSION_MODE`, `DIWAN_PI_APPROVE`, `DIWAN_MAX_TICK_TABS`) remain the only actual knobs until that lands.
### Meta
Free-form notes about diwan itself — ideas, friction points, things worth revisiting as it evolves — written by the user or by diwan as they come up, independent of any one tick run. This feeds into developing diwan itself (see `DESIGN.md`); nothing currently reads or acts on this heading automatically, it's input for whoever picks up that work next, not a `tick`/`do` responsibility.
## Concurrency contract
Multiple agents (one `tick`, several `do`) may be running against this working tree at once. To keep that manageable without a real merge system:
1. **Scope discipline.** A `diwan do` agent only ever edits inside the subtree of the heading it owns (its `DIWAN_ID` heading), plus it may read anything. The one exception is `diwan.org`'s Reports log: a `do` agent may *append* a new entry of its own there (never edit or delete an existing one — that's `tick`'s job only), since that's the only channel it has for telling the user about finished work without a tick run in between. A `diwan tick` agent may move/create headings anywhere (that's its job), and is the one that handles non-trivial `:diwan:` requests that are inherently cross-cutting rather than dispatching them to a `do` session (see the `:diwan:` tag section) — but should keep each such move or piece of cross-cutting work a small, separate commit. This makes line-level conflicts rare by construction — don't go outside your lane.
2. **No manual commit requirement.** The user is never expected to commit. `diwan tick` commits any dirty state it finds at the *start* of a run, under the user's own git identity, before computing anything — so no tick ever operates on an uncommitted/mixed workspace. Any agent (tick or do) commits its *own* changes as soon as they're a coherent unit, using the diwan identity:
```sh
flock .diwan/.lock git -c user.name=diwan -c user.email=diwan@local commit -am "<what you did and why, one line>"
```
The `flock` is required — it's how concurrent agents avoid racing each other's `git add -A && commit`. Always go through it, never a bare `git commit`.
3. **Small commits over big ones.** Prefer several small, clearly-labeled commits to one large one — this is what makes `git log` a usable report of "what did diwan actually do" and what makes `git revert` safe to use on a single change without undoing unrelated work.
## Revert model
Every automated change is a git commit, full stop. There is no separate undo mechanism to learn: `git log`, `git diff`, `git revert <sha>` are *the* way the user inspects or undoes anything diwan did. `.diwan/state.json` is the only piece of diwan state that isn't a note — it just remembers the last commit a completed `tick` left behind, so the next tick knows what's new.
## herdr
Sessions are herdr (https://herdr.dev) terminals, not a diwan-invented session store. All diwan activity lives in one herdr workspace labeled `diwan` (created on first use if missing). Within it:
- a `diwan tick` run is a tab labeled `tick-<UTC timestamp>`, with an agent of kind `$DIWAN_HARNESS` (default `claude`) started in its root pane and prompted with the tick instructions;
- a `diwan do` session for a given `DIWAN_ID` is a tab labeled exactly that `DIWAN_ID`, same pattern — this is *why* `DIWAN_ID` must be a valid herdr agent name (`^[a-z][a-z0-9_-]{0,31}$`), not an arbitrary string.
To resume a `DIWAN_ID`, `diwan` first checks whether its agent is still alive (`herdr agent get <id>`) and just prompts it if so; if the process has exited (normal between ticks), it finds the tab by label and starts a fresh agent in the same pane, in the same working directory — the heading's own content is what carries continuity, not agent conversation history. `diwan` shells out to the real `herdr` CLI (the `diwan` CLI has the exact calls, verified live against herdr 0.9.5; run `herdr --skill` to check this hasn't drifted on a newer version). When `herdr` isn't on PATH, every call prints what it would have run instead of failing, so the pipeline stays testable without herdr installed.
## Harness-agnosticism
Nothing here should assume a specific LLM product. `TICK.md` and `DO.md` are plain-language procedures; the `diwan` CLI is the only "hands" they need (plus ordinary file edits and git). Pointing a different harness at the same procedures should just work.