Files
marc 34f1f4396e Close diwan session: TODO/DONE instead of checkbox; purge evolutionary residue from all docs
1. 'Close diwan session' now uses TODO/DONE (assign_diwan_id inserts
'** TODO Close diwan session'), matching Reports. No headline anywhere
in diwan's own docs uses a [ ]/[X] checkbox now -- only :chat: list
items do, which is correct org usage (checklist items, not headline
states) and unaffected.

2. Rewrote PROTOCOL.md/TICK.md/DO.md/DESIGN.md to read as a single
coherent design rather than a changelog of how they got here. Removed:
contrastive 'not because X, but because Y' phrasing that only makes
sense if you know the prior rule, references to old tag names/formats
that no longer exist, a Claude-Code-vs-pi rationale that belongs in
DESIGN.md (if anywhere) not the agent-facing PROTOCOL.md, and 'no
:LINK: property' (never describe a convention by naming something it
doesn't have).

3. DESIGN.md got the heaviest pass since it had drifted from reality,
not just tone: it still said 'no flake.nix yet' (there is one, in
production use via samfelag), claimed several things 'not yet
re-verified live' that have since been verified multiple times
(tick's no-op gate, triage, the tag-consistency/escalation decision,
pi against real heading content via tick), and didn't mention
'diwan remote' or the actual flake.nix packaging shape anywhere.
Rewrote the verified-live claims and Known Gaps list to match current
reality, added flake.nix to the doc map and architecture section.

Live content migrated to match: diwan-dev-notes' decorate-home heading
('Close diwan session' checkbox -> TODO).
2026-11-09 23:17:17 +01:00

130 lines
15 KiB
Markdown

# PROTOCOL.md — the diwan protocol
This is diwan's generic contract: the tag system, the `diwan.org` layout, the concurrency/revert model, and the herdr topology. It describes how diwan works, independent of which notes repo it's pointed at — it ships with diwan itself (this repo), not with any one person's content. If you are a dispatched diwan agent (`tick` or `do`), read this file first (you were given its path directly in your dispatch prompt), then the *content* repo's own `AGENTS.md` — a different repo from this one, containing that deployment's specific notes schema and conventions. The two together are the complete contract; this doc never restates anything from a content repo's `AGENTS.md` (Layout, org conventions already in use) and that file never restates anything from here — if you find yourself unsure which owns something, it's a doc bug worth flagging, not something to resolve by guessing.
## `:diwan:` tag
A generic way to call diwan's attention to a heading: tagging something `:diwan:` means "I want diwan to take care of this" (or, when diwan adds it itself, "this needs diwan's attention"), without saying anything about *how*. An ordinary task that's normally just for the user becomes a delegated request the moment the tag is added:
```org
** TODO Archive all DONE entries in the inbox :diwan:
```
Can be added directly by the user, or by diwan itself when it decides some other heading (a chat reply, a plain task found during triage) needs this kind of attention — see the tag-consistency rule below. What actually happens to a `:diwan:`-tagged heading depends on what the request needs, decided by whichever diwan agent processes it (ordinarily `tick`, per `TICK.md`):
- Trivial (a quick lookup, a one-line answer): handled inline immediately, no further tag, no session.
- Non-trivial but self-contained in this heading's own subtree (independent delegated work, nothing that needs writes elsewhere): escalates to a `:session:` tag and a dedicated `diwan do` session.
- Non-trivial *and* inherently cross-cutting — spans multiple files or headings, can't be parallelized, e.g. archiving, bulk cleanup, moving things between files or folders: handled by `diwan tick` itself, directly, possibly across several tick runs — not escalated to a session, since a subtree-scoped `do` agent would just hit the same wall a human doing it alone would (see the Concurrency contract's scope rule). No `:session:` tag for this case.
## `:session:` tag
A heading tagged `:session:` has (or is about to get) a dedicated, persistent agent session handling it — a `DIWAN_ID` property and a `Close diwan session` TODO item. Note the org tag syntax when it's stacked alongside `:diwan:`: one colon-delimited run, no space (`:diwan:session:`, not `:diwan: :session:`) — the latter isn't valid org tag syntax and won't parse as two tags.
```org
* Gather information about what I could do on my next trip to greece :diwan:session:
:PROPERTIES:
:DIWAN_ID: do-greece-trip
:END:
** TODO Close diwan session
```
Can be added by the user directly, when they already know a heading needs its own ongoing session, or by diwan (ordinarily `tick`) when it decides a `:diwan:`-tagged request is non-trivial and self-contained enough to delegate (see above). `DIWAN_ID` is a herdr terminal name (see "herdr", below) — not an arbitrary id. It points directly at the live/persisted agent session handling this heading, so "jump in personally" is just attaching to that terminal, no separate lookup needed. A heading keeps the same `DIWAN_ID`/`:session:` tag across multiple tick passes until its `Close diwan session` item is marked `DONE` by the user, at which point the *next* tick retires it: closes that `DIWAN_ID`'s herdr tab (`herdr tab close`) and removes the `:session:` tag from the heading. The `DIWAN_ID` property itself is left in place as a historical record, in case the session's work is ever worth revisiting.
## `:chat:` tag
A `:chat:` heading is a conversation thread between the user and diwan, always as a child of the heading it's about. Entries are checklist-style, unread/unacted = `[ ]`, read/acted = `[X]`. diwan signs and timestamps its own entries; the user is not required to:
```org
** Chat :chat:
- [ ] Please investigate some ways I could decorate my kitchen
- [X] [diwan 2026-10-13] I have added various proposals in the Jams header
- [ ] [diwan 2026-10-21] I found the "Invite cellist" item in the inbox and think it belongs here. Please confirm.
```
**Ownership rule:** a `:chat:` heading that lives under a `:session:`-tagged ancestor is handled by that ancestor's `DIWAN_ID` session — don't spin up a separate session just for the chat.
**Promotion rule:** a `:chat:` heading with *no* `:session:`-tagged ancestor is answered directly by whichever diwan agent is processing it (ordinarily `tick`). For each unread (`[ ]`) entry: trivial — answer/act inline, mark `[X]`. Non-trivial — the same escalation decision as any other `:diwan:`-tagged heading above: a `:session:` tag if the work is self-contained, or handled by `tick` directly if it's cross-cutting.
**Tag-consistency rule:** `:session:` and `:chat:` both imply diwan's attention, so each should normally appear together with (or have an ancestor carrying) a `:diwan:` tag. This is fine:
```org
* Heading A :diwan:
** Heading B :session:
```
This is not explicit enough — the next `diwan tick` adds `:diwan:` directly to Heading B (not to Heading A) to fix it:
```org
* Heading A
** Heading B :session:
```
Same rule for `:chat:`.
## Signing format
`[diwan YYYY-MM-DD]` prefix on a chat line, as shown above. Optional on headings/body text diwan writes elsewhere; required on chat entries so the user can tell diwan's messages from their own at a glance.
## `diwan.org`
diwan's own operational file, at the root of the content repo — not a place for the user's personal notes, those belong wherever this deployment's own `AGENTS.md` says. Four top-level headings, each with a distinct job:
### Reports
This is the one place the user checks for what's new — a shortcut to everything that needs their attention, not just a highlights reel — so the bar for getting a short subheading under the top-level "Reports" heading in `diwan.org` is deliberately broad: items moved (triage), a new session dispatched, a session closed, an inline answer, a finding, a flagged ambiguity, a problem hit, or anywhere a note was left for the user to read (even one already marked `[X]`/acted-on by whoever wrote it — that means *they've* acted on it, not that the user has seen it yet). The only things that don't get an entry: a run where genuinely nothing happened at all, and a bare nudge to an idle session (not because it's unimportant, but because whoever sent it has no idea yet whether anything came of it — see the `:session:` section above; if the nudge leads to real work, that work gets its own entry once there's something to report). Keep each entry to its own short subheading, not one giant paragraph per run: this is what keeps the whole log skimmable with everything collapsed. **Whoever actually did the thing writes the entry** — `diwan tick` reports its own actions, and a `diwan do` session reports its own progress once it has something to tell the user, not before.
Format: a `TODO`/`DONE` keyword on the subheading itself, a short title that *is itself the link* to the one heading the entry is most about (fuzzy heading link, or `id:` link for a note file with an `:ID:` drawer — just plain text with no link if there's no single heading to point at), and a signed one-or-two-line body, no more. Any further links belong inline in the body text:
```org
* Reports
** TODO [[file:personal.org::*Plan the kitchen redecoration][Kitchen redecoration: flagged a budget conflict]]
[diwan 2026-11-08] The heading asks for a 3000€ budget but its linked Pinterest board is all 8000€+ kitchens — asked which one is right in the heading's own :chat: before researching further.
** DONE Archived personal.org's Tasques backlog
[diwan 2026-11-07] 45 DONE tasks moved into archive/archive.org, oldest from 2023 — personal.org's active list was getting hard to scan.
```
Unread = `TODO`, read = `DONE` — the user marks an entry `DONE` once they've seen it. Once an entry is `DONE`, the *next* `diwan tick` deletes the whole subheading, not just the keyword — this is a disposable log, not a record, and only `tick` ever deletes from it (a `do` session only ever appends its own new entries, never touches an existing one). Questions for the user still go in the relevant heading's own `:chat:`, not here — a report entry may link there, but the report itself is one-way, no back-and-forth expected.
### Tasks
Plain `TODO`/`DONE` items about maintaining diwan's own domain — the notes themselves, not any one topic in them: archiving old entries, bulk cleanup, moving files or folders around, repo-wide renames. Anyone (the user or diwan) can add one directly as a plain heading here; living under `diwan.org`'s Tasks heading is itself the signal, no `:diwan:` tag needed on the item (the heading itself may carry one, that's fine, just not required). `diwan tick` treats every open item here as inherently cross-cutting work — see the `:diwan:` tag section above — and handles it directly in its own small commits, across however many tick runs it takes, marking the item `DONE` once actually finished, or asking via a `:chat:` child first if a real judgment call needs the user's input.
### Config
Reserved for configuration values governing diwan's own behavior — tick cadence, tab retention, that kind of thing — as one legible place instead of scattered environment variables. **Not implemented yet.** The `diwan` CLI doesn't read this heading for configuration as of this writing; it's a placeholder for a future change, not a convention in force today. Env vars (`DIWAN_DIR`, `DIWAN_NOTES_DIR`, `DIWAN_PI_APPROVE`, `DIWAN_MAX_TICK_TABS`) remain the only actual knobs until that lands.
### Meta
Free-form notes about diwan itself — ideas, friction points, things worth revisiting as it evolves — written by the user or by diwan as they come up, independent of any one tick run. This feeds into developing diwan itself (see `DESIGN.md`); nothing currently reads or acts on this heading automatically, it's input for whoever picks up that work next, not a `tick`/`do` responsibility.
## Concurrency contract
Multiple agents (one `tick`, several `do`) may be running against this working tree at once. To keep that manageable without a real merge system:
1. **Scope discipline.** A `diwan do` agent only ever edits inside the subtree of the heading it owns (its `DIWAN_ID` heading), plus it may read anything. The one exception is `diwan.org`'s Reports log: a `do` agent may *append* a new entry of its own there (never edit or delete an existing one — that's `tick`'s job only), since that's the only channel it has for telling the user about finished work without a tick run in between. A `diwan tick` agent may move/create headings anywhere (that's its job), and is the one that handles non-trivial `:diwan:` requests that are inherently cross-cutting rather than dispatching them to a `do` session (see the `:diwan:` tag section) — but should keep each such move or piece of cross-cutting work a small, separate commit. This makes line-level conflicts rare by construction — don't go outside your lane.
2. **No manual commit requirement.** The user is never expected to commit. `diwan tick` commits any dirty state it finds at the *start* of a run, under the user's own git identity, before computing anything — so no tick ever operates on an uncommitted/mixed workspace. Any agent (tick or do) commits its *own* changes as soon as they're a coherent unit, using the diwan identity:
```sh
flock .diwan/.lock git -c user.name=diwan -c user.email=diwan@local commit -am "<what you did and why, one line>"
```
The `flock` is required — it's how concurrent agents avoid racing each other's `git add -A && commit`. Always go through it, never a bare `git commit`.
3. **Small commits over big ones.** Prefer several small, clearly-labeled commits to one large one — this is what makes `git log` a usable report of "what did diwan actually do" and what makes `git revert` safe to use on a single change without undoing unrelated work.
## Revert model
Every automated change is a git commit, full stop. There is no separate undo mechanism to learn: `git log`, `git diff`, `git revert <sha>` are *the* way the user inspects or undoes anything diwan did. `.diwan/state.json` is the only piece of diwan state that isn't a note — it just remembers the last commit a completed `tick` left behind, so the next tick knows what's new.
## herdr
Sessions are herdr (https://herdr.dev) terminals, not a diwan-invented session store. All diwan activity lives in one herdr workspace labeled `diwan` (created on first use if missing). Within it:
- a `diwan tick` run is a tab labeled `tick-<UTC timestamp>`, with a `pi` agent started in its root pane and prompted with the tick instructions;
- a `diwan do` session for a given `DIWAN_ID` is a tab labeled exactly that `DIWAN_ID`, same pattern — this is *why* `DIWAN_ID` must be a valid herdr agent name (`^[a-z][a-z0-9_-]{0,31}$`), not an arbitrary string.
To resume a `DIWAN_ID`, `diwan` first checks whether its agent is still alive (`herdr agent get <id>`) and just prompts it if so; if the process has exited (normal between ticks), it finds the tab by label and starts a fresh agent in the same pane, in the same working directory — the heading's own content is what carries continuity, not agent conversation history. `diwan` shells out to the real `herdr` CLI, verified live against herdr 0.9.5 (run `herdr --skill` to check this hasn't drifted on a newer version). When `herdr` isn't on PATH, every call prints what it would have run instead of failing, so the pipeline stays testable without herdr installed.
Bare `diwan` (no subcommand) focuses the local `diwan` workspace and attaches you to it interactively. `diwan remote <machine>` does the same against a herdr machine saved via `herdr machine add <ssh-target>` (see herdr's own `--help`/`--skill` for setting one up) — it focuses that machine's `diwan` workspace over the scripted `--machine` API, then execs `herdr --remote <machine>` to open an interactive attach into it. This relies on `<machine>` being usable both as a saved machine label (for the focus step) and as a raw `--remote` SSH target (for the attach) — true by default, since `herdr machine add` labels a profile after its SSH target unless `--label` overrides it.
## The harness is `pi`
diwan dispatches `pi` exclusively, started with `--approve` so it never blocks on its one-time project-trust decision for protected `.pi/*` resources, if a content repo ever has any. Override the flag via `DIWAN_PI_APPROVE` if a different one is ever needed.