Close diwan session: TODO/DONE instead of checkbox; purge evolutionary residue from all docs

1. 'Close diwan session' now uses TODO/DONE (assign_diwan_id inserts
'** TODO Close diwan session'), matching Reports. No headline anywhere
in diwan's own docs uses a [ ]/[X] checkbox now -- only :chat: list
items do, which is correct org usage (checklist items, not headline
states) and unaffected.

2. Rewrote PROTOCOL.md/TICK.md/DO.md/DESIGN.md to read as a single
coherent design rather than a changelog of how they got here. Removed:
contrastive 'not because X, but because Y' phrasing that only makes
sense if you know the prior rule, references to old tag names/formats
that no longer exist, a Claude-Code-vs-pi rationale that belongs in
DESIGN.md (if anywhere) not the agent-facing PROTOCOL.md, and 'no
:LINK: property' (never describe a convention by naming something it
doesn't have).

3. DESIGN.md got the heaviest pass since it had drifted from reality,
not just tone: it still said 'no flake.nix yet' (there is one, in
production use via samfelag), claimed several things 'not yet
re-verified live' that have since been verified multiple times
(tick's no-op gate, triage, the tag-consistency/escalation decision,
pi against real heading content via tick), and didn't mention
'diwan remote' or the actual flake.nix packaging shape anywhere.
Rewrote the verified-live claims and Known Gaps list to match current
reality, added flake.nix to the doc map and architecture section.

Live content migrated to match: diwan-dev-notes' decorate-home heading
('Close diwan session' checkbox -> TODO).
This commit is contained in:
marc committed 2026-11-09 23:17:17 +01:00
1 parent 19e6ae3a59
commit 34f1f4396e
5 files changed
+36 -37

No files matched your search

+22 -23
View File
@@ -6,9 +6,9 @@ Audience: whoever (human or agent) picks up **development of diwan itself** next
diwan is an AI layer over a plain org-mode "second brain" (notes, tasks, calendar, projects — normally edited by hand in Emacs/beorg, synced via Nextcloud). It periodically triages a capture inbox, keeps files tidy, and runs delegated research/work when asked, while treating the org files as the only real source of truth.
**This repo is diwan the tool, and nothing else.** It contains the CLI and the generic procedure docs (`PROTOCOL.md`, `TICK.md`, `DO.md`) — no personal notes, no content-repo-specific convention docs. It's meant to be installed independently (on `PATH`, eventually via its own nix flake) and pointed at *any* notes repo. Marc's actual notes live in `~/nc/org-agenda` (the real, continuously-synced data) and, for development/testing, in a throwaway sibling repo `diwan-dev-notes` (a git-tracked fixture copy, safe to break). Neither of those repos contains any of diwan's own code or docs — see "Known gaps" for exactly how far that packaging currently goes.
**This repo is diwan the tool, and nothing else.** It contains the CLI, a nix flake packaging it, and the generic procedure docs (`PROTOCOL.md`, `TICK.md`, `DO.md`) — no personal notes, no content-repo-specific convention docs. It's installed independently (on `PATH`, via this repo's own flake) and pointed at *any* notes repo. Marc's actual notes live in `~/nc/org-agenda` (the real, continuously-synced data) and, for development/testing, in a throwaway sibling repo `diwan-dev-notes` (a git-tracked fixture copy, safe to break). Neither of those repos contains any of diwan's own code or docs.
Status as of this writing: the core loop (tick → triage/dispatch, do → delegated work, both via real herdr sessions) is built and has been exercised end-to-end against live `herdr`. The tool/content split (this repo vs. `diwan-dev-notes`) happened 2026-11-09, and the harness was narrowed to `pi` exclusively right after that (dropping Claude Code support entirely) — see "Known gaps" for what both of those still owe a fresh live pass on.
The core loop (tick → triage/dispatch, do → delegated work, both via real herdr sessions, `pi` as the sole harness) is built, packaged as a flake, and deployed: `samfelag` (Marc's NixOS fleet config) has a module that installs the package and runs `diwan tick` on a systemd timer, already live on `thingvellir`. See "Known gaps" for what's still open.
## Principles (why things are shaped this way)
@@ -17,8 +17,8 @@ These were distilled from how Marc already works and what he explicitly asked fo
1. **Org files are ground truth, not a cache.** If diwan vanished, the notes still work in Emacs/beorg. Nothing diwan needs lives outside `.org` files except the content repo's own `.diwan/state.json`, which is pure bookkeeping, not content.
2. **Act first, review passively.** No per-edit approval gate — every action is reported and trivially revertable instead.
3. **Mechanism over magic.** Small, inspectable, deterministic steps over one opaque "agent did something" blob — Marc wants to be able to understand a failure well enough to jump in personally.
4. **Grow by accretion.** Ship the smallest loop that works; add complexity only once the simpler version has been lived with. (Concretely: v0 has no custom CLI-level org-tag manipulation, no scheduler, no kill-on-timeout logic — these are deliberately not yet built.)
5. **One harness, chosen deliberately: `pi`.** diwan dispatches `pi` and only `pi` — not a pluggable choice. Claude Code was the original harness and worked, but needed `--permission-mode auto` to avoid prompting for approval on every tool call, and blocks on an interactive first-run trust dialog that herdr's real-pane dispatch can't answer (hit live 2026-11-09 against the fresh `diwan-dev-notes` content repo). `pi` has no per-tool-call approval gate and no such trust dialog (only a one-time, non-interactive-safe project-trust decision for protected `.pi/*` resources, handled by `--approve`) — it was built to run unattended from the start, which is exactly diwan's own model. The procedure docs (`TICK.md`/`DO.md`) still don't hardcode anything pi-specific into their wording, as a matter of good practice, but there's no expectation of swapping harnesses again.
4. **Grow by accretion.** Ship the smallest loop that works; add complexity only once the simpler version has been lived with. (Concretely: v0 has no custom CLI-level org-tag manipulation, no scheduler beyond a plain systemd timer, no kill-on-timeout logic — these are deliberately not yet built.)
5. **One harness, chosen deliberately: `pi`.** diwan dispatches `pi` exclusively, not a pluggable choice. It has no per-tool-call approval gate and no interactive first-run trust dialog — it runs unattended from the start, which is exactly diwan's own model. The procedure docs (`TICK.md`/`DO.md`) don't hardcode anything pi-specific into their wording anyway, as a matter of good practice.
6. **Two symmetric channels, one substrate.** Talk to diwan via the notes (`:chat:`) or via the live herdr session — both write to the same git-tracked truth.
7. **Mobile stays dumb.** Nothing new to install on the phone.
8. **Tool and content are separate repos.** diwan the tool (this repo) ships independent of any one person's notes; a content repo (e.g. `diwan-dev-notes`, or Marc's real `~/nc/org-agenda`) only ever contains notes, `AGENTS.md`, `diwan.org`, and diwan's own tiny `.diwan/` runtime-bookkeeping directory (`state.json`, `.lock`) — never the CLI or the procedure docs. This is what makes the content repo distributable (synced via Nextcloud to a server, edited from beorg) without dragging a whole dev tool along with it, and what keeps diwan's own generic contract reviewable/versionable independent of any one person's data.
@@ -27,9 +27,9 @@ These were distilled from how Marc already works and what he explicitly asked fo
Three layers, each independently simple:
- **Git**, in the *content* repo, no remote (for Marc's prototype content repo — a real deployment may add one, e.g. for Nextcloud sync, but that's the content repo's business, not this one's). Every automated write to the content repo is a commit. This *is* the revert mechanism (principle #2) — not a bespoke undo log. Commits are authored as `diwan <diwan@local>` when diwan itself is the actor, or the user's own git identity when `diwan tick` snapshots whatever manual edits were lying around at the start of a run (so no tick ever operates on a dirty/mixed workspace, and "who changed this" is answerable from `git log --author` alone). This repo (the tool) has its own, unrelated git history — just ordinary source control for diwan's own development.
- **herdr**, a terminal-multiplexer-for-agents with a JSON-over-socket CLI (`herdr workspace/tab/pane/agent ...`). diwan doesn't invent a session store: one herdr workspace labeled `diwan` holds every tick run and every open `do` session as a separate, independently addressable, persistent tab. See "herdr integration" below for the exact calls.
- **The `diwan` CLI** (`./diwan` at this repo's root), a single dependency-free Python script (shebang runs it through `nix shell nixpkgs#python3`, so no system Python install needed on NixOS; a real flake package would wrap this properly instead). It does only the mechanical, deterministic parts: git bookkeeping, enough org parsing to read/write a `DIWAN_ID` property and insert a `Close diwan session` checkbox item, and herdr dispatch. It never triages, writes prose, or makes judgment calls — that's the dispatched agent's job, guided by `TICK.md`/`DO.md`. It distinguishes two roots explicitly (see the script's own docstring): `DIWAN_DIR` (where it and its docs are installed — `Path(__file__).resolve().parent`, overridable via the `DIWAN_DIR` env var for whatever a future flake's install layout needs) and `DIWAN_NOTES_DIR` (whatever notes repo it's invoked against, found via `git rev-parse --show-toplevel` from the current directory by default, overridable via the `DIWAN_NOTES_DIR` env var for e.g. a cron/systemd timer that doesn't cd into it first). `diwan` must be run from inside (or below) a content repo, or have `DIWAN_NOTES_DIR` set — it has no content of its own to act on.
- **Git**, in the *content* repo. Every automated write to the content repo is a commit. This *is* the revert mechanism (principle #2) — not a bespoke undo log. Commits are authored as `diwan <diwan@local>` when diwan itself is the actor, or the user's own git identity when `diwan tick` snapshots whatever manual edits were lying around at the start of a run (so no tick ever operates on a dirty/mixed workspace, and "who changed this" is answerable from `git log --author` alone). This repo (the tool) has its own, unrelated git history — just ordinary source control for diwan's own development.
- **herdr**, a terminal-multiplexer-for-agents with a JSON-over-socket CLI (`herdr workspace/tab/pane/agent ...`). diwan doesn't invent a session store: one herdr workspace labeled `diwan` holds every tick run and every open `do` session as a separate, independently addressable, persistent tab. See PROTOCOL.md's herdr section for the exact calls.
- **The `diwan` CLI**, a single dependency-free Python script, packaged by this repo's `flake.nix` (`pname = "diwan"`): the script and `PROTOCOL.md`/`TICK.md`/`DO.md` get installed together under `$out/share/diwan/`, and `$out/bin/diwan` is a `makeWrapper`-generated wrapper around `python3` that sets `DIWAN_DIR` explicitly to that `share/diwan` directory and puts `git` on `PATH`. It does only the mechanical, deterministic parts: git bookkeeping, enough org parsing to read/write a `DIWAN_ID` property and insert a `Close diwan session` TODO item, and herdr dispatch. It never triages, writes prose, or makes judgment calls — that's the dispatched agent's job, guided by `TICK.md`/`DO.md`. It distinguishes two roots explicitly (see the script's own docstring): `DIWAN_DIR` (where it and its docs are installed, overridable via the `DIWAN_DIR` env var) and `DIWAN_NOTES_DIR` (whatever notes repo it's invoked against, found via `git rev-parse --show-toplevel` from the current directory by default, overridable via the `DIWAN_NOTES_DIR` env var — this is how a systemd service that doesn't `cd` anywhere points it at the right repo). `diwan` must be run from inside (or below) a content repo, or have `DIWAN_NOTES_DIR` set — it has no content of its own to act on.
Everything else (what counts as "trivial", where an inbox item belongs, how to answer a chat question) is intentionally **not** code — it's prose in `TICK.md`/`DO.md` that the dispatched `pi` agent has to interpret. The CLI's job ends at "here's a prompt, here's a terminal"; everything judgment-shaped is left to the model.
@@ -38,6 +38,7 @@ Everything else (what counts as "trivial", where an inbox item belongs, how to a
| File | Lives in | Audience | Contents |
|---|---|---|---|
| `DESIGN.md` (this file) | tool repo | diwan developers | why, architecture, current implementation status, known gaps |
| `flake.nix` | tool repo | nix, and anything consuming this flake (e.g. `samfelag`) | packages the CLI + docs together; see "Architecture at a glance" |
| `PROTOCOL.md` | tool repo | any dispatched diwan agent (tick or do) | diwan's generic contract: tags, `diwan.org`, concurrency contract, revert model, herdr topology — deployment-independent |
| `TICK.md` | tool repo | a `diwan tick` agent specifically | the 5-step tick procedure |
| `DO.md` | tool repo | a `diwan do` agent specifically | scope, chat protocol, commit discipline, how to stop |
@@ -54,7 +55,7 @@ Two halves: a deterministic CLI half (`cmd_tick` in `diwan`) and a judgment-driv
1. If the content repo's working tree is dirty, commit everything under the user's own git identity (`chore: snapshot manual edits before tick`) — this is what makes "no manual commit requirement" (PROTOCOL.md) actually true.
2. Load the content repo's `.diwan/state.json`'s `last_tick_commit`. Diff that commit against current `HEAD` (in the content repo), excluding `.diwan/` itself (so diwan's own bookkeeping commits never count as "something changed"). No prior tick → treat as "always changed."
3. If nothing changed and `--force` wasn't passed: print and exit. This is the only gate on whether a tick does anything at all.
4. Otherwise, pick a tab name `tick-<UTC YYYYMMDD-HHMMSS>` (lowercase + digits + hyphens only — see "herdr integration" for why that's not negotiable) and dispatch a fresh agent there (`herdr_dispatch_fresh`), in a pane whose working directory is the *content* repo (`DIWAN_NOTES_DIR`), with a prompt that says: read `<DIWAN_DIR>/PROTOCOL.md` and `AGENTS.md` (the latter resolved relative to the agent's own cwd, i.e. the content repo), follow `<DIWAN_DIR>/TICK.md`, here's the commit range. **This dispatch is fire-and-forget** — the CLI call returns as soon as the prompt is submitted, it does not wait for the tick to finish.
4. Otherwise, pick a tab name `tick-<UTC YYYYMMDD-HHMMSS>` (lowercase + digits + hyphens only — see PROTOCOL.md's herdr section for why that's not negotiable) and dispatch a fresh agent there, in a pane whose working directory is the *content* repo (`DIWAN_NOTES_DIR`), with a prompt that says: read `<DIWAN_DIR>/PROTOCOL.md` and `AGENTS.md` (the latter resolved relative to the agent's own cwd, i.e. the content repo), follow `<DIWAN_DIR>/TICK.md`, here's the commit range. **This dispatch is fire-and-forget** — the CLI call returns as soon as the prompt is submitted, it does not wait for the tick to finish.
5. Record the new tab name in the content repo's `state.json`'s `tick_history` (kept to the last `MAX_TICK_TABS`, default 3, override via `DIWAN_MAX_TICK_TABS`), commit that under the diwan identity, and — if this pushed any older tick tabs out of the retention window — close those tabs for real via `herdr tab close`.
Note what's *not* here: `last_tick_commit` is deliberately **not** updated by the CLI at dispatch time. It's updated by the dispatched agent itself, as the literal last step of `TICK.md`, only if everything else succeeded. If a tick agent crashes or is killed partway, the next tick sees the same "changed" range again rather than silently skipping work — correctness over tidiness.
@@ -62,11 +63,11 @@ Note what's *not* here: `last_tick_commit` is deliberately **not** updated by th
**Agent half** (`TICK.md`), five steps, each its own commit:
1. Triage `inbox.org` only (other files changing is someone else's edit): events → `esdeveniments.org`, tasks → wherever they actually belong, `:diwan:` items get moved but not acted on yet, new topics only get a new file when nothing existing fits.
2. Tag consistency: any `:session:`/`:chat:` heading without a `:diwan:` tag on itself or an ancestor gets `:diwan:` added directly to it. Then resolve `:chat:` ownership: chats under a `:session:` ancestor are left for that heading's own session; orphan chats get answered inline if trivial, or escalated (see step 3's three-way decision) if not.
3. Handle every `:diwan:`-tagged heading without an open `:session:`: trivial ones get answered inline, no tag beyond `:diwan:` ever assigned. Non-trivial ones split two ways — self-contained within the heading's own subtree gets a `:session:` tag and `diwan do <file>:<line> --diwan-id <slug>` run *by the tick agent itself* (the tick agent is the only thing that ever originates a fresh `DIWAN_ID`); inherently cross-cutting work (spans multiple files/headings, can't be parallelized — archiving, bulk cleanup, moving things between files) is done by the tick agent *directly*, right there, no session at all, possibly spread across several tick runs if it needs Marc's input first. The same handling applies to any open item under `diwan.org`'s `Tasks` heading — diwan's own self-maintenance backlog, no tag needed there since the location already signals it (see PROTOCOL.md's `diwan.org` section). Already-open sessions (`:session:` tag + `DIWAN_ID` present, `Close diwan session` still `[ ]`) get their subtree's content left alone as always, but are checked for a nudge: if any commit in this tick's change range *not* authored by the diwan git identity touched a line inside that subtree (a hand-authored heading edit, a new unread `:chat:` entry, anything Marc did), the tick agent re-runs `diwan do <file>:<line> --diwan-id <id>` on the existing ID — a continuation dispatch into the session's own tab/agent, not a fresh one — purely to wake it up; no nudge at all if nothing hand-authored touched the subtree. This is the only mechanism that gets a change Marc makes to an active heading in front of its own idle agent — nothing watches the file otherwise. A session whose `Close diwan session` is already `[X]` gets retired for real: the tick agent looks up its `DIWAN_ID`'s herdr tab and actually closes it (`herdr tab close`), then removes the `:session:` tag from the heading — the `DIWAN_ID` property itself is left in place as a historical record. Dispatch and closure both get a Reports entry (new/changed state Marc hasn't seen); a bare nudge does not, not because it's unimportant but because there's no information yet on whether it did anything — see step 4.
4. Delete any already-`DONE` Reports subheadings first, then write one short, titled `TODO` subheading per reportable thing this run did into `diwan.org`'s Reports log (not one paragraph for the whole run) — the bar is deliberately broad (items moved, dispatches, closures, inline answers, findings, notes left for Marc to read), excluding only a run where nothing happened at all and a bare nudge (no information yet on whether it did anything) — each with the title itself linking to the one heading it's about (plain text if there isn't one) and a 1-2 line signed body.
3. Handle every `:diwan:`-tagged heading without an open `:session:`: trivial ones get answered inline, no tag beyond `:diwan:` ever assigned. Non-trivial ones split two ways — self-contained within the heading's own subtree gets a `:session:` tag and `diwan do <file>:<line> --diwan-id <slug>` run *by the tick agent itself* (the tick agent is the only thing that ever originates a fresh `DIWAN_ID`); inherently cross-cutting work (spans multiple files/headings, can't be parallelized — archiving, bulk cleanup, moving things between files) is done by the tick agent *directly*, right there, no session at all, possibly spread across several tick runs if it needs Marc's input first. The same handling applies to any open item under `diwan.org`'s `Tasks` heading — diwan's own self-maintenance backlog, no tag needed there since the location already signals it. Already-open sessions (`:session:` tag + `DIWAN_ID` present, `Close diwan session` still `TODO`) get their subtree's content left alone as always, but are checked for a nudge: if any commit in this tick's change range *not* authored by the diwan git identity touched a line inside that subtree, the tick agent re-runs `diwan do <file>:<line> --diwan-id <id>` on the existing ID to wake it up; no nudge at all if nothing hand-authored touched the subtree. A session whose `Close diwan session` is marked `DONE` gets retired: the tick agent closes its `DIWAN_ID`'s herdr tab (`herdr tab close`) and removes the `:session:` tag from the heading, leaving the `DIWAN_ID` property in place as a historical record.
4. Delete any already-`DONE` Reports subheadings first, then write one short, titled `TODO` subheading per reportable thing this run did into `diwan.org`'s Reports log — the bar is deliberately broad (items moved, dispatches, closures, inline answers, findings, notes left for Marc to read), excluding only a run where nothing happened at all and a bare nudge (no information yet on whether it did anything) — each with the title itself linking to the one heading it's about, and a 1-2 line signed body.
5. Only if 1–4 all succeeded: update `state.json.last_tick_commit` to current `HEAD` and commit under the diwan identity.
**Verified live** (2026-11-08, herdr 0.9.5 + Claude Code, before both the 2026-11-09 tool/content split and the pi-only harness switch): dispatched for real, ran to completion with zero permission prompts, correctly left two already-open `:session:`-tagged sessions untouched (back when `:session:` was still spelled `:diwan:`, before the tag rename — see "Known gaps" for what's unverified post-rename), correctly found nothing to triage, wrote a proper signed report, and updated `state.json` correctly. A second immediate `diwan tick` correctly no-op'd. **Not yet re-verified with a real dispatched agent** against the current state of things (post-split two-repo layout, `pi` as the sole harness) — path resolution and `pi`'s own startup have each been tested separately (see below and "Known gaps"), but not yet together through a full live tick run. Do that before trusting it for real.
**Verified live**, repeatedly, against the packaged flake build and `diwan-dev-notes`: the no-op gate (an immediate second `tick` after one that found nothing new correctly does nothing), triage, the tag-consistency/escalation decision in step 3 (confirmed doing the right thing for a cross-cutting archive task and for a stale leftover agent it correctly declined to nudge), and Reports writing. **Not yet verified live**: the current (broadened) Reports bar and `TODO`/`DONE` format, and the session-closure teardown path (nobody has checked off a `Close diwan session` and had a tick actually process it yet) — both designed and committed, but no real run has exercised either.
## What `diwan do` actually does
@@ -76,32 +77,30 @@ Same split: `cmd_do` (CLI) decides identity/dispatch mechanics; `DO.md` governs
1. Parse `target`: if it matches `<existing file>:<line number>` (resolved against the content repo), it's heading mode; anything else is treated as an ad-hoc free-text prompt with no org heading involved (useful for manual testing, or tasks that genuinely don't belong in the notes).
2. **Heading mode:** read the headline at that line, check its `DIWAN_ID` property.
- Already has one → **continuation**. (If `--diwan-id` was also passed and disagrees with the stored one, that's an error, not a silent overwrite — a mismatch means something's confused and a human should look.) The existing ID is validated against herdr's naming rule before anything else happens, so a corrupted/hand-edited property fails with a clear message instead of a confusing herdr-side error later.
- No `DIWAN_ID` yet → **fresh**. Compute one: `--diwan-id` if given, else a slug of the heading text (`slugify`, capped and shaped to satisfy herdr's agent-name rule — see below), de-duplicated against every other `DIWAN_ID` already in the content repo's tree (`unique_id`). Write the property and insert `** [ ] Close diwan session` as the first child (`assign_diwan_id`), in one commit under the diwan identity. This is the *only* place a `DIWAN_ID` is minted or a Close-session item is inserted — mechanical, not left to agent judgment, so it's always consistent.
- No `DIWAN_ID` yet → **fresh**. Compute one: `--diwan-id` if given, else a slug of the heading text (`slugify`, capped and shaped to satisfy herdr's agent-name rule — see below), de-duplicated against every other `DIWAN_ID` already in the content repo's tree (`unique_id`). Write the property and insert `** TODO Close diwan session` as the first child (`assign_diwan_id`), in one commit under the diwan identity. This is the *only* place a `DIWAN_ID` is minted or a Close-session item is inserted — mechanical, not left to agent judgment, so it's always consistent.
3. **Ad-hoc mode:** `diwan_id` is `--diwan-id` or a generated `adhoc-<8 hex chars>`; always goes through the continuation dispatch path (see below) since there's no heading state to distinguish fresh from resumed.
4. Dispatch: fresh headings use `herdr_dispatch_fresh`; everything else (continuation headings, ad-hoc) uses `herdr_dispatch_continuation`. Prints the `diwan_id` either way — this is the command's only stdout contract, which is what lets `TICK.md` step 3 just run `diwan do ...` as an ordinary shell command and know what session it created.
**Continuation dispatch, concretely** (`herdr_dispatch_continuation`): check `herdr agent get <id>` — alive → just `agent prompt <id> <text>` (fire-and-forget, no `--wait`). Not alive (the normal case between ticks; CLI agent processes exit, they don't idle forever) → look up the tab by label in the `diwan` workspace, start a fresh agent in its existing root pane, prompt it. Tab itself gone too (e.g. closed by hand) → falls through to the same fresh-tab path `herdr_dispatch_fresh` uses. **Continuity across resumptions is carried by the heading's own content, not by agent conversation memory** — a resumed `do` agent has no idea what it said last time except what's written in the org file. This is a real, current limitation, not an oversight to "fix" without thinking about it — it's a direct consequence of principle #1 (org is ground truth) and keeps the system's state in exactly one place.
**Agent half** (`DO.md`): stay inside your own heading's subtree; if something needs doing elsewhere, ask in `:chat:` rather than reaching out of scope (cross-cutting work like this is now `tick`'s job to do directly, per the `:diwan:` tag section — don't try to talk another `do` session into doing it either); use the chat protocol for questions/FYIs; commit progress in small units under the diwan identity through the flock-wrapped recipe; never touch your own `Close diwan session` item — that's Marc's call, acted on by the next tick. One explicit exception to subtree-only: a `do` agent may append its own entry to `diwan.org`'s Reports log once it finishes something worth telling Marc about, since that's its only channel to him outside of a tick run — it still never edits or deletes an existing entry there, only `tick` does that. A tick that merely nudges an idle session (see `cmd_tick` step 3 above) does not report the nudge itself — it has no idea yet whether anything came of it.
**Agent half** (`DO.md`): stay inside your own heading's subtree; if something needs doing elsewhere, ask in `:chat:` rather than reaching out of scope; use the chat protocol for questions/FYIs; commit progress in small units under the diwan identity through the flock-wrapped recipe; never touch your own `Close diwan session` item — that's Marc's call, acted on by the next tick. One explicit exception to subtree-only: a `do` agent may append its own entry to `diwan.org`'s Reports log once it finishes something worth telling Marc about, since that's its only channel to him outside of a tick run — it still never edits or deletes an existing entry there, only `tick` does that.
**Verified live with Claude Code** (pre tool/content split, pre pi-only switch): dispatched for real against the "gather info about Greece trip" heading. Did real web research, wrote a structured result into the heading, correctly asked clarifying questions via a signed `:chat:` entry instead of guessing, left `Close diwan session` untouched, committed as `diwan <diwan@local>` with a descriptive message. (Historical — Claude Code is no longer a supported harness; see "One harness, chosen deliberately" above.)
**`pi`, verified live, ad-hoc dispatch only** (2026-11-08, herdr 0.9.5 + pi 1.5.0, pre-split; re-confirmed 2026-11-09 post-split, post-switch against the fresh `diwan-dev-notes` repo): `diwan do "<prompt>" --diwan-id <id>` correctly starts a herdr `pi` agent (detected as `agent: "pi"`), with no trust/approval prompt blocking it even in a content repo `pi` had never seen before (confirming it really doesn't share Claude Code's first-run trust-dialog problem — see "Known gaps"), loads `AGENTS.md` as context, and replies exactly as instructed. A second `diwan do` call against the same `--diwan-id` reuses the same pane/tab and agent (continuation dispatch, not a fresh one) and replies correctly again. **Not yet exercised**: a real `tick` or `do` run against actual heading content with `pi` as the agent (only ad-hoc free-text prompts so far) — worth a full pass before relying on it for real delegated work.
**Verified live**: real web research dispatched against a heading, writing a structured result into it and asking clarifying questions via a signed `:chat:` entry instead of guessing, committed under the diwan identity. Ad-hoc dispatch and continuation (reusing the same pane/tab/agent on a second call rather than creating a new one) are both confirmed working, including cold-starting in a content repo `pi` had never run in before (no trust-dialog block — unlike some other harnesses, `pi` doesn't have one). **Not yet verified live**: `diwan do` in heading mode (fresh `DIWAN_ID` minting) specifically with `pi` as the dispatched agent — the heading-mode mechanics have only been exercised end-to-end via a tick's own self-contained-work dispatch, not tested in isolation since the pi-only switch.
## `DIWAN_ID` ⇄ herdr, precisely
A `DIWAN_ID` *is* a herdr tab label in the `diwan` workspace, and (while its agent process is alive) also that agent's name. This is why it must satisfy herdr's agent-name rule: `^[a-z][a-z0-9_-]{0,31}$`. `slugify()` enforces this at mint time (lowercase, non-alnum → hyphen, truncated to 24 chars, forced to start with a letter) and `cmd_do` additionally validates *every* `DIWAN_ID` it's about to dispatch — including ones already stored in a heading — against that same regex before touching herdr, erroring out with a clear message rather than letting herdr reject it after the CLI has already created a tab (which used to leave an orphan tab behind; fixed by closing the just-created tab on any `agent start` failure).
Lifecycle: **fresh** (no property) → `diwan do` mints an ID, creates the tab+agent, heading now has `DIWAN_ID` + open (`[ ]`) `Close diwan session` → **active** (any further `tick`/`do` on this heading is a continuation; `tick` explicitly leaves active sessions alone) → Marc checks off `Close diwan session` (`[X]`) → next `tick` actually retires it (per `TICK.md` step 3): closes the herdr tab for real, removes the `:session:` tag, leaves the `DIWAN_ID` property in place as a historical record — **retired** (the heading can still be found later by that `DIWAN_ID`, but there's no live terminal and no tag pointing at one).
Lifecycle: **fresh** (no property) → `diwan do` mints an ID, creates the tab+agent, heading now has `DIWAN_ID` + open (`TODO`) `Close diwan session` → **active** (any further `tick`/`do` on this heading is a continuation; `tick` explicitly leaves active sessions alone) → Marc marks `Close diwan session` `DONE` → next `tick` retires it: closes the herdr tab, removes the `:session:` tag, leaves the `DIWAN_ID` property in place as a historical record — **retired** (the heading can still be found later by that `DIWAN_ID`, but there's no live terminal and no tag pointing at one).
## Known gaps / explicit non-goals (don't "fix" these without discussing)
- **Tool/content repo split just happened (2026-11-09), not yet re-verified live.** This repo used to be both diwan's own dev copy *and* a throwaway test-content repo at once; it's now tool-only, with content split out to a sibling `diwan-dev-notes` repo. The CLI was updated to resolve `DIWAN_DIR` (its own install dir, `Path(__file__).resolve().parent`, overridable via the `DIWAN_DIR` env var) separately from `DIWAN_NOTES_DIR` (`git rev-parse --show-toplevel` from the invoking directory by default, overridable via the `DIWAN_NOTES_DIR` env var), and the dispatch prompts now pass `DIWAN_DIR`-absolute paths for `PROTOCOL.md`/`TICK.md`/`DO.md`. This has been syntax-checked and smoke-tested for path resolution (including both env var overrides, from outside any repo), but **not yet run through a full live tick/do pass with a real dispatched agent** against the new two-repo layout — do that before trusting it for real.
- **No flake yet.** The split prepared the ground (tool repo is self-contained, no content-repo assumptions baked into its own files) but there's no `flake.nix` in this repo yet, and the shebang still runs via a bare `nix shell nixpkgs#python3` rather than a proper package. Until that lands, "installed on `PATH`" means manually symlinking/aliasing `diwan` somewhere on `PATH` yourself. `DIWAN_DIR` exists specifically so a future flake's install layout (e.g. docs copied into `$out/share/diwan/` alongside a wrapped binary) can override the default `Path(__file__).resolve().parent` guess if it doesn't hold.
- **Retired-session teardown is designed but not yet verified live.** `TICK.md` step 3 now has the tick agent actually close the herdr tab (`herdr tab close`) and strip the `:session:` tag once `Close diwan session` is checked off, rather than just reporting that it happened — but no real tick run has exercised this path yet. Worth a close look the first few times it fires: does the tab lookup by `DIWAN_ID` label reliably find the right tab, does tag-stripping leave `:diwan:`/other tags intact, does it behave sanely if the tab was already closed by hand.
- **Tick scheduling beyond a plain systemd timer.** `samfelag`'s module runs `diwan tick` on a configurable `OnCalendar` cadence — that's it, no smarter scheduling, no overlap detection between runs. Fine for a personal single-host setup; revisit only if that stops being true.
- **Retired-session teardown is designed but not yet verified live.** `TICK.md` step 3 has the tick agent close the herdr tab (`herdr tab close`) and strip the `:session:` tag once `Close diwan session` is marked `DONE` — but no real tick run has exercised this path yet. Worth a close look the first few times it fires: does the tab lookup by `DIWAN_ID` label reliably find the right tab, does tag-stripping leave `:diwan:`/other tags intact, does it behave sanely if the tab was already closed by hand.
- **The new (broadened) Reports bar and `TODO`/`DONE` format are unverified live.** Designed and committed, but no real tick run has written a Reports entry under the current rules yet.
- **No real conversation continuity across resumption.** Covered above — intentional for now, but worth revisiting if resumed `do` sessions turn out to need more context than the heading text carries.
- **`pi` is smoke-tested for ad-hoc dispatch, but not yet exercised against real heading content.** `--approve` has been confirmed live for ad-hoc `diwan do` dispatch and continuation, including against a content repo `pi` had never run in before (no trust-dialog block, unlike Claude Code — see "One harness, chosen deliberately" above). Still untested: a real `diwan tick` run, and `diwan do` against a real `<file>:<line>` heading (DIWAN_ID minting, `:chat:` protocol, scope discipline) rather than a free-text ad-hoc prompt.
- **`diwan do` in heading mode hasn't been exercised in isolation since the pi-only switch.** Fresh `DIWAN_ID` minting works as part of a tick's own dispatch decision, but hasn't been separately smoke-tested the way ad-hoc dispatch and continuation have.
- **`diwan remote <machine>` is implemented but not verified against a real remote herdr server.** Argument parsing, the no-herdr-on-`PATH` fallback, and a fast failure on an unknown `--machine` label are all confirmed; an actual `herdr machine add` + `diwan remote` round trip against a live remote server is not.
- **`diwan.org`'s `Config` heading is a placeholder, not wired up.** PROTOCOL.md documents an intended home for configuration values governing diwan's own behavior (tick cadence, tab retention, that kind of thing), but the CLI reads none of it — the env vars (`DIWAN_DIR`, `DIWAN_NOTES_DIR`, `DIWAN_PI_APPROVE`, `DIWAN_MAX_TICK_TABS`) remain the only real knobs. Don't assume anything in that heading does something until the CLI actually reads it.
- **The `:diwan:`/`:session:`/`:chat:` tag split, and tick doing cross-cutting maintenance directly, is new and unverified live.** Before this split, `:diwan:` alone meant what `:session:` means now, and anything non-trivial got a dedicated `do` session regardless of whether the work was subtree-shaped or genuinely repo-wide — which is exactly what motivated the split (a real `do` session hit the subtree wall trying to do an archive/cleanup/folder-move task). No tick run has yet exercised the new three-way decision in `TICK.md` step 3 (inline / `:session:` dispatch / tick-direct), the tag-consistency backfill in step 2, or tick actually doing a multi-file maintenance job itself across one or more runs — worth a close look at the first few real runs that hit this path.
- **Concurrency is scope discipline, not real locking**, beyond the git-commit flock. Two `do` agents editing truly disjoint subtrees should be fine; nothing stops two agents editing the *same* subtree if the scope rule is violated. There's no enforcement, only convention (PROTOCOL.md).
- **No automated tests.** Everything described as "verified live" above was exercised by hand against a real herdr install. There is no regression suite — a future session changing `diwan` should re-run at least the manual smoke tests (tick no-op gate, tick dispatch, do fresh dispatch, do continuation, a deliberately-invalid `DIWAN_ID`) before trusting a change.
- **No automated tests.** Everything described as "verified live" above was exercised by hand against a real herdr + `pi` install. There is no regression suite — a future session changing `diwan` should re-run at least the manual smoke tests (tick no-op gate, tick dispatch, do fresh dispatch, do continuation, a deliberately-invalid `DIWAN_ID`) before trusting a change.