feat(api): streaming agent turns — LlmClient.stream() + SSE chat endpoint #341

Merged

james merged 1 commit from 340-agent-streaming into main 2026-06-29 01:49:24 +00:00

Conversation 1 Commits 1 Files changed 13 +1362 -94

james commented 2026-06-29 01:41:30 +00:00

Owner

Copy link

Token-by-token streaming for the built-in agent (ADR-0029 §3): a stream() method on the LlmClient, a unified streaming agent-loop core, and content-negotiated SSE on the two conversation POST routes. No DB schema change (reuses conversations/messages). Part of epic #47.

What's in it

• LlmClient.stream(req) (required interface method) yielding text_delta / tool_call / done events; generate() stays for non-loop callers. Implemented on both adapters:
  • Anthropic — SDK messages.stream(...): forwards text deltas live, accumulates tool_use blocks, emits done from normalizeResponse(stream.finalMessage()) (reused so done.result ≡ generate()).
  • OpenAI-compatible — fetch with stream: true, hand-parsed SSE chunks (buffered line split, indexed tool_calls argument accumulation, finish_reason); a shared buildResult(...) keeps streaming and non-streaming in lockstep.
• Unified loop — the turn body is now a single private async generator (driveLoopEvents). runTurn/resumeTurn keep their exact signatures and drain it (JSON behaviour byte-for-byte unchanged — the #339 loop tests stayed green untouched); streamTurn/streamResume forward its ConversationEvents. Same persistence order, MAX_TOOL_ROUNDS guard, and propose-then-confirm write-pause as before.
• SSE on the two routes — content-negotiate on Accept: text/event-stream. Setup failures (404/401/409/llm_not_configured) stay real HTTP status codes via a pre-pull of the first event before the 200 is sent; once headers are out, a mid-stream provider failure surfaces as a terminal error event. JSON mode is unchanged.

Adapter event shape: text_delta | tool_call | done. Conversation/SSE event shape (lib/agent/events.ts): text_delta, tool_call, tool_result, message (carries toMessageDto, never the raw entity), awaiting_confirmation, done, error. SSE wire: event: <type>\ndata: <json>\n\n.

Verification (all run locally on this branch)

• typecheck ✓ · lint ✓ (0 warnings)
• test against both engines (ephemeral Postgres via TEST_POSTGRES_URL): 1206 passed, 0 skipped — Postgres leg ran, no engine-specific SQL; +23 tests (adapter streaming, streaming loop event sequences + persisted-row parity, SSE route incl. JSON-mode-unchanged and 401/404-as-JSON-not-SSE)
• openapi:check ✓ up to date · openapi:coverage 116 pairs (SSE isn't a JSON DTO; no new schema)
• semgrep full CI pack set ✓ 0 findings (incl. an explicit scan of the new/changed core files)

Out of scope

• The chat UI client (next ticket consumes this). Voice. Effort/thinking tuning.

Closes #340

🤖 Generated with Claude Code

Token-by-token streaming for the built-in agent (ADR-0029 §3): a `stream()` method on the LlmClient, a unified streaming agent-loop core, and content-negotiated SSE on the two conversation POST routes. No DB schema change (reuses `conversations`/`messages`). Part of epic #47. ## What's in it - **`LlmClient.stream(req)`** (required interface method) yielding `text_delta` / `tool_call` / `done` events; `generate()` stays for non-loop callers. Implemented on both adapters: - **Anthropic** — SDK `messages.stream(...)`: forwards text deltas live, accumulates `tool_use` blocks, emits `done` from `normalizeResponse(stream.finalMessage())` (reused so `done.result` ≡ `generate()`). - **OpenAI-compatible** — `fetch` with `stream: true`, hand-parsed SSE chunks (buffered line split, indexed `tool_calls` argument accumulation, `finish_reason`); a shared `buildResult(...)` keeps streaming and non-streaming in lockstep. - **Unified loop** — the turn body is now a single private async generator (`driveLoopEvents`). `runTurn`/`resumeTurn` keep their exact signatures and **drain** it (JSON behaviour byte-for-byte unchanged — the #339 loop tests stayed green untouched); `streamTurn`/`streamResume` forward its `ConversationEvent`s. Same persistence order, `MAX_TOOL_ROUNDS` guard, and propose-then-confirm write-pause as before. - **SSE on the two routes** — content-negotiate on `Accept: text/event-stream`. Setup failures (404/401/409/`llm_not_configured`) stay real HTTP status codes via a **pre-pull** of the first event before the 200 is sent; once headers are out, a mid-stream provider failure surfaces as a terminal `error` event. JSON mode is unchanged. Adapter event shape: `text_delta` | `tool_call` | `done`. Conversation/SSE event shape (`lib/agent/events.ts`): `text_delta`, `tool_call`, `tool_result`, `message` (carries `toMessageDto`, never the raw entity), `awaiting_confirmation`, `done`, `error`. SSE wire: `event: <type>\ndata: <json>\n\n`. ## Verification (all run locally on this branch) - `typecheck` ✓ · `lint` ✓ (0 warnings) - `test` against **both engines** (ephemeral Postgres via `TEST_POSTGRES_URL`): **1206 passed, 0 skipped** — Postgres leg ran, no engine-specific SQL; +23 tests (adapter streaming, streaming loop event sequences + persisted-row parity, SSE route incl. JSON-mode-unchanged and 401/404-as-JSON-not-SSE) - `openapi:check` ✓ up to date · `openapi:coverage` 116 pairs (SSE isn't a JSON DTO; no new schema) - semgrep full CI pack set ✓ **0 findings** (incl. an explicit scan of the new/changed core files) ## Out of scope - The chat UI client (next ticket consumes this). Voice. Effort/thinking tuning. Closes #340 🤖 Generated with [Claude Code](https://claude.com/claude-code)

james added 1 commit 2026-06-29 01:41:30 +00:00

feat(api): streaming agent turns — LlmClient.stream() + SSE chat endpoint ... 729d3fdb65

Add token-by-token streaming to the built-in agent (ADR-0029 §3): a
`stream()` method on the LlmClient, a unified streaming agent-loop core,
and content-negotiated SSE on the two conversation POST routes. No DB
schema change (reuses conversations/messages).

- LlmClient gains a required `stream(req)` yielding text_delta / tool_call
  / done events; `generate()` stays for non-loop callers. Both adapters
  implement it — Anthropic via the SDK's messages.stream + finalMessage,
  OpenAI-compatible via fetch stream:true + hand-parsed SSE chunks. The
  done.result reuses the same normalization so it can't drift from
  generate().
- The agent loop is refactored to a single private async generator
  (driveLoopEvents) that is the only turn body. runTurn/resumeTurn drain
  it (JSON behaviour byte-for-byte unchanged); streamTurn/streamResume
  forward its ConversationEvents. Same persistence order, MAX_TOOL_ROUNDS
  guard, and propose-then-confirm write-pause as before.
- The messages/resume routes negotiate on Accept: text/event-stream.
  Setup failures (404/401/409/llm_not_configured) stay real HTTP status
  codes via a pre-pull of the first event; once 200 + headers are sent, a
  mid-stream provider failure surfaces as a terminal `error` event.

Closes #340

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

forgejo-actions commented 2026-06-29 01:48:56 +00:00

Copy link

📊 Test coverage

Patch coverage: no testable lines changed.

Overall (app/, lib/, db/, excluding UI per ADR-0019):

Metric	Value	Soft target
Lines	79.3% ✅	≥ 50%
Branches	70.9% ⚠️	≥ 75%
Functions	80.2%	informational

Soft thresholds per ADR-0019. Coverage is informational and does not block merge.

<!-- coverage-comment --> ## 📊 Test coverage **Patch coverage:** no testable lines changed. **Overall** (`app/`, `lib/`, `db/`, excluding UI per ADR-0019): | Metric | Value | Soft target | |---|---|---| | Lines | 79.3% ✅ | ≥ 50% | | Branches | 70.9% ⚠️ | ≥ 75% | | Functions | 80.2% | informational | Soft thresholds per [ADR-0019](docs/adr/0019-coverage-soft-targets.md). Coverage is informational and does not block merge.

james merged commit 8b758329ca into main 2026-06-29 01:49:24 +00:00

james deleted branch 340-agent-streaming 2026-06-29 01:49:24 +00:00

james referenced this pull request from a commit 2026-06-29 01:49:26 +00:00

Merge pull request 'feat(api): streaming agent turns — LlmClient.stream() + SSE chat endpoint' (#341) from 340-agent-streaming into main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

area:auth

Authentication and authorization

  

area:ci

Forgejo Actions, build, test, security, release

  

area:db

Database abstraction, migrations, persistence

  

area:infra

Containerization, deployment, foundational tooling

  

area:native

React Native app (Android + Linux Flatpak)

  

area:pwa

PWA / web frontend

  

area:service

Backend service, API routes, persistence

  

epic

High-level tracker; links several feature tickets

  

feature

A user-facing feature

  

foundation

Foundational work that other tickets build on

No labels

area:auth

area:ci

area:db

area:infra

area:native

area:pwa

area:service

epic

feature

foundation

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

james/carol!341

No description provided.