The protocol

Every consumer gets the same JSON, whichever transport it listens on.

Transports

Both bind loopback only, and both come up when the app launches — not when capture starts. That distinction matters: it is what lets POST /start work on a process that has never been told to listen, and it is the fix for a bug where stopping capture orphaned the listeners, leaving ports bound to a deallocated server that accepted connections and answered nothing.

They are not alternatives. Run both; a browser reading over SSE and a tool holding a socket can want the same stream at the same time.

Both are behind the token — ?access_token= on the SSE URL, a first frame of {"auth": "<token>"} on the WebSocket. Authentication has the whole scheme.

Frames

{
  "type": "partial",
  "text": "and this week we are looking at",
  "ts": 1788569799791,
  "seq": 41,
  "audioStart": 5.04,
  "audioEnd": 11.36,
  "lagMs": 180
}

text and ts are the original contract and have not moved. Everything else is additive: a consumer reading only text is unaffected.

On the wire, SSE delivers one frame per data: line; the WebSocket delivers one frame per message. Same bytes.

One thing on the stream is not a frame: when the engine changes, both transports carry { "event": "engine", "engine": "fluid320", "label": "Parakeet 320 ms" }. It has event where a frame has type, so a consumer that only handles frames skips it — the snippets below show the one-line check — and Engine selection says when it is sent.

Partials drive things; finals are for the record

A partial arrives roughly every 200 ms, about one word each, about 180 ms behind the speaker. That is what should drive a scroll or fire a cue.

A final arrives 1.6–3.1 s after the speech it describes, because the engine waits out its end-of-utterance debounce (1280 ms of silence) before deciding an utterance ended, and then still has to finish decoding. That is fine for what finals are for — settled text, and settling takes evidence — but nothing time-sensitive should wait for one.

text is the current utterance, not the session

Each final closes an utterance and the next partial starts empty. A consumer that wants a running transcript accumulates the finals itself.

This is not merely a preference. The underlying engine latches its end-of-utterance flag until it is reset and accumulates the transcript indefinitely otherwise, so an earlier build finalised only the first utterance of a session and grew text without bound — thirty-five seconds in, every frame still carried every word since second three, five times a second. Over a talk that is untenable.

Resetting between utterances costs the engine’s token timings, which restart at zero. The sidecar keeps its own offset so audioStart and audioEnd stay absolute on the session timeline across every reset:

That is what makes the timestamps usable for placing markers rather than merely ordering words.

On lagMs being signed, and sometimes absent

Negative means the engine reported audio ahead of what we had captured — a real clock disagreement, surfaced rather than clamped. Absent means the engine did not report timing for that hypothesis.

Why consumers should use audioEnd, not ts

Arrival time is a lossy proxy for spoken time. Delivery clusters rather than ticking evenly, so ts carries the jitter of the delivery schedule on top of the actual timing. Anything placing a marker accurately, or leading a scroll by a fixed amount, wants audioEnd.

Consuming it

const token = '…'  // the contents of ~/Library/Application Support/Sonocles/token

const es = new EventSource(`http://127.0.0.1:7357/events?access_token=${token}`)
es.onmessage = (e) => {
  const f = JSON.parse(e.data)
  if (f.event === 'engine') return picker.select(f.engine)   // not a frame
  if (f.lagMs != null) { /* trust f.audioEnd for timing */ }
}

const ws = new WebSocket('ws://127.0.0.1:7358')
ws.onopen = () => ws.send(JSON.stringify({ auth: token }))
ws.onmessage = (e) => {
  const m = JSON.parse(e.data)
  if ('status' in m) return   // the auth answer; frames have no status
  if (m.event === 'engine') return picker.select(m.engine)   // not a frame
  handle(m)
}

TOKEN="$(cat ~/Library/Application\ Support/Sonocles/token)"
curl -sN "http://127.0.0.1:7357/events?access_token=$TOKEN"

A consumer can attach before capture starts and stay attached across stops. The stream opens either way; frames arrive only while listening. The API reference has the frame schema as the code is tested against it.