S
How the viewer works
Who holds input, the components the page is built from, and the rules it follows.
SystemW
Watching the agent
The main case. A glance answers which app, what the agent just did, and whether it is stuck.
9 screensC
Taking control
Borrow the device for a moment, then hand it back. Never two drivers at once.
9 screensR
Remote and connection
Honest about the box being far away: connecting, starting, stale, unreachable, ending soon.
5 screensE
Ended and failures
What a finished or broken session looks like, and what ended it.
6 screensA
Sharing and access
Links for teammates, sign-in, and devices you can only watch.
7 screensP
From a phone
The same page at 380 points wide, thumb first.
3 screens
Who holds input
Boxes are control.holder values. Rounded ends start or finish. Click a state to open its screen.
Session upfrostline up by an agent.
Components
Existing is used as is, adapted is an existing component in a new place, new is built for this feature.
Tapped “Add environment” · 2 s agoLive · 84 ms
No input for 48 s. Handing back in 12 s.Keep control
Status lineNew
new: viewer/status-lineWhat happened, then how fresh. One line, never two.
Agent is drivingTake control
You have controlGive back
Holder chipNew
new: viewer/holder-chipAlways top right, always true.
Tap markerNew
new: viewer/action-markerBlue for the agent, grey for you.
Design rules
How this fits the product.
- The device is the heroChrome sits at the edges. By default the page shows a name, who is driving, the device, and one line.
- Never pretend to be liveEvery frame has an age. A stale frame dims and says when it is from; a still screen is not stale.
- One driver at a timeThe server holds a lease. The holder chip is always visible, and taking or losing control is always announced.
- Say it onceA notice replaces the status line; it never stacks under it or repeats the chip.
- Details on demandSteps, screen, logs and run open from the status line into a drawer, a sheet on a phone.
- Hide what can't applyNo Take control for visitors, no Home (Frostline has no home screen), no Extend without the owner token.
W1Agent tapping
control.holder = agent · stream liveThe default view. The device fills the page, the agent's last action sits under it in one line, and the tap shows where it landed.
- exists: GET /v1/devices/{id}/stream.avcc (H.264) and stream.mjpeg
- new: GET /v1/devices/{id}/events (SSE) emits action {seq, t, verb, label, point [x,y] in points, ok, ms, by: agent}
- source for actions: the run's agent-calls.jsonl (exists, written by every CLI call) tailed by the viewer
- new: frame meta envelope (tag 5) {seq, presentedAt}; latency = input-socket ping RTT (ws op 9/10 exist)
- new: GET /v1/devices/{id}/control → {holder: agent, since}
2 adapted3 new
W2Agent typing
control.holder = agent · last action = typeTyping shows the field it went into, not a finger. Secure fields show dots, never the text.
- action event for type carries field label + frame [x,y,w,h] (the CLI result already has field and responder)
- rule: when the responder is a secure text field, the event says “Typed 12 characters” (no text leaves the box)
- the marker fades after 2.5 s; the line keeps the last action until the next one
2 adapted2 new
W3Steps drawer
drawer = stepsClick the status line and it opens into the steps: every call the agent made, newest at the bottom, its result in plain words.
- new: GET /v1/devices/{id}/steps?after=seq → the run's agent-calls.jsonl, mapped to {verb, label, point, ok, ms, error}
- typed text redacted in secure fields (same rule as the events stream)
- steps keep streaming from /events while the drawer is open; the list pins to the bottom unless scrolled up
2 adapted3 new
W4Screen drawer
drawer = screenThe screen as the agent reads it. Hovering a row outlines that element on the device, so a person can see why the agent picked it.
- exists: frostline ui-tree (CLI and MCP); new: GET /v1/devices/{id}/tree → labels, roles, frames
- refetch on the next action event, never on a timer (ui-tree costs a CLI start, about 200 ms)
- hover outline uses the row's frame in points; nothing is sent to the device
2 adapted3 new
W5Logs drawer
drawer = logsThe app's own console. Real lines from this T3 Code run: its update check fails with HTTP 400, which an agent would otherwise have to dig for.
- exists: frostline logs (JS console and app stderr, cursor based); new: GET /v1/devices/{id}/logs?cursor=
- errors tint amber; secure-field text is already scrubbed by the CLI (canon4.26)
- never read files under the state folder from the page; the route goes through the CLI
2 adapted3 new
Time left and how the session is doing. Screen changes, not frame rate: a still screen is 0 whatever the app does.
- exists: frostline status → remaining_s, up_s, boot; frostline stats → app cpu_pct, rss_mb, display.screen_changes_per_s
- new: GET /v1/devices/{id}/session → {state, startedAt, boundEnd, remaining_s, kit, run}
- app fps only when stats reports one (Metal apps); never show compositor frames as app fps
2 adapted3 new
W7Device switcher
devices = [solo: T3 Code, d1: Hello, d2: free]The title is the switcher. Each device says who is driving it, so you pick the one you mean to watch.
- exists: GET /v1/devices (live sessions plus free slots)
- new fields per device: holder (agent / you / another viewer), app display name, startedAt
- Start a device opens A6's app list (exists: POST /v1/devices/{id}/open + GET /v1/ops/{opId})
2 adapted3 new
W8Pixel accurate
zoom = pixelFor checking a rendering detail: one device pixel per screen pixel, scrolls like an image. Fit is one key away.
- Fit (default), Point accurate (402 × 874) and Pixel accurate (1206 × 2622), as in Xcode's Simulator Window menu
- pixel accurate needs 3x frames: new ?scale=3 on stream.avcc, or a still from screenshot.png?native=1 (CLI --native exists)
- ⌘1 fit, ⌘2 points, ⌘3 pixels; the choice is remembered per browser
1 adapted2 new
W9Watching, dark
appearance = dark (page and device)The page follows the viewer's system appearance. The device keeps its own; here both are dark.
- page theme: prefers-color-scheme, overridable in the more menu
- device appearance is separate (frostline appearance, exists); it is an input action, so only the holder changes it
- tap marker at the agent's real tap point (300, 760)
2 adapted3 new
C1Take control
control.holder = agent · confirmingThe warning grows from the button. One decision, the consequence in plain words.
- new: POST /v1/devices/{id}/control {action: take} → {holder: viewer, viewerId, idleTimeout_s: 60, expiresAt}
- the agent's next input verb (tap, type, swipe, key...) fails with error human-has-control and next: frostline control take
- after the first take on a device, the tray is skipped for this browser for an hour (localStorage)
2 adapted3 new
C2You have control
control.holder = viewerThe device gets a blue ring and the input tools slide in beside it. Your own taps show as a grey ring.
- exists: WS /v1/devices/{id}/input (live finger, HID keys, text, rotate, back)
- new: the input socket accepts touch only from the lease holder; others get 0x90 {error: agent-has-control}
- every input renews the lease's idle timer; the server, not the page, decides when it expires
3 adapted3 new
C3Typing and paste
control.holder = viewer · focus = deviceClicking the screen gives it the keyboard. Return, arrows and Backspace go as keys, paste goes as text.
- exists: input tags 0x06 (HID key) and 0x10 (text, up to 2000 chars)
- ⌘ shortcuts stay with the browser except ⌘V (paste) and the viewer's own (⌘S screenshot, ⌘[ back)
- the tool rail's tooltips name each shortcut
3 adapted2 new
C4Handing back soon
control.holder = viewer · idle 48 sControl goes back by itself so an agent is never stuck behind a forgotten tab. The ring counts down for the last 15 s.
- lease idleTimeout_s = 60 (server side); the page shows the countdown from expiresAt
- Keep control = POST control {action: renew}
- closing the tab releases the lease at once (socket close)
3 adapted2 new
C5Given back
control.holder = agentOne click, no confirm. The rail folds back to the watch tools and the line says it once.
- POST control {action: give} → holder: agent
- the agent's CLI learns nothing extra: its next verb simply works again
- events stream emits control {holder: agent, by: viewer, reason: given}
2 adapted2 new
C6Agent took it back
control.holder = agent · reason = overrideThe agent can override, and the person always sees that it happened and when.
- new CLI: frostline control take (and MCP control) moves the lease to the agent at once
- events: control {holder: agent, reason: override}; the viewer's input socket gets 0x90 {error: control-taken}
- the person's half-finished gesture is cancelled with an end event so the app never keeps a stuck finger
2 adapted3 new
C7Agent handed you control
control.holder = viewer · reason = offeredFor “take me to this screen”: the agent drives there, then gives the device to the person.
- new CLI: frostline control give --to viewer (MCP control {action: give}) when a viewer is watching
- if no viewer is watching, the call says so and keeps the lease with the agent
- the agent's reply in chat can link straight here (view_url, exists)
3 adapted3 new
C8Agent is waiting
control.holder = viewer · agent blockedIf the agent is blocked by you, you should know. The line changes and Give back moves into it.
- the CLI's refused verb (human-has-control) also writes an action event {ok: false, error: human-has-control}
- the agent decides whether to wait or override; the page only reports
3 adapted2 new
C9Rotated
orientation = landscape-leftRotate turns the bezel, not the page. The rail stays where it was.
- exists: input tag 0x07 → frostline rotate, then a new 0x82 screen config
- Home is not offered: Frostline has no home screen (the server refuses home/lock); Back and Relaunch cover it
3 adapted2 new
R1Connecting
stream = connectingNo fake picture. The bezel holds the place, and three short steps say what it is waiting for.
- login (exists) → GET /v1/devices (exists) → first envelope on stream.avcc (JPEG seed, tag 4)
- after 10 s on one step, the step says what to check (R4)
2 adapted3 new
R2Device starting
session.state = startingWhen someone opens the page mid-boot, it shows the real boot phases from this run, not a spinner.
- exists: up's boot timeline (darlingserver 2.59 s, app_proc 2.68, first_frame 5.54, ui-ready 7.88 on this T3 run)
- new: session event {state: starting, boot: {...}} while up runs; the estimate is the last up_s for this app
- the picture appears at first_frame, before ready, tinted until ready
2 adapted3 new
R3Stale, reconnecting
stream = stale (no envelope for 8 s)When frames stop, the picture says how old it is. A still screen is not stale: the server's keepalive proves the stream is up.
- exists: keepalive (the newest frame again every 1 s) on stream.avcc
- stale = no envelope (frame or keepalive) for 2.5 s; reconnect with backoff 1, 2, 4 s
- frame meta (new tag 5) gives presentedAt, so the age is the frame's, not the socket's
2 adapted2 new
R4Can't reach the box
page cannot reach /v1/hostAfter 30 s of failed reconnects, say so and say what to check. Measured today: the old page only logged “input closed 1006”.
- fetch /v1/host fails (network) vs 401 (token, goes to A4) vs 404 device (goes to E5)
- keep the last frame, greyed, so the person still sees where things were
2 adapted2 new
R5Ending soon
session.remaining_s = 112Time left only appears when it matters: the last 5 minutes. Extend is one tray.
- exists: remaining_s and bound_end (frostline status); default bound 23 min (--minutes)
- new: POST /v1/devices/{id}/extend {minutes} and CLI frostline extend; the supervisor rereads bound_end
- watchers can extend only with the owner's token; visitors never
2 adapted3 new
E1Session ended
session.state = ended · reason = downThe last frame stays, greyed, with what ended it. Measured today: the old page kept saying “watching” over a dead frame.
- new: the server keeps the last ended session per slot for 1 h: {endedAt, reason, lastFrame, steps}
- reason from session.json (exists: down, bound, failed); lastFrame saved at down (new)
- Start again = POST open with the same app (exists), owner token only
2 adapted1 new
E2Time limit reached
session.state = ended · reason = boundName the limit and how to change it, so nobody thinks it crashed.
- reason bound comes from the supervisor at bound_end (exists)
- the copy uses the real --minutes value of that run
2 adapted1 new
E3App quit and restarted
app.exited = signal 9 · restarted = trueMeasured today on fl-clean2: the app was killed and the supervisor restarted it, and the old page showed nothing.
- the supervisor already restarts the app (new pid, app_alive true); new: app event {pid, exit, restarted}
- a restart also lands in the steps as a warning row, so it survives after the line moves on
2 adapted2 new
E4App quit, not restarted
app.alive = false · session.state = readyThe session is up but the app is gone. Relaunch is the one action; logs are one click away.
- exists: frostline relaunch; new route POST /v1/devices/{id}/relaunch (holder or owner only)
- exists: status app_alive; new: app event pushed instead of polled
2 adapted3 new
E5Simulator stopped
session.state = failed · supervisor_alive = falseDifferent words from a normal end: something broke, and a report is the useful next step.
- exists: session.json state failed / orphaned with why; frostline report builds a scrubbed bundle
- new: POST /v1/devices/{id}/report → the report path for the person to share
2 adapted3 new
E6Ended, steps
session.state = ended · drawer = stepsAfter the fact, each step shows the frame it acted on and where it tapped. Arrow keys step through.
- new: per-step frame thumbnails: the viewer saves the presented frame at each action event (one webp, ~30 KB)
- kept with the ended session for 1 h, then only steps remain
- a full clip exists only if someone pressed Record (frostline record, exists)
2 adapted4 new
A1Share a link
visitors = []One tray: watch-only by default, an expiry, and copy. The token itself is never shown.
- exists: frostline view visitor add --device --expires (1 min to 30 days) and GET /visit, POST /v1/visit
- new: POST /v1/devices/{id}/share {expires_s, control: false} → the visitor link (owner token only)
- the link works only through --public-origin or the tailnet; on loopback-only boxes the tray says so
2 adapted3 new
A2Visitor watching
viewer = visitor (watch only)A teammate sees the device, the agent's line and the tap. No take control, no drawer, no switcher.
- exists: visitor cookie scoped to one device (screenshot, streams, input, attach)
- new: visitors get /events filtered to action and session (no logs, no tree, no steps)
- new: input refused for visitors unless the link allows control
2 adapted3 new
A3Link expired
visit → 410 expiredPlain and final. No token field, nothing to try.
- exists: POST /v1/visit → 410 expired, 409 in use elsewhere, 429 too many tries
- 409 copy: “This link is open in another browser.”
2 adapted1 new
Only when there is no session cookie and no #t= link. One field, one command to get a token.
- exists: POST /v1/login (bearer → HttpOnly cookie); #t=TOKEN links from view_url sign in silently
- the token is never stored by the page beyond the cookie
2 adapted
A5Token refused
login → 401Say why it usually happens. After 10 tries the server waits; the page says how long.
- exists: 401 unauthorized, 429 too-many-attempts (10 failures)
2 adapted
A6No device
devices = [] (only free slots)Measured today: the old page showed a black rectangle and a log line. This says what is true and offers the apps on the box.
- exists: GET /v1/devices, POST /v1/devices/{id}/open {app}
- new: GET /v1/apps (frostline apps, exists in the CLI) so the buttons are the box's real apps
2 adapted1 new
A7Another member's device
multi-user box · device owner ≠ youOn a team box you can watch a teammate's device but not take it. The chip says why without naming anyone.
- exists: input_refused() (frostline_users.device_check) blocks live input for other members
- new: GET /v1/devices returns owner: self / other so the page hides Take control
2 adapted2 new
P1Phone: watch, steps, ended
viewport = 380 ptOn a phone the device still leads. Take control sits under the thumb; the drawer becomes a sheet over the bottom half.
- same routes as W1, W3, E1; layout switches below 720 px wide
- the sheet keeps the device visible above it, so steps and screen can be compared
2 adapted4 new
P2Phone: control
control.holder: agent → viewer → agentIn control, the tools move to a bottom bar; Give back stays next to the line that says you have it.
- touch on a phone is a real finger: the live finger stream (exists) needs no change
- Type opens the phone's own keyboard into a hidden field and sends 0x10 text
2 adapted4 new
P3Phone: dark states
appearance = darkThe same states in dark, where most late-night glances happen.
- page theme follows the phone; colors are tokens, so every state has both
3 adapted3 new