Flows as JSON (and the Flow Studio)

Because every [Guard] atom is a named, parameterized predicate, a Flow is fully serializable — the same DAG you build with Flow::new()…build() in Rust round-trips through JSON. This page documents that JSON format; the SessionSpec document (gemini_adk_fluent_rs::spec) that turns a flow into a complete runnable application — tools, extraction, phases, watchers, fragments, and an embedded test suite — without writing code; and the Flow Studio, the drag-and-drop editor shipped with gemini-adk-web-rs that reads, writes, validates, tests, and live-runs these documents.

Loading a flow from JSON

Flow derives Serialize/Deserialize, so this is plain serde:

let flow: Flow = serde_json::from_str(&std::fs::read_to_string("flow.json")?)?;
let compiled = flow.compile_with_tools(&["verify_identity", "charge_card"])?;

let handle = Live::builder()
    .dispatcher(dispatcher)
    .govern_compiled(compiled)
    .connect_from_env()
    .await?;

The only thing that does not serialize is Guard::custom(..) — the code-only escape hatch. Serializing a flow containing one is an error, never a silent drop.

The flow JSON format

A flow document has steps, and optionally constraints, ambient, and confirm_tools:

{
  "steps": [
    {
      "id": "verify",
      "posture": "Verify the caller's identity before anything else.",
      "allow": ["verify_identity"],
      "done": { "is_true": "identity_verified" }
    },
    {
      "id": "take_payment",
      "after": ["verify"],
      "allow": ["charge_card"],
      "done": { "called_ok": "charge_card" }
    },
    { "id": "close", "after": ["take_payment"], "terminal": true }
  ],
  "constraints": [
    { "once": "charge_card" },
    { "never_until": { "tool": "charge_card", "until": { "is_true": "ptp_confirmed" } } },
    { "require": ["close"] }
  ],
  "ambient": ["recall_context"]
}

Step fields

Field	Type	Meaning
`id`	string	Unique step id (required)
`after`	[string]	Dependency step ids — the DAG edges
`gate`	Guard	Extra eligibility beyond dependencies
`done`	Guard	Completion condition (required unless `terminal`)
`posture`	string	Instruction imposed while active
`ground`	string	State-interpolated fact template (`{key}` / `{key?yes:no}`)
`allow`	[string]	Tool whitelist while active (empty = no restriction)
`deny`	[string]	Tools forbidden while active
`terminal`	bool	Completes on eligibility, needs no `done`

Guard atoms

Guards are the serde encoding of the Pred enum (externally tagged, snake_case):

JSON	Rust constructor
`"always"`	`Guard::always()`
`{ "is_true": "key" }`	`Guard::is_true("key")`
`{ "is_set": "key" }`	`Guard::is_set("key")`
`{ "eq": ["key", value] }`	`Guard::eq("key", value)`
`{ "captured": ["a", "b"] }`	`Guard::captured(["a", "b"])`
`{ "called_ok": "tool" }`	`Guard::called_ok("tool")`
`{ "done": "step" }`	`Guard::done("step")`
`{ "all": [g, …] }`	`Guard::all([...])`
`{ "any": [g, …] }`	`Guard::any([...])`
`{ "not": g }`	`Guard::not(g)`

Graph edges: branching, merging, and loops

after entries are plain strings (unconditional) or conditional edges — objects whose guard must hold for the edge to be satisfied. Conditional edges out of one source are how a flow branches; "join": "any" on the merge step accepts whichever branch completes. The canvas renders conditional edges dashed with their guard as the label:

{ "id": "schedule",
  "after": [ { "step": "triage", "when": { "not": { "is_true": "is_emergency" } } } ] },
{ "id": "close",
  "after": [ "schedule", { "step": "triage", "when": { "is_true": "is_emergency" } } ],
  "join": "any", "terminal": true }

Loops are not back-edges — the DAG stays acyclic and statically checkable. Iteration is explicit marking surgery: a reset constraint un-latches steps on its guard’s rising edge, re-arms their on_enter, and forgives the called_ok evidence their completion guards reference (so a once on such a tool counts per latch-cycle; state keys stay the application’s to clear):

{ "reset": { "steps": ["diagnose"], "when": { "is_true": "retest_requested" } } }

Constraints

JSON	Meaning
`{ "once": "tool" }`	The tool may complete at most once
`{ "before": ["a", "b"] }`	Step `a` must be done before `b` starts
`{ "never_until": { "tool": t, "until": g } }`	Forbid `t` until `g` holds
`{ "require": ["a", "b"] }`	Steps required for flow completion
`{ "reset": { "steps": [...], "when": g } }`	Un-latch steps on `g`’s rising edge (loops)

`SessionSpec` — a runnable application as one JSON file

A session spec (gemini_adk_fluent_rs::spec::SessionSpec; the historical FlowAppSpec and MockToolSpec names in gemini-adk-server-rs are deprecated aliases and parse the same documents) wraps a flow with everything a governed Live session needs:

{
  "name": "collections",
  "instruction": "You are a debt-collection assistant.",
  "greeting": "Greet the caller and ask for their name.",
  "modality": "text",
  "tools": [
    {
      "name": "verify_identity",
      "description": "Verify the caller's identity.",
      "parameters": { "type": "object", "properties": { "name": { "type": "string" } } },
      "response": { "verified": true },
      "set_state": { "identity_verified": true }
    }
  ],
  "extract": [ … ],
  "state": { … }, "computed": [ … ],
  "memory": { … },
  "phases": [ … ],
  "watch": [ … ], "patterns": [ … ],
  "runtime": { … },
  "fragments": { … }, "use_fragments": [ … ],
  "tests": [ … ],
  "flow": { "steps": [ … ], "constraints": [ … ] }
}

In Rust, SessionSpec::apply(live, &state, &resources) configures a Live builder from the document; code-only concerns (callbacks, custom guards, middleware) are added on the returned builder afterwards. GET /api/flows/schema serves the document’s own JSON Schema — for editor autocomplete, and for validating machine-authored specs at generation time.

A conversation instead of a flow

A spec can carry a conversation, written in the conversation compiler’s vocabulary, instead of a flow. The conversation vocabulary covers stages, collected slots, commits, digressions, per-stage timing, verbatim readouts, repair and policies. The spec’s scenarios are run against it:

{
  "name": "booking",
  "modality": "audio",
  "tools": [{ "name": "book", "description": "Book the table", "response": { "confirmation": "B-1" } }],
  "extract": [
    { "name": "booking", "instruction": "Extract the party size and the requested time.",
      "schema": { "type": "object", "properties": {
        "party_size": { "type": "integer" }, "slot": { "type": "string" } } },
      "promote": [{ "field": "party_size", "policy": "overwrite" }, { "field": "slot", "policy": "overwrite" }] },
    { "name": "caller_signals", "instruction": "Set a field to true only when the caller clearly said so.",
      "schema": { "type": "object", "properties": {
        "user_confirmed": { "type": "boolean", "description": "The caller agreed to the booking read back to them." } } },
      "promote": [{ "field": "user_confirmed", "policy": "true_only" }] }
  ],
  "conversation": {
    "name": "booking",
    "stages": [
      { "id": "collect", "say": "Help the user book a table.", "collect": ["party_size", "slot"],
        "next": [{ "to": "confirm", "when": { "captured": ["party_size", "slot"] } }] },
      { "id": "confirm", "say": "Read the booking back and book it once the caller agrees.", "allow": ["book"],
        "commit": { "tool": "book", "when": { "is_true": "user_confirmed" } },
        "next": [{ "to": "done", "when": { "called_ok": "book" } }] },
      { "id": "done", "terminal": true }
    ],
    "require": ["done"]
  },
  "scenarios": [
    { "name": "cannot_book_without_confirmation",
      "steps": [{ "expect_denied": "book" }] }
  ]
}

A spec sets conversation or flow, not both.
collect names the slots a stage needs; it does not fill them. Here the booking extractor fills party_size and slot from what the caller says, and caller_signals sets user_confirmed when the caller agrees. Without them the call never leaves collect and book is never admitted.
validate compiles the conversation. Its compiled flow goes through the same checks as a hand-written one: unknown tools, and guards reading keys nothing writes. Collected slots count as written, so a missing extractor for them is not reported.
A stage’s resolve slot is filled by the declared tool of the same name (or the one named in resolver). A resolver with no declared tool is an error.
SessionSpec::run_scenarios runs every scenario through the simulator with resolvers stubbed. The Studio’s POST /api/flows/test returns the results under scenarios.
apply compiles the conversation and calls Live::converse, which installs its governed stack, extractors, timing and policies.

Tools: mock, HTTP, MCP

A declared tool without a binding is a mock: it returns its canned response (default {"ok": true}) and writes set_state into the session State. Because flow guards read the same state, is_true/captured conditions latch exactly as against real implementations — model, enforce, and demo the whole conversation before any real tool exists.

Add an "http" binding and the same tool performs a request instead (http-tools feature; {args.field} / {state.key} interpolate into the URL, headers, and body; save_response_as stores the response under a state key; set_state still applies, so guards latch identically):

{ "name": "check_availability",
  "http": { "method": "GET", "url": "https://api.example.com/slots?guests={args.party_size}" },
  "save_response_as": "availability", "set_state": { "availability_checked": true } }

MCP toolsets plug in the whole ecosystem with one line, resolved at connect time:

"mcp": ["http://localhost:3000/mcp"]

To implement one declared tool on an MCP server, give the tool an "mcp" binding: a stdio command or an http(s) URL. The runtime calls the tool of the same name on that server. The spec’s declaration is what the model sees, and set_state and save_response_as still apply. A tool can have an http binding or an mcp binding, not both.

{ "name": "book_table", "mcp": "python -m tools",
  "set_state": { "booked": true } }

In Rust, implement a declared tool in process instead. The implementation replaces the tool’s call; the declaration and state effects stay the spec’s. apply refuses an implementation for a tool the spec does not declare.

let resources = SpecResources::default().implement(SimpleTool::new(
    "book_table", "", None,
    |args| async move { Ok(bookings::create(args).await?) },
));
let live = spec.apply(Live::builder(), &state, &resources)?;

When a tool has more than one of these, the in-process implementation wins, then mcp, then http, then the mock response.

Extraction: the flow advances from speech alone

extract entries run an out-of-band model against the transcript, fill a JSON schema, and promote fields into the bare state keys guards read — so a captured(["ptp_amount","ptp_date"]) guard latches with no tool call anywhere:

"extract": [{
  "name": "promise_to_pay",
  "instruction": "Extract the payment amount and date the caller agrees to.",
  "schema": { "type": "object", "properties": {
    "ptp_amount": { "type": "number" }, "ptp_date": { "type": "string" } } },
  "trigger": "every_turn",
  "promote": [ { "field": "ptp_amount" },
               { "field": "ptp_date", "policy": "overwrite" } ]
}]

Promotion policies: keep_known (default), overwrite, true_only, non_empty; "to" renames the target key. (Live::extract_json is the code-level equivalent.) Running extraction needs a model: SpecResources.extraction_llm.

State dictionary and computed variables

The state section is the session’s data dictionary: declare each key’s type, meaning, and starting value. Declaring it is optional, but once present it powers autocomplete in the Studio’s guard editors, undeclared-key warnings at validation, typed StateKey constants in generated code, and default seeding at connect:

"state": {
  "party_size":          { "type": "number",  "description": "Guests in the party." },
  "availability_checked": { "type": "boolean", "default": false }
}

computed entries are derived variables as data — the counterpart to Guard for values. A closed expression vocabulary (key, const, add, mul, sub, div, min, max, eq, gt, gte, lt, lte, all, any, not, if, coalesce, concat, count_true) evaluates over state; the result is written to derived:{key}, dependencies are inferred from the expression, dependency cycles are load-time errors, and guards read the result by its bare key (the derived: fallback):

"computed": [{
  "key": "large_party",
  "from": { "gte": [ { "key": "party_size" }, { "const": 6 } ] }
}]

A step can then done/gate on {"is_true": "large_party"}, a watcher can watch it, and the offline simulator recomputes it after every event — the same semantics in the live runtime, the simulator, and generated code.

Durable memory

The memory section installs the memory subsystem (gemini-memory-rs) declaratively: the ambient recall_context / manage_memory tools, turn ingestion and end-of-session reconciliation, and slots that project remembered facts into governed state keys — where needs, captured, and every guard read them exactly as if the caller had just said it, so a returning caller is never asked twice:

"memory": {
  "slots": [
    { "predicate": "dietary_identity", "to": "user:diet" },
    { "predicate": "seating_preference", "to": "user:seating" }
  ]
}

At apply time the engine arrives through SpecResources.memory — any MemoryBinding implementation; gemini_memory_rs::runtime::SessionMemoryBinding wraps a MemorySession. The Studio provisions an in-process engine per run automatically. The remember effect (below) writes through the same binding.

Phases, watchers, and patterns over the same guard vocabulary

Phase transitions, watcher conditions, and pattern conditions reuse the closed Guard atoms; handlers are closed EffectSpecs — one vocabulary honored identically wherever effects fire:

Effect	Meaning
`{"set": {…}}`	write state keys
`{"context": "…"}`	inject a model-role steering turn (read, not answered)
`{"prompt": "…"}`	inject text and ask the model to respond now — the “make the model speak” effect
`{"remember": "…"}`	durably remember a note (`{state.key}` interpolates; needs the `memory` section)

"phases": [
  { "name": "greeting", "instruction": "Welcome the caller.",
    "transitions": [ { "to": "main", "when": { "is_true": "greeted" } } ],
    "on_enter": [ { "set": { "entered": true } } ] },
  { "name": "main", "tools": ["search"], "needs": ["topic"], "terminal": false }
],
"initial_phase": "greeting",
"watch": [
  { "key": "large_party", "condition": "became_true",
    "effects": [ { "context": "Mention the set-menu policy." },
                 { "remember": "Books for large parties ({state.party_size} guests)" } ] }
]

Watchers receive the live session writer, so the full effect vocabulary applies — a watcher can steer, prompt, or remember, not only mutate state. (set remains as sugar for a leading {"set": …} effect.)

Phase guards evaluate against state alone (there is no flow marking at a phase boundary), so called_ok/done atoms there are a validation error — latch a state key instead.

Temporal patterns fire when a condition holds continuously — for wall seconds or consecutive turns — the “sounded confused for 30 seconds” reactor, as data:

"patterns": [{
  "name": "stuck", "when": { "is_true": "repeating" }, "turns": 3,
  "effects": [ { "set": { "needs_help": true } },
               { "context": "The caller seems stuck — offer to summarize the options." } ]
}]

Runtime tuning: the control plane as data

Everything that is configuration rather than conversation lives in one runtime section; every field lowers to a Live builder setter, and omitted fields keep the builder’s defaults:

"runtime": {
  "temperature": 0.6,
  "thinking_budget": 1024, "include_thoughts": true,
  "transcription": { "input": true, "output": true },
  "proactive_audio": true,
  "vad": { "start_sensitivity": "high", "end_sensitivity": "low",
           "silence_duration_ms": 400 },
  "soft_turn_timeout_ms": 1500,
  "steering": "context_injection",
  "context_delivery": "deferred",
  "repair": { "nudge_after": 2, "escalate_after": 5 },
  "persistence": { "fs": { "dir": "/var/sessions" } },
  "session_id": "user-123"
}

Per-tool async calling rides on the tool declaration itself: "background": true marks a tool non-blocking (the model keeps speaking while it runs), and "scheduling": "interrupt" | "when_idle" | "silent" picks how the async response lands (scheduling implies background).

Fragments: reusable flow modules

Declare a flow fragment once, splice it anywhere under a namespace — step ids become {namespace}/{id} and internal after/done(step)/constraint references are rewritten:

"fragments": { "verify": { "steps": [ … ] } },
"use_fragments": [ { "fragment": "verify", "namespace": "v", "after": ["greet"] } ],
"flow": { "steps": [ { "id": "book", "after": ["v/confirm"], … } ] }

This is how a compliance team owns disclosure once and every flow imports it.

Embedded tests: conformance without an API key

tests script conversations as data and assert flow state at checkpoints. The script replays through the runtime’s FlowStack with the declared tools’ mock semantics, including a conversation’s digressions and repair policies (POST /api/flows/test, the Studio’s Tests button, or spec.run_tests()):

"tests": [{ "name": "premature charge is blocked", "script": [
  { "tool": "charge_card" },
  { "expect": { "blocked": ["charge_card"], "complete": false } }
]}]

Events: {"user": "…"} (turn boundary), {"tool": "name"} (mock semantics + completion, or a failure if the flow blocks it unexpectedly), {"set": {…}} (stands in for extraction), {"expect": {done, active, allowed, blocked, state, complete}}.

set re-latches the active layer without counting a user turn. Step through uses the same replay and returns L1 FlowSnapshot fields: active-layer progress, overlay_path, whole-stack complete, and terminated. A digression’s closing turn can have no missing requirements while the stack remains incomplete. These tests supply state explicitly; they do not run model extraction, phase callbacks, audio, or external tool implementations.

SessionSpec::validate_for_replay() checks the original document before replay. It retains structural errors such as conflicting HTTP/MCP bindings but does not require the http-tools feature for mock execution.

Validation: everything that can fail, fails at load time

SessionSpec::validate runs the flow compiler with the declared tools as the registry, checks fragments splice, rejects marking atoms in phase guards — and diffs the state keys guards read against the keys the session writes (tool set_state, extraction promotions, phase/watcher effects). A guard waiting on a key nothing sets is the dominant silent failure in data-authored flows; it now surfaces as a warning with a did-you-mean suggestion:

a guard reads state key 'identity_verifed' but no tool, extractor, phase, or
watcher writes it (it can never latch) — did you mean identity_verified?

The Flow Studio

cargo run -p gemini-adk-web-rs, then open http://localhost:25125/flows.

The Studio is a drag-and-drop editor over exactly this document:

Canvas — steps are nodes; drag from a node’s output port onto another node to add an after edge; click an edge to remove it. Auto-layout arranges the DAG by topological depth.
Step inspector — posture, ground template, allow/deny lists, terminal flag, and a structured guard builder covering every atom (including nested all/any/not).
Flow tab — cross-cutting constraints (once, before, never…until, require), ambient tools, confirm tools.
App tab — instruction, greeting, modality, and the mock tool editor.
JSON tab — the live document. Two-way: edit and Apply, import a file, copy, or download. The JSON you export is exactly what serde_json::from_value::<SessionSpec> (or a bare Flow) loads.
Validate — POST /api/flows/validate runs the real compiler (Flow::compile_with_tools) server-side and reports every diagnostic: unknown tools, unreachable steps, unguarded commit tools, ordering cycles, unwritten guard keys (with did-you-mean suggestions), plus advisory warnings.
Tests — replays the spec’s embedded test suite offline through the real flow stack (POST /api/flows/test) and reports each script’s result. No API key involved.
Run — starts a live session in the flow-studio app (/ws/flow-studio), passing the spec in the Start message’s config field. The session is configured by SessionSpec::apply (governance, tools, extraction, phases, watchers), and after every turn, tool call, and extraction the server pushes a flowStatus snapshot — active steps light up blue and done steps green on your canvas while you chat, and the Run tab lists admitted tools, blocked tools (with reasons, from flow_snapshot()), unmet requirements, and each active step’s guard truth tree: exactly which atom it is waiting on.
Live posture editing — while a session runs, consecutive posture or ground edits in the step inspector send updateFlowPostures; the monitor re-projects postures at every turn boundary, so the change steers the very next turn. A structural edit, load, undo, or redo stops these patches until the session restarts, so edits cannot reach an unrelated running document.

The Studio ships seven skill catalogs and six reference flows. See the Studio guide for its current layout, gallery, replay, and project export workflows.

Validate endpoint

POST /api/flows/validate
Body: a SessionSpec, or a bare flow ({"steps": [...]})

Response:

{
  "valid": true,
  "errors": [],
  "warnings": ["step 'x' has no posture — …"],
  "mermaid": "flowchart TD\n …",
  "tools": ["charge_card", "verify_identity"],
  "steps": 5
}