Add a tool to the agent's tool registry, exposed to the model during the session. Same AiTool objects as AiAgent. Returns this agent for chaining.
Tool to add to the registry.
Remove a tool from the registry by name. No-op when no tool with that name is registered. Returns this agent for chaining.
Name of the tool to remove.
Stream audio to the model over a live realtime session and return the response. The audio source is read once and streamed straight to the provider (no whole-clip buffering); the model transcribes it, may call the registered tools (executed in-script, results fed back automatically), and returns its final text. Audio must already be PCM16 @ 24kHz mono.
Accepts a Java InputStream (e.g. B.net.request.inputStream() for a live mic upload, or a
document's content stream) or a byte[] of PCM16 audio. Returns an AiCallResult with the
assistant text, stop reason, token usage, and the tool calls that were executed.
PCM16 @ 24kHz mono audio source — an InputStream (read once) or a byte[].
Like sendAudio(audio), but also persists the audio to the given Relate document field
as a WAV file. The audio is captured to a temporary file, streamed to the model, and —
when non-empty — written to the field wrapped in a WAV header (PCM16 @ 24kHz mono), so the
saved recording is directly playable. Returns the same AiCallResult as sendAudio(audio).
PCM16 @ 24kHz mono audio source.
A Relate document field (e.g. record.field('recording')) to save the WAV into.
Set the usage gate tracking flag. Returns this agent for chaining.
Custom tracking flag reported to the usage gate.
Set the realtime model identifier. Falls back to the provider's configured realtime model when unset. Returns this agent for chaining.
Realtime model identifier (e.g. "gpt-realtime").
Set the hook called with each piece of assistant text as the session streams it. See AiClientOptions.onDelta. Returns this agent for chaining.
The hook, or null to clear.
Set the hook called after each model response of the session, once its usage is counted, which may stop the turn. See AiClientOptions.onIteration. Unlike the chat path, the tool calls of that response have already run when the hook is asked — the provider streams them ahead of the response completing — so a stop keeps their results from the model and ends the turn rather than preventing them. Pass null to clear. Returns this agent for chaining.
The hook, or null to clear.
Set the hook called for each tool call before it runs, which may run it (with other arguments) or deny it. See AiClientOptions.onToolCall. {action: "defer"} is refused on this transport: a live audio session has no resume to answer a deferred call with. Pass null to clear. Returns this agent for chaining.
The hook, or null to clear.
Set the hook called with what each tool returned before the model sees it. See AiClientOptions.onToolResult. Pass null to clear. Returns this agent for chaining.
The hook, or null to clear.
Set the AI provider. Realtime audio is OpenAI-only, so this accepts only "openai" (or unset). Returns this agent for chaining.
Provider name; must be "openai".
Set the session instructions (system prompt). Returns this agent for chaining.
Session instructions / system prompt.
Reusable streaming AI agent for realtime audio (OpenAI only). Shares the connection setters, tool registry and per-iteration hooks (
onToolCall,onToolResult,onIteration,onDelta) of AiAgent, butsendAudio(...)streams audio to the model over a live session and returns the response (text + executed tool calls +trace) as an AiCallResult. One-shot per call: no chatHistory or onTurn, and noonRequest— there is no message list to show. A tool call anonToolCallhook defers is refused, because a live session has noresume. Audio must be PCM16 @ 24kHz mono. Returned byB.ai.streamingAgent().Example: ```javascript B.net.request.binaryStream((audioIn) => { // PCM16 @ 24kHz mono const result = B.ai.streamingAgent() .setSystemPrompt("You are a voice assistant. Use tools to answer.") .addTool(lookupResident) .setOnToolCall(call => call.annotations && call.annotations.destructive ? { action: "deny", reason: "not over the phone" } // "defer" is refused here : null) .sendAudio(audioIn); B.net.response.contentType("application/json"); B.net.response.stream(out => out.print(JSON.stringify({ text: result.text }))); }); ```