WebSocket Chat

Integrate real-time WebSocket chat for streaming AI agent responses with live tool execution and event updates.

Overview

The WebSocket chat API provides streaming communication with AI agents and teams, delivering real-time token-by-token responses, tool executions, and system events.

Features:

Streaming responses - Token-by-token output as the agent thinks
Real-time tool execution - See tools being called live
Multiple authentication methods - JWT, API keys, or password protection
Session persistence - Resume conversations across connections
MCP connection caching - Persistent connections reduce latency

Connection Setup

WebSocket Endpoints

Team Chat:

ws://localhost:8000/ws/team/{team_identifier}

Agent Chat:

ws://localhost:8000/ws/agent/{agent_identifier}

Team Chat (Specific Agent):

ws://localhost:8000/ws/team/{team_identifier}/agent/{agent_identifier}

Connection Example (JavaScript)

javascript

// Connect to agent
const ws = new WebSocket('ws://localhost:8000/ws/agent/my-agent?token=YOUR_JWT');

// Or with API key
const ws = new WebSocket('ws://localhost:8000/ws/agent/my-agent?api_key=YOUR_API_KEY');

// Or password-protected agent
const ws = new WebSocket('ws://localhost:8000/ws/agent/my-agent?password=AGENT_PASSWORD');

ws.onopen = () => {
  console.log('Connected to agent');
};

ws.onmessage = (event) => {
  const data = JSON.parse(event.data);
  console.log('Event:', data);
};

ws.onerror = (error) => {
  console.error('WebSocket error:', error);
};

ws.onclose = () => {
  console.log('Connection closed');
};

Authentication

Three authentication methods are supported:

1. JWT Token

Query Parameter:

ws://localhost:8000/ws/agent/my-agent?token=YOUR_JWT_TOKEN

Header:

javascript

const ws = new WebSocket('ws://localhost:8000/ws/agent/my-agent', {
  headers: {
    'Authorization': 'Bearer YOUR_JWT_TOKEN'
  }
});

2. API Key

Query Parameter:

ws://localhost:8000/ws/agent/my-agent?api_key=YOUR_API_KEY

Header:

javascript

const ws = new WebSocket('ws://localhost:8000/ws/agent/my-agent', {
  headers: {
    'X-AUTH-TOKEN': 'YOUR_API_KEY'
  }
});

3. Password-Protected

Query Parameter:

ws://localhost:8000/ws/agent/my-agent?password=AGENT_PASSWORD

Header:

javascript

const ws = new WebSocket('ws://localhost:8000/ws/agent/my-agent', {
  headers: {
    'X-Access-Password': 'AGENT_PASSWORD'
  }
});

Public Agents: No authentication required for agents/teams with access_level=PUBLIC.

Sending Messages

Send JSON messages to the WebSocket:

javascript

ws.send(JSON.stringify({
  message: "What's the weather like today?",
  session_id: "session-123"  // Optional: resume existing session
}));

Message Format:

json

{
  "message": "Your message text",
  "session_id": "optional-session-id",
  "metadata": {}  // Optional metadata
}

Without session_id: A new session is created automatically. With session_id: Conversation continues in existing session.

Sending an Attachment

Add an attachments field carrying the asset_id returned by POST /assets:

javascript

ws.send(JSON.stringify({
  message: "What does this insurance policy cover?",
  session_id: "session-123",
  attachments: ["b3f1c2a4-9e21-4a3c-8f10-7d4e6a1b9c02"]
}));

Rules:

session_id is required when attachments is present — the server re-checks that the attachment is bound to this exact session before resolving it (ownership proof, not just the UUID).
v1 supports one attachment per message. A malformed, unowned, or missing asset id rejects the turn with an error event (the turn does not execute) rather than a WebSocket close.
Text-only messages (no attachments) are completely unaffected — same event stream as before.

Persisted shape — gnosari_asset_id marker. Once a message carries an attachment, session_messages.message_data stores a structured content array instead of a plain string, with a stable gnosari_asset_id marker in place of any URL:

jsonc

// image part, as persisted — url is resolved fresh on every replay, never stored
{ "type": "input_image", "gnosari_asset_id": "b3f1c2a4-...", "image_url": null }
// PDF part, as persisted
{ "type": "input_file", "gnosari_asset_id": "b3f1c2a4-...", "file_url": null, "filename": "insurance-policy.pdf" }

The signed URL is never persisted — it is minted fresh (via GET /assets/{id}/url or the engine's injected signer) for every model call and every history render. Any code that reads message_data (session serialization, transcript search, entity extraction) must treat the gnosari_asset_id marker as opaque and never let it leak into transcript search results — it is an internal reference, not user-facing content.

Event Types

The API sends streaming events as the agent processes your message:

`agent_started`

Agent begins processing:

json

{
  "type": "agent_started",
  "agent_name": "Weather Assistant",
  "session_id": "session-abc123"
}

`token`

Token-by-token response:

json

{
  "type": "token",
  "content": "The ",
  "agent_name": "Weather Assistant"
}

`tool_call`

Tool execution started:

json

{
  "type": "tool_call",
  "tool_name": "get_weather",
  "args": {
    "location": "San Francisco"
  },
  "agent_name": "Weather Assistant"
}

`tool_result`

Tool execution completed:

json

{
  "type": "tool_result",
  "tool_name": "get_weather",
  "result": {
    "temperature": "72°F",
    "condition": "Sunny"
  },
  "agent_name": "Weather Assistant"
}

`agent_completed`

Agent finished processing:

json

{
  "type": "agent_completed",
  "agent_name": "Weather Assistant",
  "full_response": "The weather in San Francisco is sunny with a temperature of 72°F."
}

`error`

Error occurred:

json

{
  "type": "error",
  "error": "Error description",
  "agent_name": "Weather Assistant"
}

`session_summary_scheduled`

Session summary generation scheduled (sent after agent_completed):

json

{
  "type": "session_summary_scheduled",
  "session_id": "session-abc123",
  "scheduled_at": "2025-02-15T10:30:00Z"
}

Receiving Events

Process events as they arrive:

javascript

ws.onmessage = (event) => {
  const data = JSON.parse(event.data);

  switch (data.type) {
    case 'agent_started':
      console.log('Agent started:', data.agent_name);
      break;

    case 'token':
      // Append token to response area
      appendToResponse(data.content);
      break;

    case 'tool_call':
      console.log('Calling tool:', data.tool_name, data.args);
      showToolExecution(data.tool_name, data.args);
      break;

    case 'tool_result':
      console.log('Tool result:', data.result);
      break;

    case 'agent_completed':
      console.log('Complete response:', data.full_response);
      markConversationComplete();
      break;

    case 'error':
      console.error('Agent error:', data.error);
      showError(data.error);
      break;

    case 'session_summary_scheduled':
      console.log('Summary scheduled for session:', data.session_id);
      break;
  }
};

Session Management

Starting a New Session

Send a message without session_id:

javascript

ws.send(JSON.stringify({
  message: "Hello!"
}));

The agent_started event will include the new session_id.

Resuming a Session

Include the session_id from a previous conversation:

javascript

ws.send(JSON.stringify({
  message: "What did we talk about earlier?",
  session_id: "session-abc123"
}));

Session Storage

Sessions are stored in the database and include:

Full conversation history
Metadata (agent, team, user)
Summary (generated after inactivity)
Extracted entities (if configured)

Error Handling

Connection Errors

javascript

ws.onerror = (error) => {
  console.error('WebSocket error:', error);
};

ws.onclose = (event) => {
  if (event.code !== 1000) {  // 1000 = normal closure
    console.error('Connection closed unexpectedly:', event.code, event.reason);
    // Attempt reconnection
    setTimeout(reconnect, 1000);
  }
};

Message Errors

javascript

ws.onmessage = (event) => {
  const data = JSON.parse(event.data);

  if (data.type === 'error') {
    handleError(data.error);
  }
};

Reconnection Strategy

javascript

let reconnectAttempts = 0;
const maxReconnectAttempts = 5;

function reconnect() {
  if (reconnectAttempts < maxReconnectAttempts) {
    reconnectAttempts++;
    const delay = Math.min(1000 * Math.pow(2, reconnectAttempts), 30000);
    console.log(`Reconnecting in ${delay}ms (attempt ${reconnectAttempts})`);
    setTimeout(connect, delay);
  }
}

function connect() {
  ws = new WebSocket(wsUrl);
  // ... set up handlers
  ws.onopen = () => {
    reconnectAttempts = 0;  // Reset on successful connection
  };
}

MCP Connection Caching

For improved performance, the API caches MCP (Model Context Protocol) connections per WebSocket session:

First message: Initializes runner and MCP manager (slight delay)
Subsequent messages: Reuses cached connections (much faster)
Connection lifetime: Persists until WebSocket disconnects

Performance Impact:

First message: ~1-2s latency
Subsequent messages: <100ms latency

Complete Client Example

javascript

class GnosariChat {
  constructor(agentIdentifier, authToken) {
    this.agentIdentifier = agentIdentifier;
    this.authToken = authToken;
    this.ws = null;
    this.sessionId = null;
  }

  connect() {
    const url = `ws://localhost:8000/ws/agent/${this.agentIdentifier}?token=${this.authToken}`;
    this.ws = new WebSocket(url);

    this.ws.onopen = () => {
      console.log('Connected');
      this.onConnected();
    };

    this.ws.onmessage = (event) => {
      const data = JSON.parse(event.data);
      this.handleEvent(data);
    };

    this.ws.onerror = (error) => {
      console.error('Error:', error);
      this.onError(error);
    };

    this.ws.onclose = () => {
      console.log('Disconnected');
      this.onDisconnected();
    };
  }

  sendMessage(message) {
    const payload = {
      message: message
    };

    if (this.sessionId) {
      payload.session_id = this.sessionId;
    }

    this.ws.send(JSON.stringify(payload));
  }

  handleEvent(data) {
    switch (data.type) {
      case 'agent_started':
        this.sessionId = data.session_id;
        this.onAgentStarted(data);
        break;
      case 'token':
        this.onToken(data.content);
        break;
      case 'tool_call':
        this.onToolCall(data.tool_name, data.args);
        break;
      case 'agent_completed':
        this.onComplete(data.full_response);
        break;
      case 'error':
        this.onError(data.error);
        break;
    }
  }

  // Override these methods
  onConnected() {}
  onAgentStarted(data) {}
  onToken(content) {}
  onToolCall(name, args) {}
  onComplete(response) {}
  onError(error) {}
  onDisconnected() {}

  disconnect() {
    if (this.ws) {
      this.ws.close();
    }
  }
}

// Usage
const chat = new GnosariChat('my-agent', 'jwt-token-here');

chat.onConnected = () => {
  console.log('Ready to chat');
};

chat.onToken = (content) => {
  document.getElementById('response').textContent += content;
};

chat.onComplete = (response) => {
  console.log('Full response:', response);
};

chat.connect();

// Send message
document.getElementById('send-btn').onclick = () => {
  const message = document.getElementById('message-input').value;
  chat.sendMessage(message);
};

Cross-References

Sessions API Reference - Session management endpoints
Agent Access Control - Access level configuration
WebSocket Implementation - Internal WebSocket handler

WebSocket Chat ​

Overview ​

Connection Setup ​

WebSocket Endpoints ​

Connection Example (JavaScript) ​

Authentication ​

1. JWT Token ​

2. API Key ​

3. Password-Protected ​

Sending Messages ​

Sending an Attachment ​

Event Types ​

agent_started ​

token ​

tool_call ​

tool_result ​

agent_completed ​

error ​

session_summary_scheduled ​

Receiving Events ​

Session Management ​

Starting a New Session ​

Resuming a Session ​

Session Storage ​

Error Handling ​

Connection Errors ​

Message Errors ​

Reconnection Strategy ​

MCP Connection Caching ​

Complete Client Example ​

Cross-References ​

WebSocket Chat

Overview

Connection Setup

WebSocket Endpoints

Connection Example (JavaScript)

Authentication

1. JWT Token

2. API Key

3. Password-Protected

Sending Messages

Sending an Attachment

Event Types

`agent_started`

`token`

`tool_call`

`tool_result`

`agent_completed`

`error`

`session_summary_scheduled`

Receiving Events

Session Management

Starting a New Session

Resuming a Session

Session Storage

Error Handling

Connection Errors

Message Errors

Reconnection Strategy

MCP Connection Caching

Complete Client Example

Cross-References