Data Collection

Learn how to configure AI agents to automatically extract structured data (contacts, leads, appointments, etc.) from conversations.

Overview

The Entity Extraction System automatically extracts structured data from AI conversations. When users interact with agents, the system identifies and captures relevant entities based on entity types configured on each agent.

Use Cases:

Collect contact information from chat conversations
Extract lead data for CRM integration
Capture appointment details automatically
Build structured datasets from unstructured conversations

Key Features:

Automatic extraction after every conversation
Configurable schemas via entity types
Confidence scoring to filter low-quality extractions
Duplicate detection to prevent redundant records
Manual review with status workflow (NEW → REVIEWED → ACTIONED → ARCHIVED)

Quick Start

1. Define an Entity Type

Entity types are reusable templates that define what data to extract:

bash

curl -X POST "http://localhost:8000/api/v1/entity-types" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Contact",
    "description": "Person contact information",
    "icon": "user",
    "attributes": [
      {
        "name": "full_name",
        "type": "string",
        "required": true,
        "ai_hint": "The complete name of the person"
      },
      {
        "name": "email",
        "type": "email",
        "required": true,
        "ai_hint": "Email address"
      },
      {
        "name": "company",
        "type": "string",
        "required": false,
        "ai_hint": "Company or organization name"
      },
      {
        "name": "phone",
        "type": "string",
        "required": false,
        "ai_hint": "Phone number in any format"
      },
      {
        "name": "interests",
        "type": "array",
        "items_type": "string",
        "required": false,
        "ai_hint": "List of interests or topics mentioned"
      }
    ]
  }'

2. Assign Entity Type to Agent

bash

curl -X POST "http://localhost:8000/api/v1/agents/{agent_id}/entity-types/{entity_type_id}" \
  -H "Authorization: Bearer $TOKEN"

3. Extraction Happens Automatically

When a conversation session ends (after inactivity timeout), the system:

Generates a session summary
Extracts entities based on configured entity types
Creates ExtractedEntity records with confidence scores

4. Review Extracted Entities

bash

# List all extracted entities
curl "http://localhost:8000/api/v1/extracted-entities" \
  -H "Authorization: Bearer $TOKEN"

# Filter by agent
curl "http://localhost:8000/api/v1/extracted-entities?agent_id=1" \
  -H "Authorization: Bearer $TOKEN"

# Filter by status
curl "http://localhost:8000/api/v1/extracted-entities?status=NEW" \
  -H "Authorization: Bearer $TOKEN"

Entity Type Attributes

Supported Attribute Types

Type	JSON Schema	Description
`string`	`{"type": "string"}`	Text field
`number`	`{"type": "number"}`	Decimal number
`integer`	`{"type": "integer"}`	Whole number
`boolean`	`{"type": "boolean"}`	True/false
`array`	`{"type": "array"}`	List of items
`object`	`{"type": "object"}`	Nested object
`date`	`{"type": "string", "format": "date"}`	Date (YYYY-MM-DD)
`datetime`	`{"type": "string", "format": "date-time"}`	ISO 8601 datetime
`email`	`{"type": "string", "format": "email"}`	Email address
`url`	`{"type": "string", "format": "uri"}`	URL

Attribute Definition Format

json

{
  "name": "field_name",
  "type": "string",
  "required": true,
  "ai_hint": "Clear description for the LLM"
}

Fields:

name (required): Field identifier (use snake_case)
type (required): Data type from supported types
required (required): Marks the field as essential — the model prioritizes it and duplicate detection compares on it. It is NOT a generation constraint: a field the visitor never stated is omitted, never forced to an empty/0 placeholder
ai_hint (recommended): Description to guide the LLM

Array Attributes

json

{
  "name": "interests",
  "type": "array",
  "items_type": "string",
  "required": false,
  "ai_hint": "List of topics the person mentioned interest in"
}

Nested Object Attributes

json

{
  "name": "address",
  "type": "object",
  "properties": [
    {"name": "street", "type": "string", "required": false},
    {"name": "city", "type": "string", "required": false},
    {"name": "postal_code", "type": "string", "required": false}
  ],
  "required": false,
  "ai_hint": "Mailing address details"
}

Entity Extraction Flow

Visitor-Only Attribution

Collected data is what the visitor said — never what the agent said about itself. Before anything is stored, the extraction pipeline applies an attribution gate:

Every attribute's provenance (cited transcript message + verbatim quote) is verified against the session's actual messages.
An attribute survives only when its verified provenance anchors to a user (visitor) message. Assistant, system, and tool turns are context for understanding the conversation — they can never be the source of a persisted value. The assistant's own descriptions, answers, and marketing copy are things the agent said, not things the visitor said.
When the visitor confirms or selects something the assistant proposed ("yes, that one"), the value is extracted with the visitor's confirming message as its source — not the assistant's proposal.
An entity with zero surviving attributes is not created, and fields the visitor never stated are omitted entirely — never stored as "" or 0 placeholders.
Values read from a visitor-uploaded document (attachment) anchor to the visitor's upload turn: the quote lives in the document, not the transcript text, so upload turns verify without a text match.

Entity Status Workflow

Extracted entities follow a status workflow for review and action:

Status	Description	Typical Action
`NEW`	Just extracted, not reviewed	Review in dashboard
`REVIEWED`	Reviewed by user	Mark as valid
`ACTIONED`	Action taken (e.g., added to CRM)	Integration completed
`ARCHIVED`	Archived/dismissed	Not relevant

Update Entity Status

bash

curl -X PATCH "http://localhost:8000/api/v1/extracted-entities/1" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "status": "REVIEWED",
    "notes": "Verified and added to CRM"
  }'

Bulk Status Update

bash

curl -X POST "http://localhost:8000/api/v1/extracted-entities/bulk-status" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "entity_ids": [1, 2, 3],
    "status": "ACTIONED"
  }'

Confidence Scoring

The LLM assigns confidence scores based on extraction quality:

Score	Meaning	Action
1.0	Information is explicitly stated	High confidence, likely accurate
0.8-0.9	Information is clearly implied	Good confidence
0.6-0.7	Information requires some inference	Review recommended
< 0.5	Low confidence	Filtered out (below threshold)

Configuration:

bash

# Minimum confidence threshold (default: 0.5)
ENTITY_EXTRACTION_MIN_CONFIDENCE=0.5

Entities below the threshold are not created.

Filtering and Search

Filter by Entity Type

bash

curl "http://localhost:8000/api/v1/extracted-entities?entity_type_id=1" \
  -H "Authorization: Bearer $TOKEN"

Filter by Agent

bash

curl "http://localhost:8000/api/v1/extracted-entities?agent_id=1" \
  -H "Authorization: Bearer $TOKEN"

Filter by Confidence

bash

curl "http://localhost:8000/api/v1/extracted-entities?min_confidence=0.8" \
  -H "Authorization: Bearer $TOKEN"

Filter by Date Range

bash

curl "http://localhost:8000/api/v1/extracted-entities?date_from=2025-01-01&date_to=2025-01-31" \
  -H "Authorization: Bearer $TOKEN"

Search (Attributes, Agent Name, Data Type Name)

bash

curl "http://localhost:8000/api/v1/extracted-entities?search=john@example.com" \
  -H "Authorization: Bearer $TOKEN"

Also matches against the entity's agent name and entity-type (data type) name — not just attribute values.

Include Archived

bash

curl "http://localhost:8000/api/v1/extracted-entities?include_archived=true" \
  -H "Authorization: Bearer $TOKEN"

Entity Statistics

Get entity extraction statistics for dashboard:

bash

curl "http://localhost:8000/api/v1/extracted-entities/stats?days=30" \
  -H "Authorization: Bearer $TOKEN"

Response:

json

{
  "total": 145,
  "period_days": 30,
  "status_counts": {
    "NEW": 45,
    "REVIEWED": 70,
    "ACTIONED": 25,
    "ARCHIVED": 5
  },
  "type_counts": {
    "Contact": 100,
    "Lead": 30,
    "Appointment": 15
  }
}

Example: Lead Collection Agent

1. Create Lead Entity Type

bash

curl -X POST "http://localhost:8000/api/v1/entity-types" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "name": "Lead",
    "description": "Sales lead information",
    "icon": "briefcase",
    "attributes": [
      {
        "name": "company_name",
        "type": "string",
        "required": true,
        "ai_hint": "Company or organization name"
      },
      {
        "name": "contact_name",
        "type": "string",
        "required": true,
        "ai_hint": "Full name of the contact person"
      },
      {
        "name": "email",
        "type": "email",
        "required": false,
        "ai_hint": "Business email address"
      },
      {
        "name": "industry",
        "type": "string",
        "required": false,
        "ai_hint": "Industry or sector"
      },
      {
        "name": "estimated_value",
        "type": "number",
        "required": false,
        "ai_hint": "Estimated deal value in dollars"
      },
      {
        "name": "pain_points",
        "type": "array",
        "items_type": "string",
        "required": false,
        "ai_hint": "List of challenges or problems mentioned"
      },
      {
        "name": "timeline",
        "type": "string",
        "required": false,
        "ai_hint": "When they need a solution (e.g., Q1 2025, urgent, 3 months)"
      }
    ]
  }'

2. Create Sales Agent with Entity Type

bash

# Create agent (or use existing)
AGENT_ID=5

# Assign Lead entity type
curl -X POST "http://localhost:8000/api/v1/agents/$AGENT_ID/entity-types/1" \
  -H "Authorization: Bearer $TOKEN"

3. Example Conversation

User: "Hi, I'm Sarah from Acme Corp. We're a software company looking for a better task management solution."

Agent: "Hello Sarah! Great to meet you. Tell me more about your current challenges with task management."

User: "Our team of 50 struggles with visibility across projects. We need something by end of Q1 and budget around $10k."

4. Automatic Extraction

After session ends, system extracts:

json

{
  "entity_type": "Lead",
  "confidence": 0.92,
  "attributes": {
    "company_name": "Acme Corp",
    "contact_name": "Sarah",
    "industry": "software",
    "estimated_value": 10000,
    "pain_points": ["visibility across projects", "team coordination"],
    "timeline": "end of Q1"
  }
}

5. Review and Export

bash

# Get new leads
curl "http://localhost:8000/api/v1/extracted-entities?entity_type_id=1&status=NEW" \
  -H "Authorization: Bearer $TOKEN"

# Export all matching leads as a CSV download (filter-aware, streamed, no
# pagination needed -- see the Entities API reference for the full column
# list, response headers, and truncation semantics)
curl "http://localhost:8000/api/v1/entities/export?entity_type_id=1&status=NEW" \
  -H "Authorization: Bearer $TOKEN" \
  -o leads.csv

# Mark as actioned
curl -X PATCH "http://localhost:8000/api/v1/extracted-entities/1" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"status": "ACTIONED", "notes": "Added to Salesforce"}'

Configuration

Environment Variables

bash

# Model to use for entity extraction (default: gpt-4o-mini for cost efficiency)
ENTITY_EXTRACTION_MODEL=gpt-4o-mini

# Temperature for extraction (0.0-1.0, lower = more consistent results)
ENTITY_EXTRACTION_TEMPERATURE=0.1

# Minimum confidence threshold (entities below this are discarded)
ENTITY_EXTRACTION_MIN_CONFIDENCE=0.5

# Extraction timeout in seconds
ENTITY_EXTRACTION_TIMEOUT=60

Best Practices

Entity Type Design

Use clear ai_hints: Help the LLM understand what to extract

json

{"ai_hint": "Full legal name including middle name if provided"}

Mark truly required fields: Only mark fields as required if essential
json
```
{"required": true}  // Only for email, name, etc.
```

Use appropriate types: Match field types to data

json

{"type": "email"}   // Not "string" for emails
{"type": "date"}    // Not "string" for dates

Provide structure for arrays: Specify what goes in the list

json

{"ai_hint": "List of specific product names mentioned, not general categories"}

Extraction Quality

Review confidence scores: Set appropriate thresholds
Test with sample conversations: Verify extraction accuracy
Iterate on ai_hints: Refine prompts based on results
Handle duplicates: System deduplicates automatically

Cross-References

Entity Types API Reference - Full API documentation
Extracted Entities API Reference - Entity management endpoints
Entity Extraction System Architecture - Internal architecture

Data Collection ​

Overview ​

Quick Start ​

1. Define an Entity Type ​

2. Assign Entity Type to Agent ​

3. Extraction Happens Automatically ​

4. Review Extracted Entities ​

Entity Type Attributes ​

Supported Attribute Types ​

Attribute Definition Format ​

Array Attributes ​

Nested Object Attributes ​

Entity Extraction Flow ​

Visitor-Only Attribution ​

Entity Status Workflow ​

Update Entity Status ​

Bulk Status Update ​

Confidence Scoring ​

Filtering and Search ​

Filter by Entity Type ​

Filter by Agent ​

Filter by Confidence ​

Filter by Date Range ​

Search (Attributes, Agent Name, Data Type Name) ​

Include Archived ​

Entity Statistics ​

Example: Lead Collection Agent ​

1. Create Lead Entity Type ​

2. Create Sales Agent with Entity Type ​

3. Example Conversation ​

4. Automatic Extraction ​

5. Review and Export ​

Configuration ​

Environment Variables ​

Best Practices ​

Entity Type Design ​

Extraction Quality ​

Cross-References ​

Data Collection

Overview

Quick Start

1. Define an Entity Type

2. Assign Entity Type to Agent

3. Extraction Happens Automatically

4. Review Extracted Entities

Entity Type Attributes

Supported Attribute Types

Attribute Definition Format

Array Attributes

Nested Object Attributes

Entity Extraction Flow

Visitor-Only Attribution

Entity Status Workflow

Update Entity Status

Bulk Status Update

Confidence Scoring

Filtering and Search

Filter by Entity Type

Filter by Agent

Filter by Confidence

Filter by Date Range

Search (Attributes, Agent Name, Data Type Name)

Include Archived

Entity Statistics

Example: Lead Collection Agent

1. Create Lead Entity Type

2. Create Sales Agent with Entity Type

3. Example Conversation

4. Automatic Extraction

5. Review and Export

Configuration

Environment Variables

Best Practices

Entity Type Design

Extraction Quality

Cross-References