Data Collection Guide

Complete walkthrough for setting up conversational data collection with Gnosari agents. Transform conversations into structured leads, feedback, bookings, and applications.

---

Overview

Goal: Configure agents to automatically extract structured data from conversations without forms or surveys.

When to use:

• Lead capture from website visitors
• Support ticket logging
• Job application collection
• Booking/appointment requests
• Customer feedback gathering
• Any scenario where you need data from conversations

What you'll learn:

1. Choose the right template and collection mode
2. Configure agent data collection
3. Monitor collected data
4. Review and take action on data

---

Before You Start

Prerequisites:

• An existing agent (or create one during this guide)
• Understanding of what data you want to collect
• Decision on how aggressive to be (passive, opportunistic, active, guided)

Quick decision tree:

I want to...	Collection Mode
Extract data silently without asking	passive
Capture mentions + ask brief follow-ups	opportunistic
Proactively ask for all required data	active
Follow a specific script/flow	guided

---

Step 1: Choose Your Template

Templates define what data to collect. You can use existing templates or create custom ones.

List Available Templates

templates = read_resource("gnosari://templates")

for template in templates["data"]:
    print(f"Template: {template['name']}")
    print(f"Fields: {len(template['fields'])}")
    for field in template['fields']:
        print(f"  - {field['name']} ({field['type']})")

Common pre-built templates:

• Lead Capture: name, email, company, interest
• Support Ticket: issue_type, description, priority, email
• Booking Request: name, email, service, preferred_time
• Candidate Profile: name, email, resume_url, experience

Create Custom Template

If no existing template fits, create your own via gnosari_create:

result = gnosari_create(
    name="Sales Qualifier",
    instructions="Qualify sales leads through conversation...",
    empty_state_title="Talk to Sales",
    empty_state_description="Tell us about your company and needs",
    data_collection=DataCollectionInput(
        name="Enterprise Lead",
        description="Capture enterprise lead qualification data",
        fields=[
            FieldInput(name="full_name", field_type="text", description="Person's full name (first and last)", required=True),
            FieldInput(name="email", field_type="email", description="Business email address", required=True),
            FieldInput(name="company", field_type="text", description="Company or organization name", required=True),
            FieldInput(name="company_size", field_type="text", description="Number of employees (1-10, 11-50, 51-200, 200+)"),
            FieldInput(name="use_case", field_type="text", description="What they want to use the product for"),
            FieldInput(name="budget", field_type="text", description="Budget range if mentioned"),
        ],
        mode="active"
    )
)

Template best practices:

• Keep to 5-10 fields maximum
• Use specific AI hints ("Person's full legal name" NOT "name")
• Mark only truly required fields as required
• Use appropriate field types for validation

---

Step 2: Configure Collection Mode

Collection modes control how aggressive the agent is about collecting data.

passive - Silent Extraction

Behavior: Extract data without asking questions. Only capture what users volunteer.

Use case: Sentiment analysis, topic tracking, passive monitoring

data_collection={
    "template_name": "Feedback Tracker",
    "fields": [
        {"name": "sentiment", "type": "text", "ai_hint": "Overall sentiment: positive, negative, neutral"},
        {"name": "topic", "type": "text", "ai_hint": "Main topic discussed"},
        {"name": "pain_points", "type": "text", "ai_hint": "Any problems mentioned"}
    ],
    "collection_mode": "passive"
}

User experience: Seamless - they don't know data is being collected.

opportunistic - Natural Follow-ups

Behavior: Capture mentions + ask brief targeted questions if key fields are missing.

Use case: Support tickets, feedback collection, general inquiry capture

data_collection={
    "template_name": "Support Ticket",
    "fields": [
        {"name": "issue_type", "type": "text", "required": True, "ai_hint": "Category of issue"},
        {"name": "description", "type": "text", "required": True, "ai_hint": "What they need help with"},
        {"name": "email", "type": "email", "required": False, "ai_hint": "Contact email for follow-up"}
    ],
    "collection_mode": "opportunistic"
}

Example conversation:

User: "The app keeps crashing."
Agent: "I'm sorry to hear that. I'll log this issue. What's your email so I can follow up?"
User: "john@example.com"
Agent: "Thanks, John. We'll look into the crashing issue and get back to you."

User experience: Mostly natural with occasional targeted questions.

active - Proactive Collection

Behavior: Proactively ask for required fields. Guide conversation toward data collection.

Use case: Lead capture, applications, registrations where data collection is primary goal

data_collection={
    "template_name": "Lead Info",
    "fields": [
        {"name": "full_name", "type": "text", "required": True, "ai_hint": "Person's full name"},
        {"name": "email", "type": "email", "required": True, "ai_hint": "Contact email"},
        {"name": "company", "type": "text", "required": False, "ai_hint": "Company name"},
        {"name": "interest", "type": "text", "required": False, "ai_hint": "Product interest"}
    ],
    "collection_mode": "active"
}

Example conversation:

Agent: "Hi! I can help you learn about our products. What's your name?"
User: "Sarah Chen"
Agent: "Great to meet you, Sarah! What email should I use to send you details?"
User: "sarah@acme.com"
Agent: "Perfect. Are you looking at this for your company?"
User: "Yes, Acme Corp"

User experience: Clear data collection intent. Structured conversation.

guided - Custom Script

Behavior: Follow a specific script/flow defined in custom_prompt.

Use case: Multi-step workflows, surveys with required order, booking with availability checks

data_collection={
    "template_name": "Booking Request",
    "fields": [
        {"name": "service", "type": "text", "required": True, "ai_hint": "Which service they want"},
        {"name": "preferred_time", "type": "text", "required": True, "ai_hint": "Preferred date and time"},
        {"name": "name", "type": "text", "required": True, "ai_hint": "Customer name"},
        {"name": "email", "type": "email", "required": True, "ai_hint": "Contact email"}
    ],
    "collection_mode": "guided",
    "custom_prompt": """After greeting, follow this flow:
1. Ask what service they need
2. Ask for their preferred date and time
3. Collect their name and email
4. Confirm all booking details before finishing"""
}

User experience: Structured interview-style conversation.

---

Step 3: Assign Template to Agent

You can configure data collection when creating or updating an agent.

During Agent Creation

Data collection is always configured at creation time — it is a required parameter of gnosari_create:

result = gnosari_create(
    name="Lead Capture Bot",
    instructions="You are a helpful assistant for our company. Help visitors learn about our products and collect their contact information.",
    empty_state_title="Talk to Us",
    empty_state_description="Learn about our products and let us know how we can help",
    data_collection=DataCollectionInput(
        name="Lead Info",
        description="Capture visitor contact details and interest",
        fields=[
            FieldInput(name="full_name", field_type="text", description="Full name", required=True),
            FieldInput(name="email", field_type="email", description="Email address", required=True),
            FieldInput(name="company", field_type="text", description="Company name"),
        ],
        mode="active"
    ),
    publish=True,
    uri="sales-bot"
)

print(f"Agent created: {result.agent.id}")
print(f"Public URL: {result.published_url}")

Update Data Collection on Existing Agent

To change data collection on an existing agent, use gnosari_manage_data_collection:

# Create a new template and assign it to the agent
gnosari_manage_data_collection(
    action="create",
    name="Support Ticket",
    description="Log support issues from conversations",
    fields=[
        FieldInput(name="issue_type", field_type="text", description="Issue category", required=True),
        FieldInput(name="description", field_type="text", description="Problem description", required=True),
    ],
    collection_mode="opportunistic",
    gnosari_id=123  # creates AND assigns in one call
)

Toggle File Uploads (Attachments)

enable_attachments is a separate per-agent switch — it lets visitors attach a photo or PDF that the agent reads directly and extracts the data you collect. Pass it with gnosari_id; no other data-collection param is needed:

# Turn file uploads on for agent 123
gnosari_manage_data_collection(
    gnosari_id=123,
    enable_attachments=True
)

# Turn them back off
gnosari_manage_data_collection(
    gnosari_id=123,
    enable_attachments=False
)

New agents always start with attachments disabled — there is no theme or account default to inherit. Omit enable_attachments (leave it None) to leave the current setting unchanged.

---

Step 4: Monitor Collected Data

After conversations start, monitor what data your agents are collecting.

Get Overview Statistics

Use gnosari_collected_data(action="stats") first to see the big picture:

# Last 30 days across all agents
stats = gnosari_collected_data(action="stats", days=30)

print(f"Total collected: {stats.total}")
print(f"\nBy Status:")
for status, count in stats.status_counts.items():
    print(f"  {status}: {count}")

print(f"\nBy Template:")
for template_stat in stats.type_counts:
    print(f"  {template_stat.template_name}: {template_stat.count}")

Output example:

Total collected: 47

By Status:
  new: 12
  reviewed: 20
  actioned: 15
  archived: 0

By Template:
  Lead Info: 32
  Support Ticket: 15

Per-Agent Statistics

# How much data did this specific agent collect?
agent_stats = gnosari_collected_data(action="stats", gnosari_id=123, days=7)

print(f"Agent collected {agent_stats.total} records this week")

List Specific Records

After reviewing stats, drill into specific data with gnosari_collected_data(action="list"):

# Get all NEW leads from last 7 days
leads = gnosari_collected_data(
    action="list",
    template_name="Lead Info",
    status="new",
    days=7
)

print(f"Found {leads.total} new leads")
for lead in leads.data:
    print(f"\nTemplate: {lead.template_name}")
    print(f"Agent: {lead.agent_name}")
    print(f"Collected: {lead.created_at}")
    print(f"Confidence: {lead.confidence}")
    print(f"Data: {lead.attributes_summary}")

Output example:

Found 3 new leads

Template: Lead Info
Agent: Sales Bot
Collected: 2026-02-14T10:23:45Z
Confidence: 0.95
Data: {'full_name': 'Sarah Chen', 'email': 'sarah@acme.com', 'company': 'Acme Corp'}

Template: Lead Info
Agent: Sales Bot
Collected: 2026-02-14T14:15:22Z
Confidence: 0.88
Data: {'full_name': 'John Smith', 'email': 'john@techco.com'}

Search Within Data

# Find all records mentioning a specific email
results = gnosari_collected_data(
    action="list",
    search="john@example.com",
    days=30
)

Filter by Multiple Criteria

# High-confidence reviewed leads from specific agent
leads = gnosari_collected_data(
    action="list",
    gnosari_id=123,
    template_name="Lead Info",
    status="reviewed",
    days=30
)

---

Step 5: Data Lifecycle Management

Collected data moves through states as you process it:

NEW → REVIEWED → ACTIONED → ARCHIVED

Understanding Statuses

Status	Meaning	When to Use
new	Just extracted, needs review	Default after extraction
reviewed	Verified accurate	After confirming data quality
actioned	Follow-up taken	After email sent, CRM entry, etc.
archived	Completed/dismissed	No longer active

Note: Status updates happen via the Gnosari UI or API. MCP tools provide read-only access.

Daily Review Workflow

# 1. Check what's new today
new_data = gnosari_collected_data(action="list", status="new", days=1)

print(f"New records to review: {new_data.total}")

# 2. Review each entry
for entry in new_data.data:
    print(f"\n{entry.template_name}: {entry.attributes_summary}")
    print(f"Confidence: {entry.confidence}")

    # Low confidence = needs manual verification
    if entry.confidence < 0.7:
        print("Low confidence - verify data")

# 3. After review in UI, check actioned status
actioned = gnosari_collected_data(action="list", status="reviewed", days=1)
print(f"\nReviewed and ready for action: {actioned.total}")

Weekly Performance Review

# Compare this week vs last week
this_week = gnosari_collected_data(action="stats", days=7)
last_week = gnosari_collected_data(action="stats", days=14)

current_total = this_week.total
previous_total = last_week.total - current_total

print(f"This week: {current_total}")
print(f"Last week: {previous_total}")
print(f"Change: {current_total - previous_total}")

---

Common Patterns

Lead Capture Agent

Goal: Collect contact information from website visitors

result = gnosari_create(
    name="Sales Assistant",
    instructions="Help visitors learn about our products. Answer questions and collect contact info naturally.",
    empty_state_title="Talk to Sales",
    empty_state_description="Learn about our products and let us know how we can help",
    data_collection=DataCollectionInput(
        name="Lead Capture",
        description="Capture contact details from interested visitors",
        fields=[
            FieldInput(name="full_name", field_type="text", description="Person's full name", required=True),
            FieldInput(name="email", field_type="email", description="Contact email", required=True),
            FieldInput(name="company", field_type="text", description="Company name"),
            FieldInput(name="interest", field_type="text", description="Product interest"),
        ],
        mode="active"
    ),
    publish=True,
    uri="sales"
)

# Add knowledge after creation — omit `type` to auto-resolve sitemap vs.
# discovery (see the Manage Resources reference)
gnosari_manage_knowledge(
    action="create",
    name="Product Docs",
    url="https://docs.example.com",
    gnosari_id=result.agent.id
)

# Monitor daily
daily_leads = gnosari_collected_data(action="list", template_name="Lead Capture", status="new", days=1)
print(f"New leads today: {daily_leads.total}")

Support Ticket Logger

Goal: Automatically log support issues from conversations

result = gnosari_create(
    name="Support Agent",
    instructions="Answer customer questions. Log any issues or feature requests mentioned.",
    empty_state_title="Support Chat",
    empty_state_description="How can we help you today?",
    data_collection=DataCollectionInput(
        name="Support Ticket",
        description="Log support issues from conversations",
        fields=[
            FieldInput(name="issue_type", field_type="text", description="Category: bug, feature request, question, billing", required=True),
            FieldInput(name="description", field_type="text", description="What the customer needs help with", required=True),
            FieldInput(name="priority", field_type="text", description="Priority if mentioned: low, medium, high, critical"),
            FieldInput(name="email", field_type="email", description="Contact email for follow-up"),
        ],
        mode="opportunistic"
    ),
    publish=True
)

gnosari_manage_knowledge(
    action="create", name="Help Center",
    url="https://help.example.com/sitemap.xml", type="sitemap",
    gnosari_id=result.agent.id
)

# Review new tickets
tickets = gnosari_collected_data(action="list", template_name="Support Ticket", status="new", days=7)
for ticket in tickets.data:
    print(f"Issue: {ticket.attributes_summary}")

Job Application Collection

Goal: Collect candidate information during screening conversations

gnosari_create(
    name="Recruiting Assistant",
    instructions="Screen candidates for open positions. Ask about their background and collect application details.",
    empty_state_title="Join Our Team",
    empty_state_description="Tell us about yourself and your experience",
    data_collection=DataCollectionInput(
        name="Candidate Profile",
        description="Capture candidate information during screening",
        fields=[
            FieldInput(name="full_name", field_type="text", description="Candidate's full name", required=True),
            FieldInput(name="email", field_type="email", description="Contact email", required=True),
            FieldInput(name="linkedin_url", field_type="text", description="LinkedIn profile URL"),
            FieldInput(name="years_experience", field_type="number", description="Years of relevant experience"),
            FieldInput(name="current_role", field_type="text", description="Current job title"),
            FieldInput(name="interest_reason", field_type="text", description="Why they're interested in the role"),
        ],
        mode="active"
    ),
    publish=True,
    uri="careers"
)

# Review candidates
candidates = gnosari_collected_data(action="list", template_name="Candidate Profile", days=30)
print(f"Applications this month: {candidates.total}")

Booking Agent

Goal: Collect appointment booking requests with specific flow

gnosari_create(
    name="Booking Agent",
    instructions="Help customers schedule appointments. Be friendly and confirm all details.",
    empty_state_title="Book an Appointment",
    empty_state_description="Tell us what service you need and when works for you",
    data_collection=DataCollectionInput(
        name="Booking Request",
        description="Capture appointment booking details",
        fields=[
            FieldInput(name="service", field_type="text", description="Which service they want to book", required=True),
            FieldInput(name="preferred_date", field_type="text", description="Preferred date for appointment", required=True),
            FieldInput(name="preferred_time", field_type="text", description="Preferred time slot", required=True),
            FieldInput(name="name", field_type="text", description="Customer name", required=True),
            FieldInput(name="phone", field_type="text", description="Contact phone number", required=True),
        ],
        mode="guided",
        custom_prompt="""Follow this booking flow:
1. Greet and ask what service they need
2. Ask for their preferred date
3. Ask for their preferred time
4. Collect name and phone number
5. Confirm all booking details"""
    ),
    publish=True
)

# Daily bookings
bookings = gnosari_collected_data(action="list", template_name="Booking Request", status="new", days=1)
for booking in bookings.data:
    print(f"Booking: {booking.attributes_summary}")

---

Best Practices

Template Design

Do:

• Use 5-10 fields maximum (focused templates work better)
• Write specific AI hints ("Person's full legal name" NOT "name")
• Mark only truly required fields as required
• Use correct field types for validation (email, number, date)

Don't:

• Create kitchen-sink templates with 20+ fields
• Use vague field names (info, data, field1)
• Skip AI hints (they significantly improve accuracy)
• Mark everything as required (users abandon)

Collection Mode Selection

Your Goal	Use This Mode
Don't interrupt the conversation	passive
Capture what's mentioned + brief follow-ups	opportunistic
Maximize data capture proactively	active
Follow a specific script/question order	guided

Confidence Score Interpretation

Score Range	Meaning	Action
0.9-1.0	Very confident	Safe to automate
0.7-0.89	Confident	Quick review recommended
0.5-0.69	Uncertain	Needs verification
0.0-0.49	Very uncertain	Manual review required

Automation thresholds:

• High-value actions (auto-email, create account): 0.9+
• Medium-value (CRM entry, notification): 0.7+
• Review queue: 0.5-0.7
• Flag for human review: <0.5

Monitoring Cadence

Frequency	What to Check	Tool
Daily	New data to review	gnosari_collected_data(action="list", status="new", days=1)
Weekly	Collection volume and trends	gnosari_collected_data(action="stats", days=7)
Monthly	Per-agent performance	gnosari_collected_data(action="stats", group_by_agent=True, days=30) — returns all agents in one call; use gnosari_id=X to drill into a single agent
Quarterly	Overall collection health	gnosari_collected_data(action="stats", days=90)

---

Common Mistakes

Mistake 1: Too Many Required Fields

Problem: Marking 10+ fields as required causes users to abandon.

Solution: Mark only critical fields as required (name, email). Let agent collect optionals naturally.

# Bad: Everything required
fields=[
    {"name": "name", "type": "text", "required": True},
    {"name": "email", "type": "email", "required": True},
    {"name": "phone", "type": "text", "required": True},
    {"name": "company", "type": "text", "required": True},
    {"name": "title", "type": "text", "required": True},
    {"name": "budget", "type": "text", "required": True}  # Too much!
]

# Good: Only essentials required
fields=[
    {"name": "name", "type": "text", "required": True},
    {"name": "email", "type": "email", "required": True},
    {"name": "phone", "type": "text", "required": False},
    {"name": "company", "type": "text", "required": False},
    {"name": "title", "type": "text", "required": False},
    {"name": "budget", "type": "text", "required": False}
]

Mistake 2: Vague AI Hints

Problem: Generic hints like "email" or "info" produce low confidence scores.

Solution: Be specific about what to extract.

# Bad: Vague hints
{"name": "email", "type": "email", "ai_hint": "email"}
{"name": "interest", "type": "text", "ai_hint": "info"}

# Good: Specific hints
{"name": "email", "type": "email", "ai_hint": "Contact email address in format user@domain.com"}
{"name": "interest", "type": "text", "ai_hint": "Which specific product or service they're interested in"}

Mistake 3: Wrong Collection Mode

Problem: Using passive mode but expecting proactive data collection.

Solution: Match mode to your goal.

# If you want the agent to ASK for data, use active:
"collection_mode": "active"

# If you want silent extraction only:
"collection_mode": "passive"

Mistake 4: Ignoring Confidence Scores

Problem: Automating on all collected data regardless of confidence.

Solution: Set thresholds and review low-confidence data.

# Filter for high-confidence data only
high_confidence_leads = gnosari_collected_data(
    action="list",
    template_name="Lead Info",
    days=7
)

for lead in high_confidence_leads.data:
    if lead.confidence >= 0.9:
        # Auto-process
        send_to_crm(lead)
    else:
        # Queue for review
        flag_for_review(lead)

---

Next Steps

Explore More:

• Agent Creation Guide - Create agents with data collection
• Data Collection Concept - Deep dive on templates and modes
• Collected Data Tools - Complete API reference
• Manage Resources Reference - gnosari_manage_knowledge auto-resolve contract

Advanced Topics:

• Multiple data collection templates per agent (max 3)
• Custom field types and validation
• Webhook integration for real-time data export
• CRM synchronization patterns