Test Console & Turn Log
Conversational Agents are in Beta and available on Professional, Scale and Enterprise.
Test console
POST /api/v1/automation/agents/{agent_id}/test — scope automation:update.
Runs one real turn with your current configuration, skills and knowledge:
- Nothing is sent to any contact.
creditsshows what a live turn would cost. On a paid plan the first 500 test turns per UTC day per billing account are not billed; beyond that a test turn is charged like a live turn. During a free trial (no paid plan yet) each test turn uses that amount of the trial AI credits, and the account has an hourly and daily test-turn cap. - It works while the agent is inactive.
- The
handoffandcomplete_tasktools and goal completion actions are simulated: they are reported, not carried out. - The turn is written to the turn log with
surface: "test". - Limited to 20 turns per minute and 200 per hour per workspace.
Request
{
"message": "I want to return order 10042, the boots are too small",
"history": [
{"role": "user", "text": "Hi"},
{"role": "agent", "text": "Hi! How can I help?"}
],
"state": {},
"channel_type": "whatsapp",
"skill_slug": null,
"task": null,
"attachment_ids": [],
"config": "live"
}
| Field | Notes |
|---|---|
message | Max 4,000 characters. Required unless attachment_ids is set (then it may be empty). |
attachment_ids | Optional (Beta). Up to 5 files the test contact "sends" with the message: upload them first with POST /api/v1/attachments/upload and pass the returned ids. Only fresh standalone uploads are accepted: files uploaded in the last 24 hours that are not attached to any message. Any other id (unknown, older, already used in a conversation, or from another workspace) returns 422 {"code": "attachment_not_found", "attachment_ids": [...]}. The agent's read_images / read_documents switches and limits apply as in a live conversation. |
config | live (default) or draft: run the turn on the agent's unpublished draft. The response echoes the configuration that ran in config (live when there is no draft). |
history | Earlier turns, oldest first. Up to 50 items, role user or agent, text max 4,000 characters. |
state | The state returned by the previous test turn. Max 8 KB. Send it back unchanged to continue a conversation, including goals in progress. |
channel_type | Optional, e.g. whatsapp, live_chat, email. Max 32 characters. Lets you test channel-specific behaviour. |
skill_slug | Optional. Start with this skill loaded, as a Flow would. |
task | Optional. Simulate a Flow task: {"goal_prompt": "…", "input_variables": {…}, "output_schema": {…}, "skill_slug": "…"}. Max 16 KB; goal_prompt max 4,000 characters. See Agents in Flows. |
Response
{
"replies": [
{"text": "Thanks! I've noted order 10042. You can print your free return label at https://www.example.com/returns", "provenance": [{"type": "kb", "title": "Returns policy", "url": "https://www.example.com/returns"}], "buttons": null}
],
"outcome": "answered",
"outputs": null,
"handoff": null,
"model": "…",
"router": {"skills": ["order-return"], "language": "en", "needs_kb": true, "needs_web": false, "latency_ms": 380, "model": "…"},
"skills_loaded": [
{"slug": "shop-basics", "version": 2, "source": "always"},
{"slug": "order-return", "version": 3, "source": "router"}
],
"tool_calls": [
{"name": "search_knowledge", "status": "ok", "credits": 1, "latency_ms": 640, "results": 3}
],
"link_fixes": [
{"original": "https://www.exmaple.com/return", "action": "repaired", "replacement": "https://www.example.com/returns"}
],
"credits": {"messages": 1, "credits_per_message": 1, "tools": 1, "waived": 0, "by_charge": {"kb_lookup": 1, "bot_message": 1}, "billed": false, "would_bill": 2},
"attachments": [],
"checks": {
"grounding": {"verdict": "pass", "claims_checked": 2, "unsupported_count": 0, "llm_used": false, "latency_ms": 3},
"web": {"mode": "off", "searches": 0, "cache_hits": 0, "enforced": false},
"live": {"mode": "off", "reads": 0, "cached_reads": 0, "enforced": false, "source_ids": []},
"kb": {"searches": 1, "zero_hit": false},
"skill_overrides": []
},
"engine_trace": [{"step": "router", "…": "…"}],
"state": {},
"turn_id": "…"
}
| Field | Notes |
|---|---|
replies | The messages the agent would send: text, provenance (kb, faq or web sources) and buttons. |
outcome | See Turn outcomes. |
outputs | Task outputs when the agent completed a Flow task. |
handoff | {summary, reason, tag_id} when the agent handed over. |
model | The model that answered. |
router | The skill classifier's decision: skills it preloaded, detected language, needs_kb, needs_web, latency_ms, model. null when it did not run (for example when no model_selected skills exist). |
skills_loaded | Each loaded skill with its version and source: always (always-on, Flow skill or active goal), router (preloaded by the classifier) or tool (loaded by the agent with load_skill). |
tool_calls | Each tool call: name, status (ok, empty = searched but found nothing, error), credits, latency_ms, results (number of results for search_knowledge, web_search and read_live_source, otherwise null). Optional keys, present only when they apply: via (auto or enforced, see Enforced lookups; absent when the agent called the tool itself), cache_hit: true (a web search served from the cache), and for live reads source_id, fetcher and cached (the page was read within the source's ttl_minutes). |
link_fixes | Links changed by the link check: original, action (repaired or removed) and replacement. |
credits | messages, credits_per_message (legacy fixed-price turns only), tools, waived (messages that would not be billed), by_charge (credits per charge type; kb_lookup is at most 1 per reply), kb_searches (knowledge lookups the reply made), billed (always false here), would_bill and total (what a live turn would cost, with decimals). Turns charged by AI work also have level (fast or advanced) and split (input, cached_input, output, total, all in credits). Only AI credits are returned. |
attachments | One entry per file sent with the message (Beta), see Attachment trace. Empty when no files were sent. |
checks | What was checked on this turn, see Turn checks. |
engine_trace | Step-by-step debugging detail. Its content is not a stable contract; use it for troubleshooting only. For skill goals it contains a goal_actions_dry_run step with would_execute. |
state | Conversation state after the turn. Send it back with the next test turn. |
turn_id | The id of the turn log entry. |
sources | The sources the reply used (at most 12): kind (kb, faq or web), title, url. |
duration_ms | How long the turn took. |
goal_progress | The current skill goal with field keys only, or null. |
gate_reason | Set when AI is switched off for this agent: plan_feature:ai_agents, plan_feature:ai_features, ai_disabled_workspace, plan_no_ai or gate_unavailable. null otherwise. |
trace | The structured turn trace of this turn, the same shape the turn log stores in router_result._trace. |
Attachment trace
[
{"step": "attachment", "kind": "pdf", "filename": "invoice.pdf", "mime": "application/pdf", "status": "read", "reason": null, "pages_read": 10, "pages_total": 14, "truncated": true, "excerpt": "Invoice 2026-1043 …", "credits": 3, "summary": "ok"},
{"step": "attachment", "kind": "image", "filename": "photo4.jpg", "mime": "image/jpeg", "status": "skipped", "reason": "per_message_limit", "pages_read": null, "pages_total": null, "truncated": false, "excerpt": null, "credits": 0}
]
| Field | Notes |
|---|---|
kind | image or pdf |
status | read or skipped |
reason | Why a file was skipped: images_off, documents_off, unsupported_type (video, audio, stickers, GIFs), per_message_limit, per_conversation_limit, too_large, unreadable (damaged or password-protected), download_failed, model_failed. null when read. |
pages_read, pages_total, truncated | PDFs: pages read (max 10) and the document's page count. truncated: true when only the first pages were read. |
excerpt | PDFs: the start of the text that was read, for checking. null for images. |
credits | null: files have no fixed price. Reading a file is part of the reply's AI work, typically about 0.5 credits per image and 0.8 per PDF on Fast. Nothing is billed in the test console. |
summary | ok when the agent made its file summary, failed when it could not (it then says so instead of guessing). Read files only. |
To test the per-conversation limit, send the returned state back with the next turn: it carries the number of files already read and the ids of the files read so far, so later test turns see their summaries (as in a live conversation).
Uploads made for the test console that are not used in a conversation are deleted automatically after 24 hours. To have a file you upload yourself for testing cleaned up even if you never send it, upload it with POST /api/v1/attachments/upload?purpose=agent_test.
Hand-off rules in the test console
Keyword, max_turns and max_consecutive_fallbacks rules apply in the test console too. The turn number is taken from history, and the fallback counter travels in state, so keep sending state back.
Errors
| Status | When |
|---|---|
404 | "Agent not found" |
422 | Invalid request (lengths, sizes, slug format) |
403 | Free trial only: {"detail": {"code": "trial_ai_credits_exhausted", …}}, the trial AI credits are used up (the agent does not run) |
429 | {"detail": {"error": "rate_limit_exceeded", "message": "…", "retry_after": 42}} with a Retry-After header |
503 | Free trial only: {"detail": {"code": "trial_ai_credits_unavailable", …}}, the trial allowance could not be checked; retry shortly |
Turn log
GET /api/v1/automation/agents/{agent_id}/turns — scope automation:read.
The audit log of every turn the agent ran, newest first, including test console turns.
| Parameter | Notes |
|---|---|
conversation_id | Optional. Only turns of this conversation. |
page, page_size | page_size default 50, max 100 |
{
"items": [
{
"id": "…",
"agent_id": "…",
"bot_id": "…",
"conversation_id": "…",
"session_id": "…",
"flow_run_id": null,
"inbound_message_id": "…",
"surface": "inbox",
"model": "…",
"outcome": "answered",
"router_result": {"skills": ["order-return"], "language": "en", "needs_kb": true, "needs_web": false, "latency_ms": 380},
"skills_loaded": [{"slug": "order-return", "version": 3, "via": "router"}],
"tool_calls": [{"name": "search_knowledge", "status": "ok", "latency_ms": 640, "result_count": 3, "billed_credits": 1}],
"provenance": [{"type": "kb", "title": "Returns policy", "url": "https://www.example.com/returns"}],
"reply_message_ids": ["…"],
"handoff_reason": null,
"error": null,
"latency_ms": 2140,
"credits_billed": 2,
"credits_exact": 1.734,
"created_at": "2026-10-05T09:12:44Z"
}
],
"pagination": {…}
}
| Field | Notes |
|---|---|
surface | inbox (messaging channels), live_chat, email, flow or test |
contact_name, channel_type | The conversation's contact and channel (e.g. whatsapp). null for test console turns or when the conversation no longer exists. |
contact_hidden, conversation_key | contact_hidden is true when you cannot open this conversation in your inbox: contact_name, conversation_id, session_id, flow_run_id, message ids, provenance and error are then null, and tool_calls / router_result keep only status, timing, count and credit fields (router_result._trace keeps its codes and counts, with content and handoff.note_message_id set to null), handoff_reason shows only its code (for example keyword) and live source links are removed. Without the messages:read scope, tool_calls and router_result are reduced the same way on every turn, while the contact and conversation stay visible. On reduced turns, _trace.changes and _trace.reads keep their codes and the names of tags, fields, goals and other workspace settings; names the agent chose itself (memory keys), names from a visitor's identity token and earlier-conversation labels are null, and on hidden turns changes[].target.id is null too. The values the agent wrote (_trace.content.change_values) also need the contacts:read scope; without it they are null. conversation_key is an opaque id that groups the turns of one conversation, also when it is hidden. |
skills_loaded[].via | always_on, router, load_skill or flow |
handoff_reason | See Hand-off reasons |
error | A short error label when outcome is error |
credits_billed | credits_exact rounded to a whole number, for display. The invoice does not use it: it adds up the exact values and rounds only the monthly total. On older fixed-price turns (credits_exact is null) it is the price that was charged. Always 0 for test turns. |
credits_exact | The AI credits this turn costs, with decimals. AI credits are counted in fractions per request and only the monthly total is rounded. null on older fixed-price turns. |
tokens_in, tokens_out, internal_cost_usd | Retired, always null. AI usage is reported in AI credits, see GET /api/v1/billing/ai-usage. |
router_result._turn | The turn's checks, written on every turn (also hand-offs, fallbacks and gated turns). |
tool_calls[] | As stored: name, status, latency_ms, result_count, billed_credits, plus via (model, auto or enforced) on lookups and the optional cache_hit, source_id, fetcher, cached keys. |
The raw model conversation is never returned.
One turn
GET /api/v1/automation/agent-turns/{turn_id} — scope automation:read.
Returns one turn of any agent in your workspace, in the same shape as an item of the turn log (including contact_hidden and conversation_key). Use it when you only hold a turn id, for example the agent_turn_id of an answer event. Returns 404 when the turn does not exist.
Conversation timeline
GET /api/v1/automation/agents/{agent_id}/conversations/{conversation_id}/timeline — scopes automation:read and messages:read (both required: the response contains message text).
One conversation as the agent handled it, oldest first: the messages next to every turn of this agent in it, so you can read each customer message, what the agent decided, which sources and tools it used, and the reply it sent or the hand-off.
| Query parameter | Notes |
|---|---|
context_before | Messages shown before the agent's first turn, 0 to 20 (default 5). |
{
"conversation_id": "…",
"contact_id": "…",
"contact_name": "Jane Doe",
"channel_type": "whatsapp",
"conversation_status": "assigned",
"messages": [
{"id": "…", "direction": "inbound", "sender_kind": "contact", "sender_name": "Jane Doe",
"text": "Where is my refund?", "text_truncated": false, "message_type": "text",
"created_at": "2026-10-01T12:04:00Z", "status": "read", "attachments": [], "turn_id": "…",
"is_internal_note": false},
{"id": "…", "direction": "outbound", "sender_kind": "agent_bot", "sender_name": "Ava",
"text": "Let me connect you with a colleague.", "text_truncated": false, "message_type": "text",
"created_at": "2026-10-01T12:04:05Z", "status": "delivered", "attachments": [], "turn_id": "…",
"is_internal_note": false},
{"id": "…", "direction": "outbound", "sender_kind": "note", "sender_name": "Ava",
"text": "Customer asks for a refund on order 4711.", "text_truncated": false, "message_type": "text",
"created_at": "2026-10-01T12:04:06Z", "status": null, "attachments": [], "turn_id": "…",
"is_internal_note": true}
],
"turns": [{"id": "…", "outcome": "handoff", "…": "same fields as the turn log"}],
"truncated": false
}
| Field | Notes |
|---|---|
contact_id | The conversation's contact, for a link to the contact. null unless you also have the contacts:read scope. |
messages[].sender_kind | contact, agent_bot (a reply of this agent), bot (another bot), human (a team member), flow, system (any other outbound message, for example one sent through the API) or note (the internal note this agent wrote when it handed the conversation over; never sent to the customer). |
messages[].text | Plain text as shown in the inbox (emails: the plain-text body), at most 4,000 characters; text_truncated is then true. Internal notes and system events are not included, except the hand-off note described below. |
messages[].attachments | id, file_name and mime_type of the message's file. |
messages[].turn_id | The turn this message triggered (customer message), that sent it (agent reply) or whose hand-off wrote it (note). |
messages[].is_internal_note | true only for the hand-off note (sender_kind note). When a hand-off writes an internal note, its turn names it in router_result._trace.handoff.note_message_id; that note, and no other internal note, is shown next to the turn. |
turns | Every turn of this agent in the conversation, oldest first, with the same fields as the turn log. |
truncated | true when the window was capped: at most 300 messages and 200 turns are returned. The agent's own messages, its hand-off notes and up to 20 messages after its last turn (for example a colleague's reply after a hand-off) are always kept. |
Returns 404 when the conversation does not exist, the agent has no turns in it, or you cannot open the conversation in your inbox.
Turn checks
checks in the test console response and router_result._turn in the turn log have the same shape:
| Field | Notes |
|---|---|
grounding | Result of the grounding check: verdict (pass, rewritten, handoff, skipped, error), claims_checked, unsupported_count, llm_used, latency_ms, and reason for skipped/error (for example no_claims, timeout, not_run when no reply was written). |
web | mode (the effective web search mode of this reply, after skill overrides), searches, cache_hits, enforced. |
live | mode (the strongest live mode available on this reply, off when none), reads, cached_reads, enforced, source_ids (live sources read). |
kb | searches and zero_hit (the Knowledge Base was searched and found nothing). |
skill_overrides | Ids of loaded skills whose sources override applied. |
engine_trace adds live_read, grounding_check, web_cache_hit, enforced_regenerate and enforcement_skipped steps for troubleshooting. Tool arguments are stored redacted.
Turn trace
Every turn stores a structured, stable trace: why it ran, which skills it used, what it looked up, what it checked, what it changed and how it ended. The test console returns it as trace; the turn log and the conversation timeline return it in router_result._trace. Unlike engine_trace, these keys are a stable contract. The trace holds codes, counts, keys and ids only: never message text, field values or token counts.
{
"trace_version": 2,
"trigger": {"kind": "inbound", "detail": "inbound", "inbound_message_ids": ["…"], "flow": null},
"route": {
"mode": "router", "router_ran": true,
"skills": [{"slug": "order-return", "title": "Order returns", "version": 3, "how": "router", "score": null, "reason_code": "topic_match"}],
"candidates": [{"slug": "order-return", "title": "Order returns", "score": null}],
"reason_code": null
},
"retrieval": [{"tool": "search_knowledge", "query_kind": "rewritten", "hits": 3, "top_score": 0.21, "score_kind": "distance", "used": null, "via": "model", "status": "ok"}],
"checks": {
"fact_check": {"status": "passed", "checked": 2, "unsupported": 0, "action": "sent_unchanged", "reason": null},
"escalation": {"signal": "consecutive_fallbacks", "value": 0, "threshold": 3, "decision": "continue"}
},
"handoff": null,
"changes": [{"kind": "field_set", "target": {"type": "custom_field", "key": "order_number", "label": "Order number", "id": null}, "status": "applied", "reason": null, "source": "agent_tool", "tool_index": 1}],
"reads": [{"kind": "contact_field", "key": "first_name", "label": "First name", "source": "contact", "used": null}],
"closing": null,
"content": {"retrieval_queries": [{"tool": "search_knowledge", "original": "…", "rewritten": "…"}], "router_rationale": null, "handoff_note": null, "tool_io": [], "change_values": [{"index": 0, "new_value": "10042"}]}
}
| Key | Notes |
|---|---|
trace_version | 2 for turns written with the full trace. Older turns are rebuilt from the stored turn when you read them and carry 1; on those, changes is rebuilt from the tool calls and reads is empty. |
trigger | Why the turn ran. kind: inbound (a customer message), email, flow_node (a Flow's Run AI Assistant node), assist (an inbox assist action) or test. detail: activation, inbound, one_shot, rethink, instruct, verification or null. inbound_message_ids lists up to 20 customer messages answered together. flow is {flow_id, flow_name, node_id, node_label} for Flow turns, else null. |
route | How skills were chosen. mode: router, sticky, single_skill, always_on, load_skill, flow_task or none; router_ran says whether the skill classifier ran. skills[] has slug, title, version, how (always_on, router, sticky, load_skill, flow_task, pinned), score (always null for now) and reason_code (topic_match, follow_up, topic_switch, requested_human, no_specific_match, always_on, continuing_skill, flow_task, loaded_by_agent, pinned_for_test). reason_code on route itself is set when no skill was picked (no_specific_match or requested_human). |
retrieval[] | Up to 20 lookups: tool (search_knowledge, web_search, read_live_source, lookup_table), query_kind (original or rewritten), hits, top_score with score_kind: "distance" (lower means a closer match), via, status and, for cached results, cached. used is always null: the trace cannot tell which hit the reply relied on. |
checks.fact_check | The grounding check: status (passed, revised, failed, skipped, error, other), checked, unsupported, action (sent_unchanged, rewritten, replaced_with_safe_text, handoff, fallback) and reason. |
checks.escalation | The hand-off rule that was checked: signal (consecutive_fallbacks, max_turns, keyword, escalation_disabled, or email_confidence on email turns), value, threshold and decision (continue or handoff; send or draft for email). |
handoff | null unless the agent handed over. code, trigger (user_requested, model, fact_check, goal, flow_task, keyword, limit, no_answer, gate, other), to_human and note_message_id (the hand-off note, when one was written). |
changes[] | Up to 40 things the turn changed: kind (field_set, goal_field_saved, flow_output_set, tag_added, assigned, flow_started, webhook_sent, language_set, memory_saved, memory_cleared, verification_started), target {type, key, label, id}, status (applied, no_change, rejected, failed), reason, source (for example agent_tool, goal, handoff, language_detection) and tool_index. In the test console, goal actions carry dry_run: true. |
reads[] | Up to 60 items that were available to the agent: contact fields, custom fields, tags, earlier-conversation summaries, session memory carried over, pre-filled goal values and Flow input variables. kind, key, label, source; used is always null. |
closing | Set when the turn ended with complete_task or complete_goal: via, status (sent, withheld, failed, drafted), reason and model_text ("replaced" when the closing message was replaced by a safe text, else null). |
content | Added when you read the turn, never stored: retrieval_queries (tool, original, rewritten search text), handoff_note, tool_io (tool index, name and redacted args), change_values ([{index, new_value}] for changes, from redacted tool arguments, up to 400 characters) and router_rationale (always null). Removed on hidden turns; see Turn log for what each scope shows. |
Answer ratings & feedback
Agent replies in the inbox can be rated by your team with the same 👍 / 👎 / Wrong control and the same API as FAQ Bot replies. Each agent reply gets an AI answer event, keyed by the message id.
| Method | Path | Scope |
|---|---|---|
GET | /api/v1/conversations/{conversation_id}/ai-answers | conversations:read |
POST | /api/v1/ai-answers/{event_id}/rate | conversations:update |
DELETE | /api/v1/ai-answers/{event_id}/rate | conversations:update |
POST /api/v1/ai-answers/{event_id}/rate
{
"verdict": "wrong",
"reason_category": "factually_wrong",
"comment": "Return window is 30 days",
"better_answer": "You can return items within 30 days of delivery."
}
verdictisup,downorwrong.reason_categorysays why the answer was bad:factually_wrong,outdated,incomplete,off_topic,wrong_tone,policy_violation,hallucinatedorother. It is optional. For Wrong it defaults tofactually_wrong.- Each team member has one rating per answer. Posting again replaces it.
- Wrong always opens a knowledge base correction and sets
correction_opened: true. The correction is analysed and proposed as a new or updated FAQ entry in the Knowledge Base → Corrections queue, or applied automatically depending on your correction mode. This is the same path as FAQ Bot corrections. - Down with a
reason_categoryalways opens a correction (commentandbetter_answerare optional). A correction opened from Down is never applied automatically, whatever your correction mode; it always waits for review in the queue. - Down without a
reason_categoryis accepted and recorded as a score signal only, shown in the agent's feedback. For older clients, it still opens a correction when the comment or better answer contains real text (15 characters or more). - Error messages, hand-off notices and plan-limit replies are not answers, so they cannot be rated. A "no answer found" fallback can be rated, and marking it Wrong with the right answer reports a knowledge gap.
- Test console turns are not rateable.
- Corrections are classified as fact (FAQ fix) or policy (behaviour rule for the agent's skill). See Feedback loop & skill suggestions.
For agent replies, the answer event and each rating also carry the agent's provenance:
{
"agent_id": "…",
"agent_turn_id": "…",
"agent_context": {
"model": "…",
"model_tier": "standard",
"outcome": "answered",
"surface": "inbox",
"skills": [{"slug": "order-return", "title": "Order returns", "version": 3, "version_id": "…", "skill_id": "…", "via": "router"}],
"tools": ["search_knowledge"],
"web_used": false,
"is_test": false
}
}
On a rating, agent_context.turn_id points to the turn log entry. For FAQ Bot, suggestion and flow answers these fields are null.
Agent drafts
A draft the agent writes for your team (Assist mode, an operator request, or an email reply saved as a draft instead of sent) also gets an answer event, with surface: "agent_draft", outcome: "shown" and message_id: null. It carries the same sources and agent provenance and can be rated before it is sent. List a conversation's drafts with GET /api/v1/conversations/{conversation_id}/ai-answers?surface=agent_draft.
- When a teammate sends a reply that contains the draft (as written or edited), the same event gets the sent message's
message_idandoutcome: "sent". No second event is created, and earlier ratings move to that message. - A draft replaced by a newer draft, or not used in the reply that was sent, gets
outcome: "discarded". - Drafts never count as answers the agent sent until they are sent. In the feedback summary,
totals.answerscounts a draft only once it was sent.
Feedback summary
GET /api/v1/automation/agents/{agent_id}/feedback — scope automation:read.
| Parameter | Notes |
|---|---|
days | 1–90, default 30. Answers and ratings are kept for 90 days. |
skill | Optional skill slug. Filters series and recent. Use __none__ for answers where no skill was loaded. |
recent_limit | 1–100, default 20 |
{
"agent_id": "…",
"days": 30,
"skill": null,
"totals": {"up": 41, "down": 6, "wrong": 3, "rated": 50, "answers": 812},
"by_skill": [
{"slug": "order-return", "title": "Order returns", "versions": [2, 3], "up": 12, "down": 4, "wrong": 2, "total": 18}
],
"series": [{"date": "2026-10-05", "up": 3, "down": 1, "wrong": 0}],
"recent": [
{
"rating_id": "…",
"ai_answer_event_id": "…",
"verdict": "wrong",
"reason_category": "factually_wrong",
"comment": null,
"better_answer": "You can return items within 30 days of delivery.",
"rated_by_user_id": "…",
"rated_at": "2026-10-05T09:20:11Z",
"conversation_id": "…",
"message_id": "…",
"question": "How long do I have to return?",
"answer": "You have 14 days.",
"skills": [{"slug": "order-return", "title": "Order returns", "version": 3}],
"model_tier": "standard",
"model": "…",
"turn_id": "…",
"correction_id": "…",
"correction_status": "proposed"
}
],
"truncated": false
}
| Field | Notes |
|---|---|
totals.answers | Rateable agent answers in the window, rated or not |
by_skill | Ratings per loaded skill, sorted by down + wrong. An answer that loaded two skills counts for both. __none__ = no skill loaded. |
series | One entry per day, oldest first, including days without ratings |
recent | Latest down and wrong ratings with the question, answer, correction and the linked KB correction, if one was opened |
truncated | true when the window held more than 5,000 ratings and only the newest were counted |