Knowledge Base for Bots

The Knowledge Base (KB) is what makes an FAQ Bot useful. Without KB content, the FAQ Bot only has its personality prompt and the LLM's general training data to draw on; with a well-curated KB it can answer specific, up-to-date questions about your product, policies, and workflows.

This page focuses on how to organize KB content for bots. For the API reference (search endpoints, document CRUD, scopes), see the Knowledge Base API guide.

How KB search works

Every KB folder is a separate Vertex AI RAG corpus. When an FAQ Bot runs a query:

1. It searches each attached folder in parallel via vector similarity.
2. The top chunks across all folders are merged, deduplicated, and reranked by distance (lower = more relevant).
3. The merged context is sent to Gemini, which generates a grounded reply citing the chunks it used.
4. A confidence score is computed from the chunks' relevance and the LLM's grounding agreement.

The FAQ Bot compares that confidence to the bot's confidence_threshold. Above threshold → reply. Below threshold → escalate (or refuse, in KB-only mode).

Folder strategy

System folders

Every tenant starts with these system folders:

Folder	Purpose
General	Catch-all for documents you upload directly.
Websites	Auto-populated by the website crawler. One folder regardless of how many domains you crawl.
Managed FAQ	The curated question-and-answer entries the FAQ Bot answers from. Generated FAQ entries land here by default. See FAQ entries and their lifecycle.
Conversation History	Auto-generated summaries of resolved conversations. They are stored for reference (on the conversation and the contact timeline) and as input for generated FAQ entries. This folder is never searched and never used to answer a customer, so attaching it to an FAQ Bot does not make it an answer source.

You can attach the General, Websites, and Managed FAQ folders (plus any custom folders) to an FAQ Bot.

note

Conversation summaries are not part of the Knowledge Base: their folders (Conversation History, and the legacy Ticket Summaries folder of older workspaces) never appear in the folder tree and cannot be attached. Curated answers live in Managed FAQ.

Custom folders — when to split

Create a custom folder when you want isolation between content sets. Common reasons:

• Per-product — separate folders for Product A and Product B so a sales FAQ Bot for Product A never accidentally cites Product B's pricing.
• Per-language — one folder per locale, attached to the language-specific FAQ Bot.
• Per-audience — internal-only knowledge (KB_internal) separate from customer-facing knowledge (KB_external).
• Per-region — region-specific policies, hours, or compliance docs.

tip

Don't over-split. Each folder is a separate corpus and a separate parallel vector search. 2–4 folders attached to a bot is fine; 20 is wasteful and slower.

Supported content types

Type	How to add	Best for
PDF	Upload via dashboard or POST /knowledge-base/documents	Product manuals, policy docs, brochures
DOCX	Upload	Internal SOPs, longer-form articles
TXT / MD	Upload or paste	Snippets, READMEs, release notes
FAQ entry (text entry)	"New entry" on the Knowledge Base → Managed FAQ page — a question and an answer	Questions where being exactly right matters: policies, pricing, procedures. FAQ entries follow a review lifecycle — see below.
Manual (text entry)	"Add Manual" button — longer free-text	How-to guides written directly in the dashboard
Website crawl	"Crawl website": supply a URL; the sitemap is detected automatically	Public marketing site, public help center

What works well

• Short, focused chunks. A 500–1500-word doc on a single topic outperforms a 50-page doc covering ten things.
• Clear titles. "Return policy for electronics" > "policy_v3_FINAL.pdf".
• Real customer questions. If you've answered the same question in support 20 times, write it as an FAQ.
• Up-to-date content. Stale info will be retrieved and confidently cited. Re-crawl after website changes (see below) and prune obsolete docs quarterly.

What works poorly

• Image-heavy PDFs (slide decks, infographics). The crawler extracts text but loses the visual context.
• Login-gated pages. The crawler runs unauthenticated; it can only see public URLs.
• Spreadsheets. Tabular data doesn't chunk well — convert key tables into Markdown manually if you need them.
• Wikis with stale redirects. Crawl scope creep. Start from the narrowest URL that covers your content and add exclude rules for the rest.
• Several languages of the same site. An answer draws on up to 5 sources. When several of them are translations of the same article, they crowd out the other pages that would have helped. Import one language (see Website crawls).

FAQ entries and their lifecycle

FAQ entries — the question-and-answer pairs in the Managed FAQ folder — are what an FAQ Bot answers from most directly, so they have a review lifecycle that plain uploaded documents do not. This matters if you drive generation, or read entry usage numbers, via the API.

The four states

Every entry is in exactly one state, and the state is the switch that decides whether the FAQ Bot can use it:

status	Retrievable?	Meaning
draft	No	Written or generated, not yet signed off. Invisible to the FAQ Bot.
published	Yes	Live — the only state that is in the RAG corpus and can be retrieved.
needs_review	No	Was live, but an agent flagged the answer or a correction demoted it. Pulled out of answering until a person looks.
archived	No	Retired, kept for the record.

Publishing and unpublishing take effect within moments. A generated entry does not answer customers until it is published — so with FAQ generation on Create drafts (the default), nothing new answers until a reviewer publishes it.

Where entries come from

An entry's origin is one of manual, auto_conversation (generated at conversation close), migrated (backfilled from ticket history), or correction. FAQ generation has three modes under Settings → AI → Knowledge base & FAQ: Off, Create drafts (default — new entries land as drafts), and Publish automatically. Every generated entry goes through a PII-scrubbing pass first, and a near-duplicate of an existing entry is folded in rather than duplicated. Bulk Import from ticket history always produces drafts, whatever the mode.

The review queue

Drafts, needs_review entries, and published entries an agent has flagged are gathered into a Review queue, together with AI corrections waiting for a decision and documents that failed to process. A reviewer works through them one at a time: publishes (approve), archives (reject), clears the flag, approves or rejects a correction, or retries a failed document. Flags are no longer permanent: editing or re-publishing an entry clears its flags automatically. To read the queue through the API, see Review queue.

Usage counters

Each entry carries three counts that are easy to confuse:

Counter	Field	What it means
Sent	times_in_sent_reply	Times the entry's answer was actually sent to a customer — including by bots, flows, and the Team Chat assistant, not only answers an agent forwarded from /suggest.
Retrieved	times_retrieved	Times the FAQ Bot pulled the entry in as a candidate while composing an answer. An entry can be retrieved often but sent rarely.
Sources	duplicate_count	How many customer questions the entry was built from, plus any near-duplicates folded into it. It is a provenance count — not a warning that duplicates exist.

Finding and merging duplicates

Two near-identical entries compete for retrieval and neither reliably wins. The Possible duplicates group on the KB health page clusters them, and you can preview exactly what merging a cluster would combine — the summed usage, every question phrasing kept as an alternate, and which copies would be retired.

note

One-click merge execution is not yet enabled in production — today the merge runs in preview only (see the merge endpoint). Until it ships, consolidate by hand: pick the best entry, copy anything useful from the others into it, and archive the rest.

The reviewer workflow (the review queue, corrections, merge and KB health) lives under Knowledge Base in the app. To drive it programmatically, see the FAQ entries API and the review queue API.

Website crawls

For a website-crawl folder:

Setting	Description
Start URL	The URL to crawl from (e.g., https://example.com/help). The crawl stays on that host and under that path, so it does not wander from the help center into your blog.
Sitemap	Detected automatically (sitemap.xml or robots.txt). With a sitemap the crawler works from its list; otherwise it follows links from the start URL.
Exclude paths	Path prefixes to leave out, matched on whole segments: /en excludes /en/pricing but not /enterprise. Up to 50.
Exclude query parameters	lang excludes every URL with ?lang=...; lang=en only that value. Up to 20.
Keep language + skip translations	Keeps one content language and skips pages that are translations of a page in that language. Pages that exist only in another language are kept.
Re-crawl	Started by you from the folder or the crawled website. Exclude rules and language settings are saved and reused by every re-crawl.

Crawls run in the background. You'll get a Slack/email notification on completion if you've set that up; otherwise the folder page shows a progress bar. Pages are imported in batches automatically. You don't manage that.

Multilingual websites

Import one language per website. The crawl dialog (and the preview endpoint) detects the languages of a site and suggests exclude rules for the others. If your main language sits at the root without a prefix (example.com in German, plus /en, /fr, /it), crawl example.com and exclude /en, /fr and /it. Your bot still answers in the customer's language. Details: Multilingual websites.

Multi-bot folder reuse

You can — and should — attach the same KB folder to multiple FAQ Bots. There's no duplication cost: the corpus is shared.

Common pattern:

• One General folder with company-wide policies.
• One Websites folder with the public help center.
• Per-product folders.

Then:

FAQ Bot	Attached folders
Support EN	General, Websites, Product-A
Sales EN	General, Websites, Product-A
Support DE	General, Websites-DE, Product-A-DE

The bot list view shows a "Used by N bots" badge per folder so you can see reuse at a glance.

Recrawl & freshness

• Documents you uploaded are static. To update them, edit and re-upload (replaces the previous version).
• Manuals are edited in place.
• FAQ entries are edited in place, but they follow a draft → published lifecycle: only published entries are retrievable. Editing a published entry re-indexes it immediately; moving it back to draft removes it from answering. See FAQ entries and their lifecycle.
• Website crawls refresh when you re-crawl them (from the folder or the crawled website). The first crawl indexes everything; re-crawls only re-import pages whose content changed, and reuse the exclude rules and language settings saved with the website.

warning

Avoid re-crawling huge sites daily unless you genuinely change them daily. Each crawl takes time to process, and every new or changed page costs AI credits (1 per 10 pages).

Cost & billing

Bot answers are billed in AI credits. Knowledge Base lookups are free. Each answer is charged by the AI work it does: what the bot reads (its instructions, the conversation and the passages it found) and what it writes.

See AI Credits for the full price list and Knowledge Base pricing.

What's next

• Build an FAQ Bot — attach KB folders to a bot.
• Prompt Engineering — tune kb_only_mode and confidence_threshold.
• Knowledge Base API reference — programmatic CRUD and search endpoints.